Received: (at 61386-done) by debbugs.gnu.org; 25 Feb 2023 12:27:21 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 25 07:27:21 2023 Received: from localhost ([127.0.0.1]:39318 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pVte5-0003FA-Cm for submit <at> debbugs.gnu.org; Sat, 25 Feb 2023 07:27:21 -0500 Received: from mail-wr1-f42.google.com ([209.85.221.42]:34576) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pVte3-0003Er-U1 for 61386-done <at> debbugs.gnu.org; Sat, 25 Feb 2023 07:27:20 -0500 Received: by mail-wr1-f42.google.com with SMTP id r18so1815778wrx.1 for <61386-done <at> debbugs.gnu.org>; Sat, 25 Feb 2023 04:27:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=NMt+wgNHjW2ugFKgEozLDxgr6uX4iN5efq+LF34nvPw=; b=d3LA3aSIkdtWfeI4R95Sn0td9CLmZFhqJg5+2FZmIPILVvXSgun2BUBkmqksCL7AGE zReyZYtfPjdcACp9E6pSK0+denVUJhvMah7yDJdbDRCEKo74Lg/5uTkaNoKoz7YHLuFb 6HSGjVaFi3qDGzvDDEnwRhESrZatJPd4CdOvzLBU8h50Vt0HgbemimeLDIh+Ewssy7gg d0Ajz0tBY4XorQ/q4zFGiozfWIikDzK5ekAENpbLddsuGMtZY029Pkd26rIF5WFPeoDZ HSjRN0sudyKLRBNEWhjYgvRVruBNj0kGx2qErdwnla1squNS+AWU/KLX5iJA2LcyL9Im Hs/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=NMt+wgNHjW2ugFKgEozLDxgr6uX4iN5efq+LF34nvPw=; b=cUM3ESAJSTrMl28h1Sg5dOnrb35oUsrq0TKDICS0lSGZFIcSUBzOjBqBpG/JanS8E8 qPh4H0EmFup5y6Kmryd5daFMkVim0nrzZo1exDHNHrFn7WQQyG4HKvNHVvmP803HMjIC LT6foX7XOTunnasDtdtAMtLBz6PxUnfmEkidSNaB6VEb0As7NVqyvyYBuE+c2bUEKi9i JwtrNCN1RtM5Mn7m1d7vL7+q283eufpY8yj8hxCQKhvTTg7Q1hAD1VyO7oMCXdGcjNkN ffwCKRSiVQBfWamoxqW8C/UV1h5mMi4Adn92auqo+fgsUqvCeqDj/KgCMntzExvz1qKZ Hz0w== X-Gm-Message-State: AO0yUKWFOqR+Eb5NkTuSRmnUNS0J01uImcR6YKUdpf1sHCPAo6/iwO/3 Icr40xGmCbiZzMK9u5DYM18= X-Google-Smtp-Source: AK7set+ws7/KO1xINAvbVvClima7aj9sl9KvRmegOgIRP1IzrAgmkpPC8aFb3er58d0NTfZ5zz3UmA== X-Received: by 2002:a5d:65c6:0:b0:2c5:7c7a:78f7 with SMTP id e6-20020a5d65c6000000b002c57c7a78f7mr15337973wrw.45.1677328033859; Sat, 25 Feb 2023 04:27:13 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id f12-20020a05600c154c00b003eb2e33f327sm3496586wmg.2.2023.02.25.04.27.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 25 Feb 2023 04:27:13 -0800 (PST) Message-ID: <45cde6e0-cbdc-47b4-8f5b-7746b31c9965@HIDDEN> Date: Sat, 25 Feb 2023 12:27:11 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> <B2E79E10-EF8A-4C97-8651-C418B0394C0C@HIDDEN> <32a03b65-ae98-7417-bcc7-e89836191a35@HIDDEN> <A6564408-3B97-4632-BBBC-916B16DF1CD1@HIDDEN> <0f15c455-c125-d80e-2e38-56a2eb0f429e@HIDDEN> <7BE36D87-17A4-46B2-BDA1-2280689A86FE@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <7BE36D87-17A4-46B2-BDA1-2280689A86FE@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386-done Cc: Paul Eggert <eggert@HIDDEN>, Gnulib bugs <bug-gnulib@HIDDEN>, 61386-done <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 24/02/2023 23:23, George Valkov wrote: > Are we in a state where I can update OpenWRT to the latest coreutils master or perhaps it would be better to wait until you fix the tests and coreutils-9.2 is released? You mentioned the release is coming soon? Probably best to wait for the 9.2 release at this stage. Best guess is 1.5 weeks away. cheers, Pádraig.
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386-done) by debbugs.gnu.org; 24 Feb 2023 23:24:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 24 18:24:03 2023 Received: from localhost ([127.0.0.1]:38567 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pVhQ3-00083t-7u for submit <at> debbugs.gnu.org; Fri, 24 Feb 2023 18:24:03 -0500 Received: from mail-wm1-f49.google.com ([209.85.128.49]:45946) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pVhQ1-00083N-Nh for 61386-done <at> debbugs.gnu.org; Fri, 24 Feb 2023 18:24:02 -0500 Received: by mail-wm1-f49.google.com with SMTP id d41-20020a05600c4c2900b003e9e066550fso499563wmp.4 for <61386-done <at> debbugs.gnu.org>; Fri, 24 Feb 2023 15:24:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1677281036; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Qxz6rELuNzthuT7yWffiIXodDbRrWvnD805/eIx7OYE=; b=M7XzTg5Cidcr/c+Dz5Yl3AIArRtTUf2dx2c4LWmLjsR+okfXbhdhxJMWbffIkDBM23 U/UBe/hoXyCu87KvTK9IGvpA6YfCmHAxCzD8RDnivieR+xglzQXJmK5waVkY/Iy0+ovW dB+gnjbtGFW0p2ZS2Mp1g9WegCj9pGINTxwfYho/gACLLgyLkIny7y6WJfhzJNqgYpq1 tBmaHTzbg6UvImIfp5Ue63gkOAb7Y+HPP403N8UUMnkw1lz5c/B0/n4n+n4NsTHIHMjq btZIDrX6weIn/Wx4xkmaRSNdvdQw8fiu4DCBaYI72VQKzs7hPe0ircBWNBJ5Byx48Hpc Fi0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677281036; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qxz6rELuNzthuT7yWffiIXodDbRrWvnD805/eIx7OYE=; b=jb2tN9fhkg7txN3OrD5QDABXWmZ9rZwF0tyPCv1WFKL6+GnY0rNkGWZrRa7eypSNRc xVkVFZHLe5PLkf6fnnu92Jp1/8yVQu/Tj4vmlaFgrcbI2YcMsPTptmWq8dOXMsl9pISV EUJgXMG7BnXhoW6pY7lGdNwrL2MnMLaA8FB5j+Vqi+3RF1h/d3eWYGXfyKTKm9sl+Uc4 8I3tfDez2gEZdFVWPPb8YYL4YuZN33DZlYLELWy8tNsVs+7sxmSi2gnNB2J4vdhmrHti hQotJF12MZTPERu/NolOWy6QoBjwNdM38KChKBT/fN53moRh7WGhInF2nwP5yyKXltWT sZ0Q== X-Gm-Message-State: AO0yUKV8LVzbcA65elMLo+XxpItuT//iDbIMxWD+pWwhXhlZtmWiSiRF RRcvqqBmUUT0cbVB3i7Mq9k= X-Google-Smtp-Source: AK7set85qBjJcVsItQzZzsqAtEELSVNf9o8iXalWr9wnB75lXr6a5krHpCEwpqKof2b3vzc5dzJ/4w== X-Received: by 2002:a05:600c:5112:b0:3e8:7ca3:8424 with SMTP id o18-20020a05600c511200b003e87ca38424mr11616293wms.1.1677281035505; Fri, 24 Feb 2023 15:23:55 -0800 (PST) Received: from smtpclient.apple (212-39-89-230.ip.btc-net.bg. [212.39.89.230]) by smtp.gmail.com with ESMTPSA id fc19-20020a05600c525300b003e215a796fasm694679wmb.34.2023.02.24.15.23.53 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Feb 2023 15:23:54 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <0f15c455-c125-d80e-2e38-56a2eb0f429e@HIDDEN> Date: Sat, 25 Feb 2023 01:23:52 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <7BE36D87-17A4-46B2-BDA1-2280689A86FE@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> <B2E79E10-EF8A-4C97-8651-C418B0394C0C@HIDDEN> <32a03b65-ae98-7417-bcc7-e89836191a35@HIDDEN> <A6564408-3B97-4632-BBBC-916B16DF1CD1@HIDDEN> <0f15c455-c125-d80e-2e38-56a2eb0f429e@HIDDEN> To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386-done Cc: Paul Eggert <eggert@HIDDEN>, Gnulib bugs <bug-gnulib@HIDDEN>, 61386-done <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On 2023-02-25, at 12:47 AM, P=C3=A1draig Brady <P@HIDDEN> = wrote: >=20 > On 24/02/2023 22:06, George Valkov wrote: >> If I revert a0803c4bad6f8e11bb05effcc42ef433f4fc3f9b, the requirement = to press enter after PASS: tests/rm/isatty.sh is fixed. >=20 > Ah very useful info. > I'll test this on my macOS 13.2 system. >=20 >> Sometimes it might randomly hang: gdb after >> PASS: tests/rm/r-4.sh >> ^Cmake[1]: *** Deleting file 'tests/rm/r-root.log' >> ^C^C^C >=20 > This is the same gdb issue, which I'll skip similarly > to the tail-2/inotify tests. >=20 >=20 > Since all issues related to sparse copy on macOS are now addressed, > I'm marking this bug as done. >=20 > The above bugs are unrelated, and I'll take them from here. >=20 > thanks again for all the testing. Thank you P=C3=A1draig! You and Paul did a lot of work, and I=E2=80=99m = happy the issue is resolved. I=E2=80=99ll let you know if I get any = update from Apple. Are we in a state where I can update OpenWRT to the latest coreutils = master or perhaps it would be better to wait until you fix the tests and = coreutils-9.2 is released? You mentioned the release is coming soon? Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386-done) by debbugs.gnu.org; 24 Feb 2023 22:48:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 24 17:48:01 2023 Received: from localhost ([127.0.0.1]:38498 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pVgrB-0004bM-1x for submit <at> debbugs.gnu.org; Fri, 24 Feb 2023 17:48:01 -0500 Received: from mail-wr1-f51.google.com ([209.85.221.51]:44901) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pVgr9-0004ac-GY for 61386-done <at> debbugs.gnu.org; Fri, 24 Feb 2023 17:47:59 -0500 Received: by mail-wr1-f51.google.com with SMTP id 6so711240wrb.11 for <61386-done <at> debbugs.gnu.org>; Fri, 24 Feb 2023 14:47:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=tE98CoC79dHQpfoIxDV0jxl7Db3sjEs1DN4skzAA/Ks=; b=XRGz8tb+KowT/KAuS3Pubkmw3AoGitDiYxHPdeLathBv20FUkt+6nBE2v9GizyGlEc gHUaLi1Ntz7seDxFqX30OdMfpLrCPayBrDwCSy4fqThbWjX6tPuB742Sww7QOpT3JbK8 iQLg7Nq6ZsjZm5JzDNlGd1jV1otGAzlG8ZZKAbrL1ZepMyiRWOiGb3429ShewciNmMXs jmAzJbtP0ndSnUMhLCfXln+5f0WgrS+idF/KYha5aHOoM2hzGP1DUeJoxfGPxzeWRknM 0cJlqVghkGLB7LYlXn36S5f6UFdCugsBDGxnH2CjHPlsBYc5jKXekyt/dcDWVehWUOHy g1XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tE98CoC79dHQpfoIxDV0jxl7Db3sjEs1DN4skzAA/Ks=; b=v88I7Tyog3/fE3DE3oO/yozObfQgmRpFxN7rrqNPFFPm3P9nYxQos74jawBCpl2B7d fq3Od1XOpGwSTveecNhCjqfg4cr5RF9/0QF8ClVt16JszIg6fIa8XEDNX2p41KTmtiHd +ICMhdyGYk23lWNbKRxLOdbU2ovavxLSMzvhs5Q6YP0K3mbvgyU/nIvKvNFpeKJ7+mRJ 1doIZvFcOjW1ky+eCN08cikuHE9d2cOMqbKV3NvAZhlRdr3NxS5q7OVJ+ZT865UugxCd kyZ/+kpj+mNKhJ9Wmk/9fvBJvPFPqWSzi7SdAnPAxL5SNOFstxu0ubTPAxZebfoIdaUQ YTKQ== X-Gm-Message-State: AO0yUKUMLKXiWPURJ2UFUWQ09sXETBbz0NfbEkxilqEwi6nggWYN3hmT lYS7UWGu/w6vaa6qI6TdOpA= X-Google-Smtp-Source: AK7set9c/JYFG+DCtkPx+Tmquj0h1WXJ5eDqCmmfqYx61qaaqd9VFQ4mVYNOF1D2nAbQlbZK3KKQsA== X-Received: by 2002:a05:6000:10c8:b0:2c5:8505:be46 with SMTP id b8-20020a05600010c800b002c58505be46mr14305298wrx.24.1677278873491; Fri, 24 Feb 2023 14:47:53 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id p20-20020a05600c205400b003daf6e3bc2fsm6851559wmg.1.2023.02.24.14.47.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 24 Feb 2023 14:47:52 -0800 (PST) Message-ID: <0f15c455-c125-d80e-2e38-56a2eb0f429e@HIDDEN> Date: Fri, 24 Feb 2023 22:47:52 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> <B2E79E10-EF8A-4C97-8651-C418B0394C0C@HIDDEN> <32a03b65-ae98-7417-bcc7-e89836191a35@HIDDEN> <A6564408-3B97-4632-BBBC-916B16DF1CD1@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <A6564408-3B97-4632-BBBC-916B16DF1CD1@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386-done Cc: Paul Eggert <eggert@HIDDEN>, Gnulib bugs <bug-gnulib@HIDDEN>, 61386-done <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 24/02/2023 22:06, George Valkov wrote: > If I revert a0803c4bad6f8e11bb05effcc42ef433f4fc3f9b, the requirement to press enter after PASS: tests/rm/isatty.sh is fixed. > Ah very useful info. I'll test this on my macOS 13.2 system. > Sometimes it might randomly hang: gdb after > PASS: tests/rm/r-4.sh > ^Cmake[1]: *** Deleting file 'tests/rm/r-root.log' > ^C^C^C This is the same gdb issue, which I'll skip similarly to the tail-2/inotify tests. Since all issues related to sparse copy on macOS are now addressed, I'm marking this bug as done. The above bugs are unrelated, and I'll take them from here. thanks again for all the testing. Pádraig
George Valkov <gvalkov@HIDDEN>:Pádraig Brady <P@HIDDEN>:Received: (at 61386) by debbugs.gnu.org; 24 Feb 2023 22:06:49 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 24 17:06:49 2023 Received: from localhost ([127.0.0.1]:38409 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pVgDJ-0003XL-CS for submit <at> debbugs.gnu.org; Fri, 24 Feb 2023 17:06:49 -0500 Received: from mail-ed1-f49.google.com ([209.85.208.49]:34546) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pVgDH-0003X7-GB for 61386 <at> debbugs.gnu.org; Fri, 24 Feb 2023 17:06:48 -0500 Received: by mail-ed1-f49.google.com with SMTP id cq23so3030043edb.1 for <61386 <at> debbugs.gnu.org>; Fri, 24 Feb 2023 14:06:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1677276401; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=WzNIAEHCw1OPr/tHJ/1hDkPGyZrYVwjfAOwck0C3sqM=; b=fYmcxsW+mgXdzKntGES+0oYEoUc4HV8ZB/fIsjQG2NZt8sEiRU/69EIeLhySucfNgT K2RigW4a1QnXmM3kqawSX3Fvap35SbwdtpC/Y3kHg0ZIw64zuZk8364gLMie0+7JZGk5 Qy/1KKtmehTc/omvIYEpSXMPAWFKe5Jb+SXtntIu1agk66xhUZxKPAmVg+uGEBprrLCF nDqUvoXFChrwn2rFm9GX6d7M5q5slOGeJpuuO5ZPZNQJrAIstmVwWtDpQe/dvLm0zfDp nCts/b/EVOj+5j3A/6JoS1Ttu7ojNRLxAusHv3sTPMNoTFigasUEBv1r0V7P+AQFWXpt wU6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677276401; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WzNIAEHCw1OPr/tHJ/1hDkPGyZrYVwjfAOwck0C3sqM=; b=bq7gMGAVZS2geDHyXkrIAQ2Z0h4Bd+gM/9zhvpI5LuDaDYaKanNacy73Jeb8z4NVx4 Ph5qgOyWjEuKvYUkADLuJcA4JT8uIPS61cI4Z1oMe2Qk0NZQfyOYvXZ1pjkSAsI+Mj95 E5oYhxLLeBmiJdeCaotuvQynSydo299uAkDR9zmrYpVnwTt7WSUynEgVWxzZ4s9/dZ9B otECc+SNtAxmpT96WdVf+ZWE7KktTSb4mYdqn/TtjJp3krFg5ePhupolXIjbBgmuK2he 6OpJZPX2BvyQXPQ3Jslqy/z0FmQwDuImWeRbQnCRss9AyUMVLKBbz3Lxf3EjJiTd0bg8 CS/w== X-Gm-Message-State: AO0yUKVeWY0B7/OvooHjHJTsk/X0jNeDKrzSEE7naNHbExpGlc2rG/lr Yp7OrIUnS8XVrMYHVt4swnU= X-Google-Smtp-Source: AK7set+ttErE92tiuIQTgrgcG9s9EOEA8Yt70ZGmHvvU2jd02mBAWVpqY2OuPP+vD6K2srfEB0BxIw== X-Received: by 2002:a17:906:10a:b0:8b1:78bc:3103 with SMTP id 10-20020a170906010a00b008b178bc3103mr27402237eje.7.1677276401332; Fri, 24 Feb 2023 14:06:41 -0800 (PST) Received: from smtpclient.apple (212-5-158-230.ip.btc-net.bg. [212.5.158.230]) by smtp.gmail.com with ESMTPSA id e6-20020a170906248600b008e53874f8d8sm42636ejb.180.2023.02.24.14.06.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Feb 2023 14:06:40 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <32a03b65-ae98-7417-bcc7-e89836191a35@HIDDEN> Date: Sat, 25 Feb 2023 00:06:38 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <A6564408-3B97-4632-BBBC-916B16DF1CD1@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> <B2E79E10-EF8A-4C97-8651-C418B0394C0C@HIDDEN> <32a03b65-ae98-7417-bcc7-e89836191a35@HIDDEN> To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386 Cc: Paul Eggert <eggert@HIDDEN>, Gnulib bugs <bug-gnulib@HIDDEN>, 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) If I revert a0803c4bad6f8e11bb05effcc42ef433f4fc3f9b, the requirement to = press enter after PASS: tests/rm/isatty.sh is fixed. Sometimes it might randomly hang: gdb after PASS: tests/rm/r-4.sh ^Cmake[1]: *** Deleting file 'tests/rm/r-root.log' ^C^C^C ps -A |grep gdb 55051 ttys020 0:00.09 gdb -nx --batch-silent -return-child-result = --eval-command=3Dset exec-wrapper\011\011\011\011\011 = env 'LD_PRELOAD=3D:./k.so' --eval-command=3Dbreak = '/Volumes/coreutils/coreutils-2023-02-24.d/src/remove.c:377' = --eval-command=3Dsource bp.py --eval-command=3Drun -rv --one-file-system = dir --eval-command=3Dquit rm even though I have applied macos-gdb-hang.patch. Other times it goes = past that: r-root.sh: skipped test: internal test failure: maybe LD_PRELOAD or gdb = doesn't work? and eventually the tests complete. If it hangs and I killall -9 gdb, it continues r-root.sh: skipped test: internal test failure: maybe LD_PRELOAD or gdb = doesn't work? SKIP: tests/rm/r-root.sh PASS: tests/rm/readdir-bug.sh =E2=80=A6 Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 24 Feb 2023 19:34:13 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 24 14:34:13 2023 Received: from localhost ([127.0.0.1]:38275 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pVdpc-0007tO-RF for submit <at> debbugs.gnu.org; Fri, 24 Feb 2023 14:34:13 -0500 Received: from mail-wm1-f44.google.com ([209.85.128.44]:43958) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pVdpb-0007t9-9u for 61386 <at> debbugs.gnu.org; Fri, 24 Feb 2023 14:34:12 -0500 Received: by mail-wm1-f44.google.com with SMTP id l7-20020a05600c4f0700b003e79fa98ce1so149741wmq.2 for <61386 <at> debbugs.gnu.org>; Fri, 24 Feb 2023 11:34:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=TA6MrxA3fZBeJ9n1eBHDFe/tarx1qCfRwkII4A85MFk=; b=iFQu1gt6d3VGmy+xgyDjKBqh/hkn27KEPzocZw0xgYahBAHL+wUPfDQ3oIp3fqNcx9 syGL9i6leqb1QQ4G/WLr96h8FWxxnE+ljFARuL5ld412g+OhBnBZhqkMUjMj1mF2NXKD Ors1lZm4w/3rZ3B7YooVMlkPqbtQXoRrHH4JtBlOgxAkQElOpSAIebRxxZdIY+ALyaZD cV8B/NXYH+JdzHtXvcegNmwbu/DcQFnUUnWl6qBEH47QEJgVMU4CioW4b+TBSHcq3nWu wsOdBdB3CPGrK/CH8S2dgY4jeUu9YqkVD30IWInfxhFlNj+kfxtjb+Pr2JwZTIaEW+U7 zOug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TA6MrxA3fZBeJ9n1eBHDFe/tarx1qCfRwkII4A85MFk=; b=PjKZgGB/qvy31fj9jFvrFcj/Snnvmg+fA2URrODtFBndk+XU62Tw7mPnYIKLN3gPxM 5QsyI4pRRBjOYLBlW95nhxmgHEtPlK12vmTxBOwnwp/6Ke+G+H4RhMt1zRR4EfU3bSx3 cbGBnK65mSFtK+bz5F6DnOeWP/t/YGnIEdV+Bfi+CWJSbc/2syP0br8VD54T5bPmz+zb K51p+le4fAE6D08Xem/f6MRl/HcuCKjzB5q/Hv6Xzeff4ENp477durJLgzAKoI6yz5Kx LhU0ix+TQqj9kjnSSyt1YFr3EYynwmo4rw1wkGzIzH6pYQ2NYMDSbi1M9Ls3Zd1qf+Uh 3+2g== X-Gm-Message-State: AO0yUKXVKFx45onG+p+2p2EQ477f6TU8woOhGFBzevL13U/GCubp4hBE X9shSY7LChwWS32EyD2M+wo= X-Google-Smtp-Source: AK7set/fdpShsAP7hwqkS16/If9Kz6fqmn8BJMTf0oAVAYq600AW0yJpCfbcp3pdLyOX63rYEFvzyQ== X-Received: by 2002:a7b:ce89:0:b0:3e1:bfc:f57b with SMTP id q9-20020a7bce89000000b003e10bfcf57bmr12866414wmj.2.1677267245310; Fri, 24 Feb 2023 11:34:05 -0800 (PST) Received: from smtpclient.apple (212-39-89-230.ip.btc-net.bg. [212.39.89.230]) by smtp.gmail.com with ESMTPSA id p11-20020a5d458b000000b002c707b336c9sm9815297wrq.36.2023.02.24.11.34.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Feb 2023 11:34:04 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <32a03b65-ae98-7417-bcc7-e89836191a35@HIDDEN> Date: Fri, 24 Feb 2023 21:34:02 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <4AB3B99F-2390-4BAA-B465-2C936A00A304@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> <B2E79E10-EF8A-4C97-8651-C418B0394C0C@HIDDEN> <32a03b65-ae98-7417-bcc7-e89836191a35@HIDDEN> To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386 Cc: Paul Eggert <eggert@HIDDEN>, Gnulib bugs <bug-gnulib@HIDDEN>, 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On 2023-02-24, at 5:43 PM, P=C3=A1draig Brady <P@HIDDEN> = wrote: >=20 > On 24/02/2023 14:33, George Valkov wrote: >>> On 2023-02-24, at 12:23 AM, Paul Eggert <eggert@HIDDEN> wrote: >>>=20 >>> On 2/20/23 13:14, P=C3=A1draig Brady wrote: >>>> Since https://github.com/coreutils/gnulib/commit/4db8db34 >>>> gnulib has been unconditionally replacing lseek() on macos. >>>> Now with SEEK_DATA undefined that replaced lseek() >>>> will run the code intended for BeOS. >>>> So either the lseek.m4 or lseek.c needs adjusting accordingly. >>>=20 >>> Good catch, thanks. I updated the Gnulib patch accordingly; see = attached. >> Nice: I just downloaded a fresh copy of Savannah, and it already = includes the attached patch, so no action needed on my side. >> The copy created by cp is good. I noticed that you have added a = =E2=80=94debug switch, so I gave it a test: >> 1. If the target exists I get this output. I would assume zeroes = means that all data is read from the source, but pages containing only = zeroes are skipped, while pages containing data are written to the = target, which is fine. I would guess 4 KB page granularity or some form = of detection takes place. >=20 > Exactly. >=20 >> ./coreutils-2023-02-24/src/cp --debug cc1-mmap cc1-ori >> 'cc1-mmap' -> 'cc1-ori' >> copy offload: avoided, reflink: unsupported, sparse detection: zeros >> 2. If the target does not exist, the file is cloned. You might want = to report something about that in the debug output. Otherwise the clone = is good. >> ./coreutils-2023-02-24/src/cp --debug cc1-mmap cc1-ori >> 'cc1-mmap' -> 'cc1-ori=E2=80=99 >=20 > Yes that was a mistake. > Should be fixed with > = https://git.sv.gnu.org/gitweb/?p=3Dcoreutils.git;a=3Dcommitdiff;h=3D65bb27= 656 Looks good commit 65bb2765646df18488b266e6c1851593d3f9c966 ./coreutils-2023-02-24.b/src/cp --debug cc1-mmap cc1-ori=20 'cc1-mmap' -> 'cc1-ori' copy offload: unknown, reflink: yes, sparse detection: unknown >> My first attempt to run the tests stopped here, so I had to interrupt = it 10 minutes later with Control+C. >=20 >> ^Cmake[1]: *** Deleting file 'tests/tail-2/inotify-race.log' >> ^C^C^C^C^C^C >> killall gdb >> ps -A |grep gdb >> 29584 ?? 0:00.09 gdb -nx --batch-silent --eval-command=3Dbreak = 1475 --eval-command=3Drun --pid=3D29583 -f file --eval-command=3Dquit = tail >> 23269 ttys010 0:00.09 gdb -nx --batch-silent --eval-command=3Dbreak = 1475 --eval-command=3Drun --pid=3D23268 -f file --eval-command=3Dquit = tail >> killall -9 gdb >> ps -A |grep gdb >> Killing gdb allowed the tests to continue, I had to do it twice: >=20 > That's awkward. > That test documents the issue with protecting the gdb invocation with = timeout(1). > I suppose we could restrict this test to inotify capable systems, > which would have the side effect of avoiding its use on non linux = systems. > The attached patch does that. Still hanging out there after PASS: tests/rm/isatty.sh There were no gdb instances to kill, but if I press enter, it continues empty-inacc.sh: set-up failure:=20 ERROR: tests/rm/empty-inacc.sh PASS: tests/rm/empty-name.pl =E2=80=A6 = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-07= -65bb2765646df18488b266e6c1851593d3f9c966-patch.txt coreutils: git checkout cf80f988eeb97cc3f8c65ae58e735d36f865277b gnulib: git checkout 32c16c45d7378b014d9aac6130104c4d02a9acdb ./bootstrap && ./configure && make clean && make -j 16 make check-TESTS # still hangs: gdb git checkout -b cf80-macos-gdb-hang git am < macos-gdb-hang.patch make clean && make -j 16 make check-TESTS # completes successfully Clone and configure another fresh copy; gnulib at master; test various = coreutils commits, applying macos-gdb-hang.patch on top of them; make = clean && make -j 16 cf80f988eeb97cc3f8c65ae58e735d36f865277b hangs: gdb I would suspect either some change in gnulib or some of the other = repositories is causing these. I cannot checkout random gnulib commits, = since the build breaks completely. Check various commits of gnulib gnulib 7352d9fec59398c76c3bb8a2ef86ba58818f0342 master hangs: ENTER gnulib bb3fd10e6309f017618a12b2c10d3bfb813bfc08 hangs: ENTER gnulib f77a31de60963c994cd9b42c8088be0e734962d7 aclocal-1.16: error: = aclocal: file 'm4/build-to-host.m4' does not exist Trying to revert some commits and go back in time: git revert f77a31de60963c994cd9b42c8088be0e734962d7 fails git revert 1e29238e40d118d4f769f7516700dd4fc494bfcd fails > thanks for all the testing. Look P=C3=A1draig, I=E2=80=99m glad to help, but this is really taking a = lot of energy, these tests took another full day, and I=E2=80=99d be = thankful if we can make everything work sooner. I=E2=80=99ve been = spending many hours each day for a few weeks now. I need to finish my = own tasks and find a job. In the old build directory if I ./bootstrap && ./configure and make -j = 16, the tests complete. It is using these checkpoints: coreutils: git checkout cf80f988eeb97cc3f8c65ae58e735d36f865277b gnulib: git checkout 32c16c45d7378b014d9aac6130104c4d02a9acdb However if I clone a fresh copy ./bootstrap && ./configure, then check = the same commits, ./configure again and make -j 16, I need to also apply = your gdb patch, otherwise it hangs. And on master I need your patch and = I need to press enter after PASS: tests/rm/isatty.sh. Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 24 Feb 2023 15:43:49 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 24 10:43:49 2023 Received: from localhost ([127.0.0.1]:37958 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pVaEf-0001Za-65 for submit <at> debbugs.gnu.org; Fri, 24 Feb 2023 10:43:49 -0500 Received: from mail-wm1-f48.google.com ([209.85.128.48]:46819) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pVaEc-0001ZM-VY for 61386 <at> debbugs.gnu.org; Fri, 24 Feb 2023 10:43:47 -0500 Received: by mail-wm1-f48.google.com with SMTP id m14-20020a7bce0e000000b003e00c739ce4so1981600wmc.5 for <61386 <at> debbugs.gnu.org>; Fri, 24 Feb 2023 07:43:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:sender:from:to:cc:subject :date:message-id:reply-to; bh=2ukjsCcPdiUN2qx1LoopIVt5rrKpX6zrzPS5FQbDQSM=; b=b3jvInDWzJ0STFjbQl0kJCm2bWcE6Z5W/9QiWERIulbM4FyMBlParjQn1e9ZuzpAl+ Tu5MV3TvnTYkB4oKT5tjRVoMJ5+l5wRdhdGQC5C77I6Gof4qJ2Z1a0H+6aupAH0zVqAR X9YKs0+oDykF+gFB0ZNpfjDy8+93MSFNVPdr95vldA80c8HVNChrTyEA3CUxE21W5ANR wnAVCwIL7xPzecMSIYvlRrY18fiNHQQZYt089s5vLvljFCzNQzgazKc+ZGJxeTaRZlEa bh3vT8/1yaT3yVFvdgZWZMOmc8zPWS3e/+K/w3AEkgzJPJtNaFk3P+eM+YS1X5jTtHcL /PFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=2ukjsCcPdiUN2qx1LoopIVt5rrKpX6zrzPS5FQbDQSM=; b=5v4C4dlRBVKgCFNuPhmAh+lZYQfnRr3iYz6ixVrMAUoKlFJ4jht66bLhrdqrkdK6j6 VXJBk7uy4CEeM5P3kwHtpjeUaVM63au7rgEQU2hLHuZiZ3H+ZzoIYSt9qhOIETuPcDBm XNMsDWrdwc4KdkmmOa0+DToaEN12Etu0Mt9noKEcCrKb50PvONFLE1ZDt8jKu4w4lB6I w3lpfN/meCKVr7CVPkv02UBW0lhdfZ4do7IIejJSADHdtJLzR58V8CjHuF7uPDL3TcTf SRCXkYoFZOBMYyWIb00ehC3StRGAJBS+dESvm3g3dlLZiHZ7THzAXi/HF1lOe6hO4we5 y/9Q== X-Gm-Message-State: AO0yUKUMul1sF3L9DvT5y5fgH8bAabaZVJkcw08bguuPf+kYvTFGDPvm AzupWNnC40awKI3TgaRp6iM= X-Google-Smtp-Source: AK7set9NTT5Z4SnL+fgXn9PrGHeQ3fajxv6suOaQL+Yllz7CotVysFOQkYf7UiXYPyLTsUN1MKYFdw== X-Received: by 2002:a05:600c:3318:b0:3ea:dc1b:90c with SMTP id q24-20020a05600c331800b003eadc1b090cmr3746561wmp.20.1677253420699; Fri, 24 Feb 2023 07:43:40 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id p9-20020a05600c1d8900b003e203681b26sm3262906wms.29.2023.02.24.07.43.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 24 Feb 2023 07:43:39 -0800 (PST) Content-Type: multipart/mixed; boundary="------------mtGtzMaaRULHOdwNZpuwQbgT" Message-ID: <32a03b65-ae98-7417-bcc7-e89836191a35@HIDDEN> Date: Fri, 24 Feb 2023 15:43:39 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, Paul Eggert <eggert@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> <B2E79E10-EF8A-4C97-8651-C418B0394C0C@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <B2E79E10-EF8A-4C97-8651-C418B0394C0C@HIDDEN> X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) This is a multi-part message in MIME format. --------------mtGtzMaaRULHOdwNZpuwQbgT Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 24/02/2023 14:33, George Valkov wrote: > >> On 2023-02-24, at 12:23 AM, Paul Eggert <eggert@HIDDEN> wrote: >> >> On 2/20/23 13:14, Pádraig Brady wrote: >>> Since https://github.com/coreutils/gnulib/commit/4db8db34 >>> gnulib has been unconditionally replacing lseek() on macos. >>> Now with SEEK_DATA undefined that replaced lseek() >>> will run the code intended for BeOS. >>> So either the lseek.m4 or lseek.c needs adjusting accordingly. >> >> Good catch, thanks. I updated the Gnulib patch accordingly; see attached. > > Nice: I just downloaded a fresh copy of Savannah, and it already includes the attached patch, so no action needed on my side. > > The copy created by cp is good. I noticed that you have added a —debug switch, so I gave it a test: > 1. If the target exists I get this output. I would assume zeroes means that all data is read from the source, but pages containing only zeroes are skipped, while pages containing data are written to the target, which is fine. I would guess 4 KB page granularity or some form of detection takes place. Exactly. > ./coreutils-2023-02-24/src/cp --debug cc1-mmap cc1-ori > 'cc1-mmap' -> 'cc1-ori' > copy offload: avoided, reflink: unsupported, sparse detection: zeros > > 2. If the target does not exist, the file is cloned. You might want to report something about that in the debug output. Otherwise the clone is good. > > ./coreutils-2023-02-24/src/cp --debug cc1-mmap cc1-ori > 'cc1-mmap' -> 'cc1-ori’ Yes that was a mistake. Should be fixed with https://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=65bb27656 > My first attempt to run the tests stopped here, so I had to interrupt it 10 minutes later with Control+C. > ^Cmake[1]: *** Deleting file 'tests/tail-2/inotify-race.log' > ^C^C^C^C^C^C > > killall gdb > ps -A |grep gdb > 29584 ?? 0:00.09 gdb -nx --batch-silent --eval-command=break 1475 --eval-command=run --pid=29583 -f file --eval-command=quit tail > 23269 ttys010 0:00.09 gdb -nx --batch-silent --eval-command=break 1475 --eval-command=run --pid=23268 -f file --eval-command=quit tail > > killall -9 gdb > ps -A |grep gdb > > Killing gdb allowed the tests to continue, I had to do it twice: That's awkward. That test documents the issue with protecting the gdb invocation with timeout(1). I suppose we could restrict this test to inotify capable systems, which would have the side effect of avoiding its use on non linux systems. The attached patch does that. thanks for all the testing. Pádraig --------------mtGtzMaaRULHOdwNZpuwQbgT Content-Type: text/x-patch; charset=UTF-8; name="macos-gdb-hang.patch" Content-Disposition: attachment; filename="macos-gdb-hang.patch" Content-Transfer-Encoding: base64 RnJvbSA5NTJjOGJhZDRjMjk3ZWQ0OGNkZGJiODFmMTAzMGEzNTgxMmNhOTgwIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/UD1DMz1BMWRyYWlnPTIwQnJhZHk/ PSA8UEBkcmFpZ0JyYWR5LmNvbT4KRGF0ZTogRnJpLCAyNCBGZWIgMjAyMyAxNTo0MDozNyAr MDAwMApTdWJqZWN0OiBbUEFUQ0hdIHRlc3RzOiBhdm9pZCBnZGIgaGFuZyBvbiBtYWNPUwoK KiB0ZXN0cy90YWlsLTIvaW5vdGlmeS1yYWNlLnNoOiBSZXN0cmljdCB0aGUgdGVzdCB0bwpp bm90aWZ5IGNhcGFibGUgc3lzdGVtcyB0byBhdm9pZCB0aGUgaGFuZyB3aXRoIHNvbWUgZ2Ri cy4KKiB0ZXN0cy90YWlsLTIvaW5vdGlmeS1yYWNlLnNoOiBMaWtld2lzZS4KLS0tCiB0ZXN0 cy90YWlsLTIvaW5vdGlmeS1yYWNlLnNoICB8IDMgKysrCiB0ZXN0cy90YWlsLTIvaW5vdGlm eS1yYWNlMi5zaCB8IDMgKysrCiAyIGZpbGVzIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygrKQoK ZGlmZiAtLWdpdCBhL3Rlc3RzL3RhaWwtMi9pbm90aWZ5LXJhY2Uuc2ggYi90ZXN0cy90YWls LTIvaW5vdGlmeS1yYWNlLnNoCmluZGV4IGM3MjJmYjlmYS4uNjNmOTA2NTM2IDEwMDc1NQot LS0gYS90ZXN0cy90YWlsLTIvaW5vdGlmeS1yYWNlLnNoCisrKyBiL3Rlc3RzL3RhaWwtMi9p bm90aWZ5LXJhY2Uuc2gKQEAgLTIzLDYgKzIzLDkgQEAKIC4gIiR7c3JjZGlyPS59L3Rlc3Rz L2luaXQuc2giOyBwYXRoX3ByZXBlbmRfIC4vc3JjCiBwcmludF92ZXJfIHRhaWwgc2xlZXAK IAorZ3JlcCAnXiNkZWZpbmUgSEFWRV9JTk9USUZZIDEnICIkQ09ORklHX0hFQURFUiIgPi9k ZXYvbnVsbCAmJiBpc19sb2NhbF9kaXJfIC4gXAorICB8fCBza2lwXyAnaW5vdGlmeSBpcyBu b3Qgc3VwcG9ydGVkJworCiAjIFRlcm1pbmF0ZSBhbnkgYmFja2dyb3VuZCBnZGIvdGFpbCBw cm9jZXNzCiBjbGVhbnVwXygpIHsKICAga2lsbCAkcGlkIDI+L2Rldi9udWxsICYmIHdhaXQg JHBpZApkaWZmIC0tZ2l0IGEvdGVzdHMvdGFpbC0yL2lub3RpZnktcmFjZTIuc2ggYi90ZXN0 cy90YWlsLTIvaW5vdGlmeS1yYWNlMi5zaAppbmRleCA4OWIwMmM2Y2YuLjE5MjE5YjcyZSAx MDA3NTUKLS0tIGEvdGVzdHMvdGFpbC0yL2lub3RpZnktcmFjZTIuc2gKKysrIGIvdGVzdHMv dGFpbC0yL2lub3RpZnktcmFjZTIuc2gKQEAgLTIyLDYgKzIyLDkgQEAKIC4gIiR7c3JjZGly PS59L3Rlc3RzL2luaXQuc2giOyBwYXRoX3ByZXBlbmRfIC4vc3JjCiBwcmludF92ZXJfIHRh aWwgc2xlZXAKIAorZ3JlcCAnXiNkZWZpbmUgSEFWRV9JTk9USUZZIDEnICIkQ09ORklHX0hF QURFUiIgPi9kZXYvbnVsbCAmJiBpc19sb2NhbF9kaXJfIC4gXAorICB8fCBza2lwXyAnaW5v dGlmeSBpcyBub3Qgc3VwcG9ydGVkJworCiAjIFRlcm1pbmF0ZSBhbnkgYmFja2dyb3VuZCBn ZGIvdGFpbCBwcm9jZXNzCiBjbGVhbnVwXygpIHsKICAga2lsbCAkcGlkIDI+L2Rldi9udWxs ICYmIHdhaXQgJHBpZAotLSAKMi4yNi4yCgo= --------------mtGtzMaaRULHOdwNZpuwQbgT--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 24 Feb 2023 14:33:34 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 24 09:33:34 2023 Received: from localhost ([127.0.0.1]:36473 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pVZ8f-0007ja-0m for submit <at> debbugs.gnu.org; Fri, 24 Feb 2023 09:33:34 -0500 Received: from mail-wm1-f48.google.com ([209.85.128.48]:34641) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pVZ8c-0007jN-2P for 61386 <at> debbugs.gnu.org; Fri, 24 Feb 2023 09:33:30 -0500 Received: by mail-wm1-f48.google.com with SMTP id o38-20020a05600c512600b003e8320d1c11so4216793wms.1 for <61386 <at> debbugs.gnu.org>; Fri, 24 Feb 2023 06:33:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=GkngYp3xyA6RcH8LeeHpvFm8ehCc5z2OIxGHjKQm7NQ=; b=BJJc/zynx3NTEp8Xj+9Qph0zgoiOZcdD+DBieGHUsFTjB1u6dktyXEcD7qlrmZxpfg ZbIY1K9SHhnMTHqgUGko4eJ9EcmATeknPNhFhPqtNfTv+Y/le1SlJe6OP+jHN4ik0cHB 4Y/9fWXu0+u7A1rbiZpI4WvmwjDL4gIgcDYU9f50x0rMhdhPmn0YFIucKokuT11Rco1m AlmowfaaMD/BhATo2KaT7f3Rxaw0ZA1fO5Dp/LU/lFPpZn+wvzgwdBysS8nWlkkOnXeH YwbQGEio6dCTX9yo7Nw9tL9iaIR5BI0SO4bglHIIwSiT8K0Eboi5SdwKF9g9hDbUrAWH /ngQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GkngYp3xyA6RcH8LeeHpvFm8ehCc5z2OIxGHjKQm7NQ=; b=ev0R3qOwNIx9EaOEBk//91XIyGinF5TCkyxc1cvdg956CYcvRKjAWo84qvTAbKLzNf RG74tbUMPDzW3wLdQKhFOUwqk3Ar6cJ3phr0QXhBJzS3F8wXjf1qbZ1wh5XY8+4XsdE9 urgjB64vC3jcYtBVvPrfYPvPKwR2ZNfWoc7Iey8Cw8jS8eKI9Dc2abhueYb7WvGvxY1T 9FSNJffOLfjRPPWsoAsLoPpCX7mfA3lKBpTp0C1fLgRLIe2Ul03hE41Bnh7CmDVf45tt NhBisl/er56ZYzGcPqhtd6LzpK9LeCoyQr9hudO1Ilq2KJxV1uBf3amNNaim4Oqsvukm Vlkw== X-Gm-Message-State: AO0yUKWbKhB+pAWZ4XSXcbUnJFGxiYIK755+kmhnIkyN+M7Kd3QJtmcB r+Jlld6zhECTpuBth55JlGA= X-Google-Smtp-Source: AK7set9vNhVA+KP6cduO/cRgZ1D/l22V1o9kDUCWgQeZymj3T1M/ynP0fN0veMUGoLWlFH2waZ7YQg== X-Received: by 2002:a05:600c:28e:b0:3db:2922:2b99 with SMTP id 14-20020a05600c028e00b003db29222b99mr12099779wmk.4.1677249203965; Fri, 24 Feb 2023 06:33:23 -0800 (PST) Received: from smtpclient.apple (212-39-89-230.ip.btc-net.bg. [212.39.89.230]) by smtp.gmail.com with ESMTPSA id u9-20020a05600c210900b003e2243cfe15sm2945483wml.39.2023.02.24.06.33.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Feb 2023 06:33:22 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> Date: Fri, 24 Feb 2023 16:33:20 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <B2E79E10-EF8A-4C97-8651-C418B0394C0C@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> To: Paul Eggert <eggert@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386 Cc: Gnulib bugs <bug-gnulib@HIDDEN>, 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On 2023-02-24, at 12:23 AM, Paul Eggert <eggert@HIDDEN> wrote: >=20 > On 2/20/23 13:14, P=C3=A1draig Brady wrote: >> Since https://github.com/coreutils/gnulib/commit/4db8db34 >> gnulib has been unconditionally replacing lseek() on macos. >> Now with SEEK_DATA undefined that replaced lseek() >> will run the code intended for BeOS. >> So either the lseek.m4 or lseek.c needs adjusting accordingly. >=20 > Good catch, thanks. I updated the Gnulib patch accordingly; see = attached. Nice: I just downloaded a fresh copy of Savannah, and it already = includes the attached patch, so no action needed on my side. The copy created by cp is good. I noticed that you have added a =E2=80=94d= ebug switch, so I gave it a test: 1. If the target exists I get this output. I would assume zeroes means = that all data is read from the source, but pages containing only zeroes = are skipped, while pages containing data are written to the target, = which is fine. I would guess 4 KB page granularity or some form of = detection takes place. ./coreutils-2023-02-24/src/cp --debug cc1-mmap cc1-ori 'cc1-mmap' -> 'cc1-ori' copy offload: avoided, reflink: unsupported, sparse detection: zeros 2. If the target does not exist, the file is cloned. You might want to = report something about that in the debug output. Otherwise the clone is = good. ./coreutils-2023-02-24/src/cp --debug cc1-mmap cc1-ori 'cc1-mmap' -> 'cc1-ori=E2=80=99 My first attempt to run the tests stopped here, so I had to interrupt it = 10 minutes later with Control+C. make check-TESTS make[1]: Entering directory '/Volumes/coreutils/coreutils-2023-02-24' PASS: tests/misc/help-version.sh PASS: tests/misc/help-version-getopt.sh Then I ran it again, and I can see the tests run very slowly now. This = time it hanged after: PASS: tests/rm/isatty.sh ^Cmake[1]: *** Deleting file 'tests/rm/empty-inacc.log' make[1]: *** [Makefile:21399: tests/rm/empty-inacc.log] Error 130 make: *** [Makefile:21381: check-TESTS] Interrupt: 2 make check-TESTS make[1]: Entering directory '/Volumes/coreutils/coreutils-2023-02-24' PASS: tests/misc/help-version.sh PASS: tests/misc/help-version-getopt.sh ^Cmake[1]: *** Deleting file 'tests/tail-2/inotify-race.log' ^C^C^C^C Now it hangs here and I cannot interrupt it. I don=E2=80=99t see any CPU = usage related to the tests. My laptop is idle. Attempting to close the = Terminal window gave me this warning: Do you want to terminate running processes in this tab? Closing this tab will terminate the running processes: gdb, bash, make = (2), sh (4). That=E2=80=99s a good culprit. Two days ago brew updated gdb to version = 13.1 and there was a message asking me to sign it with some entitlements = to make it more functional. So, I signed gdb. At first though make = check-TESTS does not play nice when gdb is signed, so I removed the = signature, but that did not help. Next I restored version 12.1_2 and ran = make check-TESTS again, which still hangs at the same points. So gdb = signature and version have nothing to do with this new issue. Next I ran the tests on top of my previous unpatched build directory = coreutils cf80f988eeb97cc3f8c65ae58e735d36f865277b, gnulib = 32c16c45d7378b014d9aac6130104c4d02a9acdb and it works fine, so I would = assume something related to the tests has been broken recently in = coreutils or gnulib. I restored gdb 13.1 signed and the tests completed = again. Back to latest unmodified coreutils = 5c8c2a5161c0b6f84212778f694c526105f10dab, gnulib = 7352d9fec59398c76c3bb8a2ef86ba58818f0342, the tests hang. make check-TESTS make[1]: Entering directory '/Volumes/coreutils/coreutils-2023-02-24' PASS: tests/misc/help-version.sh PASS: tests/misc/help-version-getopt.sh ^Cmake[1]: *** Deleting file 'tests/tail-2/inotify-race.log' ^C^C^C^C^C^C killall gdb ps -A |grep gdb 29584 ?? 0:00.09 gdb -nx --batch-silent --eval-command=3Dbreak = 1475 --eval-command=3Drun --pid=3D29583 -f file --eval-command=3Dquit = tail 23269 ttys010 0:00.09 gdb -nx --batch-silent --eval-command=3Dbreak = 1475 --eval-command=3Drun --pid=3D23268 -f file --eval-command=3Dquit = tail killall -9 gdb ps -A |grep gdb Killing gdb allowed the tests to continue, I had to do it twice: = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-06= -5c8c2a5161c0b6f84212778f694c526105f10dab-ori.txt > On 2/20/23 02:21, George Valkov wrote: > > What is the correct way to apply your patch? >=20 > Sounds like patching is a bit of a hassle, so to make things easier I = installed the attached patch into Gnulib, and propagated this into = Coreutils. I=E2=80=99m starting to think I made a mistake by trying to apply your = patch inside coreutils, since it already had links to the affected = files. I guess I should have applied the patch inside coreutils/gnulib? So my question was: what command do you use to apply your patch and in = which directory do you run it? > You should be able to get the latest version of Coreutils from = savannah.gnu.org, and then run './bootstrap; ./configure; make' if you = have suitable development tools like Autoconf. You can run ./bootstrap = on a GNU/Linux platform and then copy the directory to macOS and run = './configure; make' on macOS. Please give it a try. All the development tools should already be installed on the Mac using = brew. I can confirm autoconf is installed: brew list | grep autoconf autoconf I always start with these commands: git clone https://github.com/coreutils/coreutils.git cd coreutils git submodule foreach git pull origin master ./bootstrap ./configure make -j 16 git checkout -b 2023-02-24 Next I apply the changes you want, make clean ; make -j 16 Finally I proceed with testing. > > If you know what conditions are required to reproduce the issue on = FreeBSD, >=20 > I don't. I'm relying on FreeBSD bug report 256205. I cited that in the = attached patch. I see they reproduce the bug on both arm64 and x64. Based on the = conversation their issue is very similar: they use a linker to craft a = file, that linker is likely to use mmap. But their issue is fixed with = fsync. And they also talk about nulls. macOS is indeed based on BSD. Also if I read correctly, they do not experience the issue when working = with the root file-system on FreeBSD, its the same on macOS: I need to = mount an APFS sparse disk image, they use nullfs. Comments say they seem to have fixed it partiality in = 42881526d401e7a9c09241e392b7ffa18cfe11d6, and then completely later. I = am too busy to play with FreeBSD at the moment, as it takes more = advanced preparation. > On 2023-02-24, at 2:51 AM, P=C3=A1draig Brady <P@HIDDEN> = wrote: >=20 >> On 2/20/23 02:21, George Valkov wrote: >> > What is the correct way to apply your patch? >> Sounds like patching is a bit of a hassle, so to make things easier I >> installed the attached patch into Gnulib, and propagated this into >> Coreutils. >=20 > The latest coreutils that should include all fixes for this issue is = at > https://www.pixelbeat.org/cu/coreutils-9.1.160-5c8c2.tar.gz > That should be buildable with the standard ./configure && make combo P=C3=A1draig, I compiled from your archive, cp produces a good copy, = however make check-TESTS hangs the same way as with the clone I compiled = on the latest Savannah master. Good luck! Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 24 Feb 2023 00:51:18 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 23 19:51:18 2023 Received: from localhost ([127.0.0.1]:35635 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pVMIw-0000oE-Cf for submit <at> debbugs.gnu.org; Thu, 23 Feb 2023 19:51:18 -0500 Received: from mail-wm1-f53.google.com ([209.85.128.53]:40520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pVMIu-0000nz-SM for 61386 <at> debbugs.gnu.org; Thu, 23 Feb 2023 19:51:17 -0500 Received: by mail-wm1-f53.google.com with SMTP id fm20-20020a05600c0c1400b003ead37e6588so979782wmb.5 for <61386 <at> debbugs.gnu.org>; Thu, 23 Feb 2023 16:51:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=DjoosDptj6GmuajCLqTo4kqHt9OAiumYTUqpIE9h1QU=; b=Dx5iDd0VQIDQV86TAsmEsNRDB+9Mq6w5TTNVpPUrr/qL06Qb2vWgMg/FfdKmZfrkOs ooke7TihrMiFCe+Tv8/AI8Dy5XYmcxxeyDaEO8/q1IKKX4aSJueQqz/dbOA5Y12Jd678 5ubu7Pj2sn0OQpsadc5iDG5gc2UL0XlJfVYEMb9rowrOadI2vcLbrtTARpnKVEXMUd5v pm4kUMXH5CtMnlH0YP9c8QGFuXfbM0BByllzKUS46Qae7oL9Q3OBhNwwZAB+zxwgTU4x VStYq1TmatJp+grKh2zrTz4snjwWQNEgOJSZjZtD6DFUhpy34/IQ1DKfnB2ewWXiBKee PQrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DjoosDptj6GmuajCLqTo4kqHt9OAiumYTUqpIE9h1QU=; b=m/L207dKjVFjbtS2l+CqOfON0gX+wObxLBSje6w2Bjt8XH9+kTA0V5bsfs9NOAgUAs 4QXaH8Z0RJhyfOUJ7QymqK8CPUOdfq7fka1ZgXuhiyUFkjurAKNhJJnKiLhUNRKyGKS7 +/ToL4AhM1fYJEAUeW4ruYnQaM/3kQlnu7vSKKpZjSXGNPppBvOnam24sRTnRRsJQBNV 5ZG+sbtL4UKraTasc68rYO5hYVLfhxrOb8ZsegEfyVGzINk1lY+Ia5EG/jXkp8VnOLna jx+i3s99mu74qNOORBaZMBi399CvHhd2TlG4XtNs0rkYN/EMMCU1K8OCIpJy4/UqtO6T dW7w== X-Gm-Message-State: AO0yUKUVzxdZ9e39zoYhx1cV3WNJnc5u80i0Nn78dBtitWUoN2C4XOXW ZIeb8v2MClznx7zG1IxCssw= X-Google-Smtp-Source: AK7set8g//AY3Zui3p7W+th5T0t+7hMMtSkQQwWZmUCvF7sefv1UM7XUOUGveFuZtusLoqzWBrYkWA== X-Received: by 2002:a05:600c:13d5:b0:3e2:40e:6d70 with SMTP id e21-20020a05600c13d500b003e2040e6d70mr10348796wmg.36.1677199869972; Thu, 23 Feb 2023 16:51:09 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id e9-20020a05600c218900b003dc434b39c7sm6232620wme.0.2023.02.23.16.51.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 23 Feb 2023 16:51:09 -0800 (PST) Message-ID: <f89b4024-a83c-3a8a-f21d-21ae1d788033@HIDDEN> Date: Fri, 24 Feb 2023 00:51:08 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 23/02/2023 22:23, Paul Eggert wrote: > On 2/20/23 13:14, Pádraig Brady wrote: >> Since https://github.com/coreutils/gnulib/commit/4db8db34 >> gnulib has been unconditionally replacing lseek() on macos. >> Now with SEEK_DATA undefined that replaced lseek() >> will run the code intended for BeOS. >> So either the lseek.m4 or lseek.c needs adjusting accordingly. > > Good catch, thanks. I updated the Gnulib patch accordingly; see attached. Cool. > On 2/20/23 02:21, George Valkov wrote: > > What is the correct way to apply your patch? > > Sounds like patching is a bit of a hassle, so to make things easier I > installed the attached patch into Gnulib, and propagated this into > Coreutils. The latest coreutils that should include all fixes for this issue is at https://www.pixelbeat.org/cu/coreutils-9.1.160-5c8c2.tar.gz That should be buildable with the standard ./configure && make combo thanks!, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 23 Feb 2023 22:23:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 23 17:23:40 2023 Received: from localhost ([127.0.0.1]:35512 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pVK04-0005Ou-1S for submit <at> debbugs.gnu.org; Thu, 23 Feb 2023 17:23:40 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:35728) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pVK01-0005Oe-BR for 61386 <at> debbugs.gnu.org; Thu, 23 Feb 2023 17:23:38 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 7F284160056; Thu, 23 Feb 2023 14:23:31 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id vTmZrTi6SyM0; Thu, 23 Feb 2023 14:23:30 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id E8D25160085; Thu, 23 Feb 2023 14:23:29 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu E8D25160085 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1677191010; bh=gt5L5LaMXthEaKK9aqA5bUvojxI+Pw/Uhlfr5LWqrsE=; h=Content-Type:Message-ID:Date:MIME-Version:Subject:To:From; b=O7EhIHcC8fEbBNzO8AsAbBiQphJvT2vg4FHLKqNIfVcg/wW91v5IJCkYLUYSUiQcy szoke/hbAAiQITR1MD6VRnIkc0W14XxSMztOn8vqqCpXLLgXBcd9DVGatTBCXDBk3Q gvss0p4CS5k4pI+7m/kre0bS3J/v/0n3NNHrFekc= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id OT-_mLs8gLHF; Thu, 23 Feb 2023 14:23:29 -0800 (PST) Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id C4271160056; Thu, 23 Feb 2023 14:23:29 -0800 (PST) Content-Type: multipart/mixed; boundary="------------9jDZ5WtlHvlD3ziuT301Hn57" Message-ID: <b51075cb-42b2-b176-ddbe-b2d066e49986@HIDDEN> Date: Thu, 23 Feb 2023 14:23:29 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) This is a multi-part message in MIME format. --------------9jDZ5WtlHvlD3ziuT301Hn57 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable On 2/20/23 13:14, P=C3=A1draig Brady wrote: > Since https://github.com/coreutils/gnulib/commit/4db8db34 > gnulib has been unconditionally replacing lseek() on macos. > Now with SEEK_DATA undefined that replaced lseek() > will run the code intended for BeOS. > So either the lseek.m4 or lseek.c needs adjusting accordingly. Good catch, thanks. I updated the Gnulib patch accordingly; see attached. On 2/20/23 02:21, George Valkov wrote: > What is the correct way to apply your patch? Sounds like patching is a bit of a hassle, so to make things easier I=20 installed the attached patch into Gnulib, and propagated this into=20 Coreutils. You should be able to get the latest version of Coreutils from=20 savannah.gnu.org, and then run './bootstrap; ./configure; make' if you=20 have suitable development tools like Autoconf. You can run ./bootstrap=20 on a GNU/Linux platform and then copy the directory to macOS and run=20 './configure; make' on macOS. Please give it a try. > If you know what conditions are required to reproduce the issue on=20 FreeBSD, I don't. I'm relying on FreeBSD bug report 256205. I cited that in the=20 attached patch. > Is it ok if I continue testing on FreeBSD 13.1 Sure, that's fine. Possibly the bug is fixed in FreeBSD 13.1; if so,=20 perhaps we can improve performance on 13.1 by changing "__FreeBSD__ <=20 14" to something else. On 2/20/23 13:25, George Valkov wrote: > there is no need for that hack in lseek.c. Yes, we don't need any new hack in lseek.c. And strictly speaking, even=20 the existing macOS-specific hack in lseek.c (look for __APPLE__) is no=20 longer needed, since (with the changes I just installed) lseek.c is no=20 longer compiled for macOS. However, assuming Apple fixes the macOS bug in (say) macOS 14, so that=20 we change the "99990000" to "140000" in Gnulib's lib/unistd.in.h and=20 m4/lseek.m4, we'll likely need that hack back because it works around an=20 incompatibility between GNU-or-FreeBSD-or-Solaris SEEK_DATA and macOS=20 SEEK_DATA; see=20 <https://www.gnu.org/software/gnulib/manual/html_node/lseek.html>. So I=20 thought it nicer to leave it in for now; it has zero runtime cost on all=20 platforms. --------------9jDZ5WtlHvlD3ziuT301Hn57 Content-Type: text/x-patch; charset=UTF-8; name="0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch" Content-Disposition: attachment; filename="0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch" Content-Transfer-Encoding: base64 RnJvbSA3MzUyZDlmZWM1OTM5OGM3NmMzYmI4YTJlZjg2YmE1ODgxOGYwMzQyIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBTdW4sIDE5IEZlYiAyMDIzIDAwOjA1OjI0IC0wNjAwClN1YmplY3Q6IFtQQVRD SF0gbHNlZWs6IGF2b2lkIFNFRUtfSE9MRSBidWdzIGluIEZyZWVCU0QsIG1hY09TCk1JTUUt VmVyc2lvbjogMS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD1VVEYtOApD b250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0CgpUaGlzIGF0dGVtcHRzIHRvIGZpeCA8 aHR0cHM6Ly9idWdzLmdudS5vcmcvNjEzODY+LCBhIGJ1ZyBpbiBHTlUgY3AKY2F1c2VkIGJ5 IGEgc2VyaW91cyBkYXRhIGNvcnJ1cHRpb24gYnVnIGluIEZyZWVCU0QgYW5kIG1hY09TLgoq IGRvYy9wb3NpeC1mdW5jdGlvbnMvbHNlZWsudGV4aTogTWVudGlvbiB0aGUgYnVnLgoqIGxp Yi91bmlzdGQuaW4uaCAoU0VFS19EQVRBLCBTRUVLX0hPTEUpOiBVbmRlZiBpbiBtYWNPUyA8 IDEzIGFuZApGcmVlQlNEIDwgMTQuICBGcmVlQlNEIGZpeGVkIHRoZSBidWcgc29tZXRpbWUg ZHVyaW5nIEZyZWVCU0QgMTMKPGh0dHBzOi8vYnVncy5mcmVlYnNkLm9yZy8yNTYyMDU+LCBz byB0aGUgIkZyZWVCU0QgPCAxNCIgaXMKY29uc2VydmF0aXZlLiAgSXTigJlzIHVua25vd24g d2hlbiBBcHBsZSB3aWxsIGZpeCBtYWNPUyBzbyB1c2UKbWFjT1MgIjk5OTkiIGFzIGEgcGxh Y2Vob2xkZXIuCiogbTQvbHNlZWsubTQgKGdsX0ZVTkNfTFNFRUspOiBSZXBsYWNlIGxzZWVr IGlmIG9uIG9uZSBvZiB0aGUKYWJvdmUgcGxhdGZvcm1zLgotLS0KIENoYW5nZUxvZyAgICAg ICAgICAgICAgICAgICAgICB8IDE0ICsrKysrKysrKysrKysrCiBkb2MvcG9zaXgtZnVuY3Rp b25zL2xzZWVrLnRleGkgfCAgNSArKysrKwogbGliL3VuaXN0ZC5pbi5oICAgICAgICAgICAg ICAgIHwgMTggKysrKysrKysrKysrKysrKysrCiBtNC9sc2Vlay5tNCAgICAgICAgICAgICAg ICAgICAgfCAzMiArKysrKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLQogNCBmaWxlcyBj aGFuZ2VkLCA2MyBpbnNlcnRpb25zKCspLCA2IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBh L0NoYW5nZUxvZyBiL0NoYW5nZUxvZwppbmRleCBjMWNhNjEwNTQ4Li41ZTQzNmY1YjNhIDEw MDY0NAotLS0gYS9DaGFuZ2VMb2cKKysrIGIvQ2hhbmdlTG9nCkBAIC0xLDMgKzEsMTcgQEAK KzIwMjMtMDItMjMgIFBhdWwgRWdnZXJ0ICA8ZWdnZXJ0QGNzLnVjbGEuZWR1PgorCisJbHNl ZWs6IGF2b2lkIFNFRUtfSE9MRSBidWdzIGluIEZyZWVCU0QsIG1hY09TCisJVGhpcyBhdHRl bXB0cyB0byBmaXggPGh0dHBzOi8vYnVncy5nbnUub3JnLzYxMzg2PiwgYSBidWcgaW4gR05V IGNwCisJY2F1c2VkIGJ5IGEgc2VyaW91cyBkYXRhIGNvcnJ1cHRpb24gYnVnIGluIEZyZWVC U0QgYW5kIG1hY09TLgorCSogZG9jL3Bvc2l4LWZ1bmN0aW9ucy9sc2Vlay50ZXhpOiBNZW50 aW9uIHRoZSBidWcuCisJKiBsaWIvdW5pc3RkLmluLmggKFNFRUtfREFUQSwgU0VFS19IT0xF KTogVW5kZWYgaW4gbWFjT1MgPCAxMyBhbmQKKwlGcmVlQlNEIDwgMTQuICBGcmVlQlNEIGZp eGVkIHRoZSBidWcgc29tZXRpbWUgZHVyaW5nIEZyZWVCU0QgMTMKKwk8aHR0cHM6Ly9idWdz LmZyZWVic2Qub3JnLzI1NjIwNT4sIHNvIHRoZSAiRnJlZUJTRCA8IDE0IiBpcworCWNvbnNl cnZhdGl2ZS4gIEl04oCZcyB1bmtub3duIHdoZW4gQXBwbGUgd2lsbCBmaXggbWFjT1Mgc28g dXNlCisJbWFjT1MgIjk5OTkiIGFzIGEgcGxhY2Vob2xkZXIuCisJKiBtNC9sc2Vlay5tNCAo Z2xfRlVOQ19MU0VFSyk6IFJlcGxhY2UgbHNlZWsgaWYgb24gb25lIG9mIHRoZQorCWFib3Zl IHBsYXRmb3Jtcy4KKwogMjAyMy0wMi0xOCAgQnJ1bm8gSGFpYmxlICA8YnJ1bm9AY2xpc3Au b3JnPgogCiAJY29uZmlnbWFrZTogQWRkIHN1cHBvcnQgZm9yICRidWlsZF9vcyAhPSAkaG9z dF9vcy4KZGlmZiAtLWdpdCBhL2RvYy9wb3NpeC1mdW5jdGlvbnMvbHNlZWsudGV4aSBiL2Rv Yy9wb3NpeC1mdW5jdGlvbnMvbHNlZWsudGV4aQppbmRleCAyZjhlMmI1ODc3Li4zNDcwNTI0 YjEyIDEwMDY0NAotLS0gYS9kb2MvcG9zaXgtZnVuY3Rpb25zL2xzZWVrLnRleGkKKysrIGIv ZG9jL3Bvc2l4LWZ1bmN0aW9ucy9sc2Vlay50ZXhpCkBAIC0zNyw0ICszNyw5IEBAIElSSVgg Ni41LgogQGl0ZW0KIFNvbWUgc3lzdGVtcyBkbyBub3Qgc3VwcG9ydCBAY29kZXtTRUVLX0RB VEF9IGFuZCBAY29kZXtTRUVLX0hPTEV9OgogQUlYLCBIUC1VWCwgTWljcm9zb2Z0IFdpbmRv d3MsIE5ldEJTRCwgT3BlbkJTRC4KK0BpdGVtCitTb21lIHN5c3RlbXMgaGF2ZSBhIGJ1Z2d5 IEBjb2Rle1NFRUtfREFUQX0gYW5kIEBjb2Rle1NFRUtfSE9MRX0sCithbmQgR251bGliIHdv cmtzIGFyb3VuZCB0aGUgcHJvYmxlbSB2aWEgQGNvZGV7I3VuZGVmIFNFRUtfREFUQX0KK2Fu ZCBAY29kZXsjdW5kZWYgU0VFS19IT0xFfToKK0ZyZWVCU0QgMTMsIG1hY09TIDEyLgogQGVu ZCBpdGVtaXplCmRpZmYgLS1naXQgYS9saWIvdW5pc3RkLmluLmggYi9saWIvdW5pc3RkLmlu LmgKaW5kZXggYmZjNTAxZTVhNy4uOGJhOTg2Nzg5NCAxMDA2NDQKLS0tIGEvbGliL3VuaXN0 ZC5pbi5oCisrKyBiL2xpYi91bmlzdGQuaW4uaApAQCAtNDAsNiArNDAsMjQgQEAKICMgdW5k ZWYgX0dMX0lOQ0xVRElOR19VTklTVERfSAogI2VuZGlmCiAKKy8qIEF2b2lkIGxzZWVrIGJ1 Z3MgaW4gRnJlZUJTRCwgbWFjT1MgPGh0dHBzOi8vYnVncy5nbnUub3JnLzYxMzg2Pi4KKyAg IFRoaXMgYnVnIGlzIGZpeGVkIGFmdGVyIEZyZWVCU0QgMTM7IHNlZSA8aHR0cHM6Ly9idWdz LmZyZWVic2Qub3JnLzI1NjIwNT4uCisgICBVc2UgbWFjT1MgIjk5OTkiIHRvIHN0YW5kIGZv ciBhIGZ1dHVyZSBmaXhlZCBtYWNPUyB2ZXJzaW9uLiAgKi8KKyNpZiBkZWZpbmVkIF9fRnJl ZUJTRF9fICYmIF9fRnJlZUJTRF9fIDwgMTQKKyMgdW5kZWYgU0VFS19EQVRBCisjIHVuZGVm IFNFRUtfSE9MRQorI2VsaWYgZGVmaW5lZCBfX0FQUExFX18gJiYgZGVmaW5lZCBfX01BQ0hf XyAmJiBkZWZpbmVkIFNFRUtfREFUQQorIyBpZmRlZiBfX0VOVklST05NRU5UX01BQ19PU19Y X1ZFUlNJT05fTUlOX1JFUVVJUkVEX18KKyMgIGluY2x1ZGUgPEF2YWlsYWJpbGl0eU1hY3Jv cy5oPgorIyBlbmRpZgorIyBpZiAoIWRlZmluZWQgTUFDX09TX1hfVkVSU0lPTl9NSU5fUkVR VUlSRUQgXAorICAgICAgfHwgTUFDX09TX1hfVkVSU0lPTl9NSU5fUkVRVUlSRUQgPCA5OTk5 MDAwMCkKKyMgIGluY2x1ZGUgPHN5cy9mY250bC5oPiAvKiBJdCBhbHNvIGRlZmluZXMgdGhl IHR3byBtYWNyb3MuICAqLworIyAgdW5kZWYgU0VFS19EQVRBCisjICB1bmRlZiBTRUVLX0hP TEUKKyMgZW5kaWYKKyNlbmRpZgorCiAvKiBHZXQgYWxsIHBvc3NpYmxlIGRlY2xhcmF0aW9u cyBvZiBnZXRob3N0bmFtZSgpLiAgKi8KICNpZiBAR05VTElCX0dFVEhPU1ROQU1FQCAmJiBA VU5JU1REX0hfSEFWRV9XSU5TT0NLMl9IQCBcCiAgICYmICFkZWZpbmVkIF9HTF9JTkNMVURJ TkdfV0lOU09DSzJfSApkaWZmIC0tZ2l0IGEvbTQvbHNlZWsubTQgYi9tNC9sc2Vlay5tNApp bmRleCBmZDRmMWYyN2QzLi42ZTFhYjZmZmFhIDEwMDY0NAotLS0gYS9tNC9sc2Vlay5tNAor KysgYi9tNC9sc2Vlay5tNApAQCAtMSw0ICsxLDQgQEAKLSMgbHNlZWsubTQgc2VyaWFsIDEy CisjIGxzZWVrLm00IHNlcmlhbCAxMwogZG5sIENvcHlyaWdodCAoQykgMjAwNywgMjAwOS0y MDIzIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbiwgSW5jLgogZG5sIFRoaXMgZmlsZSBpcyBm cmVlIHNvZnR3YXJlOyB0aGUgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9uCiBkbmwgZ2l2ZXMg dW5saW1pdGVkIHBlcm1pc3Npb24gdG8gY29weSBhbmQvb3IgZGlzdHJpYnV0ZSBpdCwKQEAg LTcwLDkgKzcwLDI5IEBAIEFDX0RFRlVOKFtnbF9GVU5DX0xTRUVLXSwKICAgICBSRVBMQUNF X0xTRUVLPTEKICAgZmkKIAotICBkbmwgbWFjT1MgU0VFS19EQVRBIGlzIGluY29tcGF0aWJs ZSB3aXRoIG90aGVyIHBsYXRmb3Jtcy4KLSAgY2FzZSAkaG9zdF9vcyBpbgotICAgIGRhcndp biopCi0gICAgICBSRVBMQUNFX0xTRUVLPTE7OwotICBlc2FjCisgIEFTX0lGKFt0ZXN0ICRS RVBMQUNFX0xTRUVLID0gMF0sCisgICAgW0FDX0NBQ0hFX0NIRUNLKFt3aGV0aGVyIFNFRUtf REFUQSB3b3JrcyBidXQgaXMgaW5jb21wYXRpYmxlIHdpdGggR05VXSwKKyAgICAgICBbZ2xf Y3ZfZnVuY19sc2Vla193b3Jrc19idXRfaW5jb21wYXRpYmxlXSwKKyAgICAgICBbQUNfUFJF UFJPQ19JRkVMU0UoCisgICAgICAgICAgW0FDX0xBTkdfU09VUkNFKAorICAgICAgICAgICAg IGRubCBVc2UgbWFjT1MgIjk5OTkiIHRvIHN0YW5kIGZvciBhIGZ1dHVyZSBmaXhlZCBtYWNP UyB2ZXJzaW9uLgorICAgICAgICAgICAgIGRubCBTZWUgLi4vbGliL3VuaXN0ZC5pbi5oIGFu ZCA8aHR0cHM6Ly9idWdzLmdudS5vcmcvNjEzODY+LgorICAgICAgICAgICAgIFtbI2luY2x1 ZGUgPHVuaXN0ZC5oPgorICAgICAgICAgICAgICAgI2lmIGRlZmluZWQgX19BUFBMRV9fICYm IGRlZmluZWQgX19NQUNIX18gJiYgZGVmaW5lZCBTRUVLX0RBVEEKKyAgICAgICAgICAgICAg ICMgaWZkZWYgX19FTlZJUk9OTUVOVF9NQUNfT1NfWF9WRVJTSU9OX01JTl9SRVFVSVJFRF9f CisgICAgICAgICAgICAgICAjICBpbmNsdWRlIDxBdmFpbGFiaWxpdHlNYWNyb3MuaD4KKyAg ICAgICAgICAgICAgICMgZW5kaWYKKyAgICAgICAgICAgICAgICMgaWYgOTk5OTAwMDAgPD0g TUFDX09TX1hfVkVSU0lPTl9NSU5fUkVRVUlSRUQKKyAgICAgICAgICAgICAgICMgIGRlZmlu ZSBMU0VFS19XT1JLU19CVVRfSVNfSU5DT01QQVRJQkxFX1dJVEhfR05VCisgICAgICAgICAg ICAgICAjIGVuZGlmCisgICAgICAgICAgICAgICAjZW5kaWYKKyAgICAgICAgICAgICAgICNp Zm5kZWYgTFNFRUtfV09SS1NfQlVUX0lTX0lOQ09NUEFUSUJMRV9XSVRIX0dOVQorICAgICAg ICAgICAgICAgICNlcnJvciAiTm8gbmVlZCB0byB3b3JrIGFyb3VuZCB0aGUgYnVnIgorICAg ICAgICAgICAgICAgI2VuZGlmCisgICAgICAgICAgICAgXV0pXSwKKyAgICAgICAgICBbZ2xf Y3ZfZnVuY19sc2Vla193b3Jrc19idXRfaW5jb21wYXRpYmxlPXllc10sCisgICAgICAgICAg W2dsX2N2X2Z1bmNfbHNlZWtfd29ya3NfYnV0X2luY29tcGF0aWJsZT1ub10pXSkKKyAgICAg aWYgdGVzdCAiJGdsX2N2X2Z1bmNfbHNlZWtfd29ya3NfYnV0X2luY29tcGF0aWJsZSIgPSB5 ZXM7IHRoZW4KKyAgICAgICBSRVBMQUNFX0xTRUVLPTEKKyAgICAgZmldKQogXSkKLS0gCjIu MzkuMgoK --------------9jDZ5WtlHvlD3ziuT301Hn57--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 22 Feb 2023 13:59:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 22 08:59:51 2023
Received: from localhost ([127.0.0.1]:58386 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pUpew-0003Ne-Sm
for submit <at> debbugs.gnu.org; Wed, 22 Feb 2023 08:59:51 -0500
Received: from mail-wm1-f50.google.com ([209.85.128.50]:33733)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pUpeu-0003NR-L5
for 61386 <at> debbugs.gnu.org; Wed, 22 Feb 2023 08:59:50 -0500
Received: by mail-wm1-f50.google.com with SMTP id
k14-20020a05600c1c8e00b003e22107b7ccso1200991wms.0
for <61386 <at> debbugs.gnu.org>; Wed, 22 Feb 2023 05:59:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=1QufiMbVoJ5HsTrd/KZatzh8BJOJRZewIU8zMObr/mY=;
b=Gz845i1Ld60UOrLe1RPJq90uH7Zv7z3BIlsfCgLU3R1ncWM8W4ix7bxbRHrS6PKRl7
F6EgYvnHI5xyMQUNJrrB2V3iPTNDCP7WjQ+30g16eTfVUWM5O/KUtOcJfb4RVEH1X+uY
Aiyw4IqZQcNFeSWkOnGKNb26FQmmRHhlfrwi+YTtsrwUoPtwrzStOsBd2W/UG1X3IpKR
lTz8kr7CO2YxiviAbW9yz4STo6G+7Lzx6NqdaXeKfFhsodqsUkOAvaReBJON388SCDFI
ByVrafwqqIt5moG+IAmyJ9De5tdTsPG9ePwZY4GBuZIKGANVsQtc+NjXVPU4ynq9qlFv
DAkg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=1QufiMbVoJ5HsTrd/KZatzh8BJOJRZewIU8zMObr/mY=;
b=5XEz3q6TvuX6vkNOVgfKFG9h6PVdBA57m+qeCV+QwK49m4xGcesG10lF+u54YWGNdx
sM263rJ8FxNwYd6whxqe/RU7/qWeYE/g98Y0X/I8R57sECNC0ZI/XnRf35AxUWlP7GXN
UTdCsh6c3pAqRrAtM53xrg9HC7uVccjHCe4BJ+XyYztjct1churOgN2Jf93QHCooXDNp
Jc2JJpKEp4QP3Df+10YM3gvMioOwPfbVW8TIcuIxUMuaBNe5UxXqjGyjbyvmiIxunbrC
F2wpWJnry5yVkEIqnWoDcyfSfemr29TyJldRKmQWdRtHmNQ8Y6+LSJCArz1plSPopDVi
tYgw==
X-Gm-Message-State: AO0yUKVUu/Ihv59UG1LVxU6ub74t8XHN+XcIEQnlEiAqUz1wZYTuqtS0
36DRejDaKJTVnYttA39cFCk=
X-Google-Smtp-Source: AK7set9QcGoOv9fm18t0JrA8yi+yYA6v0HJK8rIEjzi7rQA72FxmYQuLw/oZAdu2m1OltRKn/yKbyg==
X-Received: by 2002:a05:600c:350f:b0:3e2:cc:d8d8 with SMTP id
h15-20020a05600c350f00b003e200ccd8d8mr7956146wmq.2.1677074381292;
Wed, 22 Feb 2023 05:59:41 -0800 (PST)
Received: from smtpclient.apple (212-39-89-88.ip.btc-net.bg. [212.39.89.88])
by smtp.gmail.com with ESMTPSA id
n30-20020a05600c3b9e00b003e206cc7237sm8990384wms.24.2023.02.22.05.59.39
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Wed, 22 Feb 2023 05:59:40 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <842D303D-7BFD-4418-B3F4-A61CEB4FC903@HIDDEN>
Date: Wed, 22 Feb 2023 15:59:38 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <770F59B0-A4BF-4A9D-B219-18EBCE960FAD@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
<f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
<63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN>
<f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN>
<1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN>
<E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN>
<3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN>
<4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN>
<9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN>
<55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN>
<9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN>
<eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN>
<47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN>
<3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN>
<842D303D-7BFD-4418-B3F4-A61CEB4FC903@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>,
Paul Eggert <eggert@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 1.5 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: I received an update from Apple > We reproduced the issue
and are investigating. > Our engineers are investigating the root cause of
the issue you reported. If we need more information from you, we’ [...]
Content analysis details: (1.5 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (gvalkov[at]gmail.com)
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust
[209.85.128.50 listed in list.dnswl.org]
-0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[209.85.128.50 listed in wl.mailspike.net]
1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
[212.39.89.88 listed in dnsbl.sorbs.net]
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)
I received an update from Apple
> We reproduced the issue and are investigating.
> Our engineers are investigating the root cause of the issue you =
reported. If we need more information from you, we=E2=80=99ll add a =
comment and send you an email.
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 20 Feb 2023 21:25:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 20 16:25:37 2023
Received: from localhost ([127.0.0.1]:53827 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pUDfE-0006US-Nl
for submit <at> debbugs.gnu.org; Mon, 20 Feb 2023 16:25:37 -0500
Received: from mail-wm1-f48.google.com ([209.85.128.48]:52928)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pUDfD-0006UG-4Y
for 61386 <at> debbugs.gnu.org; Mon, 20 Feb 2023 16:25:36 -0500
Received: by mail-wm1-f48.google.com with SMTP id n31so1767725wms.2
for <61386 <at> debbugs.gnu.org>; Mon, 20 Feb 2023 13:25:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=eGigkQr6EzagY74qRb5LTQh6e6sYysY/EcocLqrXwZE=;
b=kQLziiDJFBt8XltmZhCxUKAAYRpecEQke8eiJ6Ysy2mTwiPHJE9XSRcuI5HvRwb3Z1
jDYYASexNoa46hb3z9/aafQ16TlGtBKuqgzQn7mzrlSClxylcmWyqnr+Tgz3yAnjGaep
cOYSNSDva8b1BwWphNJz5QwRS+D4zf30oytjuqOyrdE3vXHh8+sbOvgCwn9CHZ/B5+Q6
J7UzC3RHzdW4qx19Dy/BWverATk5mRd3t/GvZsTWC+PCxZ7tk/9HX5e2loM4siXO+GO/
2DX78g1f+1nbwEO7NqXKA8pE1+cOyqMLkeMlumTu11KmpEKst7lNXNrFtbosXN1+Kn0p
vzbw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=eGigkQr6EzagY74qRb5LTQh6e6sYysY/EcocLqrXwZE=;
b=ah9hPke3w2xY84UYj4nhLJ26PXDo+Mr6gh/Dpdm8oKQAKRFSctF3cO5zIqs0XE9p+S
XRGR8TL3xsVNOktk//LSGvGGL7vBb55s10EyRSG/HbfU89p+2yqUUM4HIq/s0/X2p8Hn
EotG70rioZQEBezBitvT0+/Gp8CTO8dliZc2RxAAixH5eum5aOC63T/2Tv4cgGoeoM+V
oVKyzHJaarxF02GeCg/whZxVczQKYh/NUdCNr0yzB3WV279d3OUAdk8X+tgnd56Rh0Lh
xuetP6Ojl6kXYJCnres/L5SM2dsYiF9ah+GBxymPayM18OUxhZffvaXqSgs0UI6ftQNo
Z/Pg==
X-Gm-Message-State: AO0yUKUrF9hPfX2GmJaSRAn/psogWjzzu+n+nAuFOEo7xSrKYtL0ZkJI
hperEMtj44oPUoXd8QUcTj8=
X-Google-Smtp-Source: AK7set9+C0o/6zzJs0RbL66Z9Xrq5C6AG8ln4f37aRbnV1rY2E0BVflqNTrHgqiyStpqsVFnwg35IA==
X-Received: by 2002:a7b:ce89:0:b0:3e1:bfc:f57b with SMTP id
q9-20020a7bce89000000b003e10bfcf57bmr1954363wmj.2.1676928328626;
Mon, 20 Feb 2023 13:25:28 -0800 (PST)
Received: from smtpclient.apple (212-39-89-88.ip.btc-net.bg. [212.39.89.88])
by smtp.gmail.com with ESMTPSA id
p15-20020a1c544f000000b003dc4a47605fsm2317000wmi.8.2023.02.20.13.25.27
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Mon, 20 Feb 2023 13:25:28 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN>
Date: Mon, 20 Feb 2023 23:25:26 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <842D303D-7BFD-4418-B3F4-A61CEB4FC903@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
<f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
<63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN>
<f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN>
<1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN>
<E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN>
<3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN>
<4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN>
<9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN>
<55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN>
<9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN>
<eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN>
<47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN>
<3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 1.5 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: > On 2023-02-20, at 11:14 PM, Pádraig Brady wrote: > > On
20/02/2023 19:35, George Valkov wrote: >>> On 2023-02-20, at 7:49 PM, Pádraig
Brady wrote: >>> >>> On 20/02/2023 15:02, George Valkov wr [...]
Content analysis details: (1.5 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (gvalkov[at]gmail.com)
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust
[209.85.128.48 listed in list.dnswl.org]
-0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[209.85.128.48 listed in wl.mailspike.net]
1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
[212.39.89.88 listed in dnsbl.sorbs.net]
X-Debbugs-Envelope-To: 61386
Cc: Gnulib bugs <bug-gnulib@HIDDEN>, Paul Eggert <eggert@HIDDEN>,
61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)
> On 2023-02-20, at 11:14 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 20/02/2023 19:35, George Valkov wrote:
>>> On 2023-02-20, at 7:49 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>>>=20
>>> On 20/02/2023 15:02, George Valkov wrote:
>>>> Hi Paul, the following tests fail after your patch:
>>>> FAIL: tests/split/filter.sh
>>>> FAIL: tests/split/b-chunk.sh
>>>> FAIL: tests/split/l-chunk.sh
>>>=20
>>> These look unrelated and may be due to some other change in your =
environment.
>>> Specifically lseek() is failing on /dev/null and returning:
>>> split: /dev/null: cannot determine file size
>> I deleted the lines that were introduced by the patch in unistd.in.h, =
then make clean; make -j 16 and ran all tests: back to 5 failed. Then I =
restored the deleted lines, rebuild and I got 9 failed tests again.
>=20
> Oh right sorry.
> I think I see what's happening.
> Since https://github.com/coreutils/gnulib/commit/4db8db34
> gnulib has been unconditionally replacing lseek() on macos.
> Now with SEEK_DATA undefined that replaced lseek()
> will run the code intended for BeOS.
> So either the lseek.m4 or lseek.c needs adjusting accordingly.
As I reported in one of my previous comments, there is no need for that =
hack in lseek.c. Either use the original lseek from macOS and make sure =
it doesn't return cached data, or disable sparse support until it gets =
fixed by Apple. The more people report it to them the higher the chance =
for them to take any action.
Let me know when you fix it.
Good luck!
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 20 Feb 2023 21:14:46 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 20 16:14:46 2023 Received: from localhost ([127.0.0.1]:53804 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pUDUk-0006DH-2P for submit <at> debbugs.gnu.org; Mon, 20 Feb 2023 16:14:46 -0500 Received: from mail-wm1-f42.google.com ([209.85.128.42]:53770) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pUDUi-0006D0-5E for 61386 <at> debbugs.gnu.org; Mon, 20 Feb 2023 16:14:45 -0500 Received: by mail-wm1-f42.google.com with SMTP id l6so1788270wms.3 for <61386 <at> debbugs.gnu.org>; Mon, 20 Feb 2023 13:14:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=jQiT+3VAtJnCIjPRz5NPpWNeu1cBOs1yKEAjgb3r6zA=; b=Hvgw6yq7zrN9hxK+iitdR3quwhwpxxx2nM9sZHUyxfXEh6HFmhfa2URitF4sGlGCMB MWHDA+CoVvW61wsCTbXvK97uAJ/MbEosBEGWoJgX/2kS9jp3yoKCeeMP7dWfp5W1c5sx 8X2BDhJfSFHHd4Kj7Zfea4aMvGTNgPxzy4HYH0hqoEk0zF58WzvrZcCPzEjwqlMgKyrT t+o02x8OzYmiV29mVVf1rfPLJpaRfnVoPJBKeay9GF3pu7yEBd6+BcSp1GU5W+vwBfxx CrBXWXVJq/1lSOp8HEt3RMsPzgN6W7F803v8JwYqnd82KWJjNKW86NfNiuz8qbWYd4HY GhPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jQiT+3VAtJnCIjPRz5NPpWNeu1cBOs1yKEAjgb3r6zA=; b=ZNmHxSUmEIPbx6irWMyfMdkWiKwmQktNURJqdXoShQWGjqndfRmf46vwKOpxKoNUdg LXwx2woN7VdM8REAQbgZ4ecn5vsBLp8DD+BlDixDX4WgFh6feerZlG8AZLhFp+WIRvo6 8eaRCv6P5frxv5jL+/5lePT9M7VMdjuV398RVqQMQIYOjZ4HlTy2vkVHXs2o+Tt0eO0N CcsqHOMAG6/RJQdslnAeHegaAtrhgozEjU9nbUly4rAp1sqhoQFXcUEBdnTZZAq2h0/y 6erUY7aHvWj4MroAyD1Om2KivWkwJtuXvyCS2od+vSPjrjEbBA3nFGURAFN1NCcwpjHI FWkA== X-Gm-Message-State: AO0yUKUgSjNadU5Vj/HoiRFAQwaNuIMJ6LZqoF5M2MoxyMcotsLBSnKN zdk5EvwC2wX9hORavrXFxZc= X-Google-Smtp-Source: AK7set+2Ft1GBF8Id4xbUO7m+UEQuh4hljJvGsMICVpiy3SrWnzjnMdUGJwxgB/mWwu+mJuazSraUA== X-Received: by 2002:a05:600c:1708:b0:3df:f71b:3f68 with SMTP id c8-20020a05600c170800b003dff71b3f68mr7183217wmn.39.1676927678114; Mon, 20 Feb 2023 13:14:38 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id c6-20020a05600c0ac600b003dc434b39c7sm2102746wmr.0.2023.02.20.13.14.37 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 20 Feb 2023 13:14:37 -0800 (PST) Message-ID: <3d96ebaf-3c26-9c2d-18d2-7dc57c84226f@HIDDEN> Date: Mon, 20 Feb 2023 21:14:36 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, Paul Eggert <eggert@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 20/02/2023 19:35, George Valkov wrote: > >> On 2023-02-20, at 7:49 PM, Pádraig Brady <P@HIDDEN> wrote: >> >> On 20/02/2023 15:02, George Valkov wrote: >>> Hi Paul, the following tests fail after your patch: >>> FAIL: tests/split/filter.sh >>> FAIL: tests/split/b-chunk.sh >>> FAIL: tests/split/l-chunk.sh >> >> These look unrelated and may be due to some other change in your environment. >> Specifically lseek() is failing on /dev/null and returning: >> split: /dev/null: cannot determine file size > > I deleted the lines that were introduced by the patch in unistd.in.h, then make clean; make -j 16 and ran all tests: back to 5 failed. Then I restored the deleted lines, rebuild and I got 9 failed tests again. Oh right sorry. I think I see what's happening. Since https://github.com/coreutils/gnulib/commit/4db8db34 gnulib has been unconditionally replacing lseek() on macos. Now with SEEK_DATA undefined that replaced lseek() will run the code intended for BeOS. So either the lseek.m4 or lseek.c needs adjusting accordingly. cheers, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 20 Feb 2023 19:35:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 20 14:35:22 2023
Received: from localhost ([127.0.0.1]:53692 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pUBwY-0003ap-0I
for submit <at> debbugs.gnu.org; Mon, 20 Feb 2023 14:35:22 -0500
Received: from mail-wr1-f54.google.com ([209.85.221.54]:47031)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pUBwT-0003aV-UV
for 61386 <at> debbugs.gnu.org; Mon, 20 Feb 2023 14:35:20 -0500
Received: by mail-wr1-f54.google.com with SMTP id t13so2211082wrv.13
for <61386 <at> debbugs.gnu.org>; Mon, 20 Feb 2023 11:35:17 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20210112; t=1676921712;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=VO/Pg+wnwiNSf1XNzWtJYyPdmdFuy2dOi1ckn0KCjO8=;
b=ooey9Xv1s0d8Gy4Yi8szrXPn0/UWDq37CCr3Xrp8hTbRHoT/q8i3BHxxcbbx6VHIJh
gW+z1xSuAJraUJAiXlYdUif5u83X333Fonp4dPSkdhXXGgteZQsfH8m1bAybzZfTabM/
YxFXGnFqbEF4V9n9RpjbEJF8JomiXeN6nCzd29xfY05I/fFFf6aMlbdQbPQFD8eAUOTQ
XhmfwVL5pruoSCkMYKkoyT/EVdLE7FVAFwJxVhkWgyTqvfZUIYThzCsPNFKkxfh+MWNC
mHzdmirJyGvxzYMF4n1GQStHYptPgCG1ERPpu6aERfCt5vZSHCgJoQMD6T09J4sZgyLg
25yQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112; t=1676921712;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=VO/Pg+wnwiNSf1XNzWtJYyPdmdFuy2dOi1ckn0KCjO8=;
b=K6WhynQNrMD5ibXJUK1foSg6ZovDMi7Pg861FEnbLsRLRZAwZkScDTu5Sa96brONSR
fJDm2cT2N5BqURboyvqBTglE5WtYOp79B4vNYuxJpXuGvIDyDi/T55hOhO5jc3n/qHro
LfyroPDyxRd+vr2b+JrTlEHNrcivY/SNleytz20ghiWbXa3iKMm/hzr+aTC+ZjvKHE03
2G7lvqU75/hjjikQYJou3wawJidxUy07pPAeIz/Cr+x2s+E827iD5D6brURmnB8Y6ywM
7QeDnF8Hw7wR5RjHDiO/arWGWfsKjO4m2xnpTayXGOXpnKiDmDqyMocTwiXwLmiOZc0L
3Ovg==
X-Gm-Message-State: AO0yUKWZPVwJdmt2A17F9AIfIeBolGRoZaFfWNKI8IC12Prvg+HQ3QTO
FWzm3pC1TGBdARo+X/UNAvU=
X-Google-Smtp-Source: AK7set9M7GmLCDrUh6uTHIfBP80Z+7Fo7tNiPapVtmXo+kTTs8ePkAv7VUl01K145qtwWC+7a0/05w==
X-Received: by 2002:a05:6000:1d0:b0:2c5:60e2:ed6e with SMTP id
t16-20020a05600001d000b002c560e2ed6emr1242253wrx.0.1676921711735;
Mon, 20 Feb 2023 11:35:11 -0800 (PST)
Received: from smtpclient.apple (212-39-89-88.ip.btc-net.bg. [212.39.89.88])
by smtp.gmail.com with ESMTPSA id
w15-20020a5d680f000000b002c5a6f4aa53sm3793126wru.105.2023.02.20.11.35.10
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Mon, 20 Feb 2023 11:35:10 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN>
Date: Mon, 20 Feb 2023 21:35:08 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <47AFA460-9EC0-4E72-92C4-EC628FDF557A@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
<f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
<63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN>
<f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN>
<1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN>
<E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN>
<3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN>
<4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN>
<9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN>
<55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN>
<9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN>
<eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 1.5 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: > On 2023-02-20, at 7:49 PM, Pádraig Brady wrote: > > On
20/02/2023 15:02, George Valkov wrote: >> Hi Paul, the following tests fail
after your patch: >> FAIL: tests/split/filter.sh >> FAIL: tests/ [...]
Content analysis details: (1.5 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (gvalkov[at]gmail.com)
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.0 SPF_PASS SPF: sender matches SPF record
1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
[212.39.89.88 listed in dnsbl.sorbs.net]
-0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[209.85.221.54 listed in wl.mailspike.net]
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust
[209.85.221.54 listed in list.dnswl.org]
X-Debbugs-Envelope-To: 61386
Cc: Gnulib bugs <bug-gnulib@HIDDEN>, Paul Eggert <eggert@HIDDEN>,
61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)
> On 2023-02-20, at 7:49 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 20/02/2023 15:02, George Valkov wrote:
>> Hi Paul, the following tests fail after your patch:
>> FAIL: tests/split/filter.sh
>> FAIL: tests/split/b-chunk.sh
>> FAIL: tests/split/l-chunk.sh
>=20
> These look unrelated and may be due to some other change in your =
environment.
> Specifically lseek() is failing on /dev/null and returning:
> split: /dev/null: cannot determine file size
I deleted the lines that were introduced by the patch in unistd.in.h, =
then make clean; make -j 16 and ran all tests: back to 5 failed. Then I =
restored the deleted lines, rebuild and I got 9 failed tests again.
>> FAIL: tests/cp/sparse-perf.sh
>=20
> As expected.
> I've a fix for this locally
> which leverages another patch to determine
> dynamically if we support SEEK_HOLE.
All right! Cheers mate!
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 20 Feb 2023 17:49:31 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 20 12:49:31 2023 Received: from localhost ([127.0.0.1]:53458 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pUAI7-0000hq-6y for submit <at> debbugs.gnu.org; Mon, 20 Feb 2023 12:49:31 -0500 Received: from mail-wm1-f46.google.com ([209.85.128.46]:56143) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pUAI3-0000hY-55 for 61386 <at> debbugs.gnu.org; Mon, 20 Feb 2023 12:49:29 -0500 Received: by mail-wm1-f46.google.com with SMTP id m23so1473868wms.5 for <61386 <at> debbugs.gnu.org>; Mon, 20 Feb 2023 09:49:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=6DRr4NGn/Zf17OiEhWxoFWjIcPploVM9mwm2b1i72tE=; b=N5ojJ8ALj0fFiXZLMyEJU60F0sZLJUFX2Kz0ZN1BuhYMphvIra4kxfxHrkTdsXXtoi GpJoK23Cd8tVTAPSii/2RJq/vLGaYZ5qCnOxXPVf2Vm6WvYK/pVR0zrbdZ0WT/6gjxK+ m8cYzlGhNDqTiV5QarlDsSomDScQxGg1WMs1kjioDDvx/9q52E0jrmjdm9+iYf97vzqD nyp9A6eLTE+3RYeXpvkuruaD2cZGthVt7mnIqOAkzjwCbHbHiL/aw0yamlu2i4ImvJs7 eKrw5ih4xnxl4YzzGHoHoHpNnuTRyCp7+BiZzyLQUlU0rfAyIOav/8sx30INkUdSlMI3 2Rew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6DRr4NGn/Zf17OiEhWxoFWjIcPploVM9mwm2b1i72tE=; b=F43HXo5jK23erbIt4yGGLxUC82aH72nPkWkTS+xFTuCC2qS5CiVpgm9IMYeSwLE00+ FTR2SCVFBlTTDFjU0YfK5tCg3Jtk0eaOQ4D8GuFv9ZMKVNRjnbPSrBLfI452kmiAKmTO ZyjnUMtQPmd45Eya5BWhokce2auZ4Wl8WUX/CI0BrNK3QckPk3U7oT3fjk8u7f2MIydw lphHHhnYBKU/kF9mqFPXRr/fqKKraluIJoVorRQcZoFIM/zs2Yoz3MgQV37JujMotUB7 XWTfZ1L1tijkcYqXP7r1UcM7yWTAvPP178d6KEHQA7V5D2NNLTrwPRpzy5pOop65w51r EC/A== X-Gm-Message-State: AO0yUKUNN3TYDLc4ZOfC3b02QQCaB+eo4CwsW6CVrKQDXmxW6OsA8Fst QzGXjhhlIe4pCWv6OWl8Jg8= X-Google-Smtp-Source: AK7set+OHau+Q22kQydkh2L5/b4679eaPJ0Y+H7zw8FwpSOFeQpuGitEJPmxa1lq54dPCsJ1Fp40iQ== X-Received: by 2002:a05:600c:80f:b0:3df:e46f:c226 with SMTP id k15-20020a05600c080f00b003dfe46fc226mr745363wmp.16.1676915361112; Mon, 20 Feb 2023 09:49:21 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id i21-20020a1c5415000000b003e208cec49bsm2074096wmb.3.2023.02.20.09.49.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 20 Feb 2023 09:49:20 -0800 (PST) Message-ID: <eee0e3c9-aebe-7fa1-7aa4-343179e46fa8@HIDDEN> Date: Mon, 20 Feb 2023 17:49:19 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, Paul Eggert <eggert@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 20/02/2023 15:02, George Valkov wrote: > Hi Paul, the following tests fail after your patch: > FAIL: tests/split/filter.sh > FAIL: tests/split/b-chunk.sh > FAIL: tests/split/l-chunk.sh These look unrelated and may be due to some other change in your environment. Specifically lseek() is failing on /dev/null and returning: split: /dev/null: cannot determine file size > FAIL: tests/cp/sparse-perf.sh As expected. I've a fix for this locally which leverages another patch to determine dynamically if we support SEEK_HOLE. cheers, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 20 Feb 2023 15:03:09 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 20 10:03:09 2023 Received: from localhost ([127.0.0.1]:53265 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pU7h6-0008GZ-VK for submit <at> debbugs.gnu.org; Mon, 20 Feb 2023 10:03:09 -0500 Received: from mail-wm1-f45.google.com ([209.85.128.45]:42982) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pU7h5-0008G4-3V for 61386 <at> debbugs.gnu.org; Mon, 20 Feb 2023 10:03:08 -0500 Received: by mail-wm1-f45.google.com with SMTP id iv11-20020a05600c548b00b003dc52fed235so1072734wmb.1 for <61386 <at> debbugs.gnu.org>; Mon, 20 Feb 2023 07:03:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1676905381; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=iz97zm1jDWgJxgn12XlFs4sDvQYs1e4xUEx3IqXC1Zo=; b=RlhWyTczLsELG9Z0P/i2pDVGoiW/PRcFed9C5mZDC7euygTqgmA0+wa3G8LcYNOJLd X0c8bYcM23LrqxdsWI0iVdCxWSOstNX1A4mtChD1CvbOBWaMonXAIzNJRk3ksT0b1AzS x0VJqK9ZrWQmZ+lmkPQHL1jZvABOWhnSD3+JjGU5nohi9Ufypx1ETWArC9qlVWR1lA/G bmqzC2gAUKxwhMXSZ9BvL+0fEwKVS4C8XVeH3gQVkinJKMqzdMAnEq2Ib4dgox1v+6K2 UjxLmoi37N2fthYOSQAuLVWMrdPZr4tHmaGvc+NY4VKl3I98onZHm5MqMo9PRfkD1UA5 eRkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1676905381; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iz97zm1jDWgJxgn12XlFs4sDvQYs1e4xUEx3IqXC1Zo=; b=hldYZhIOsXRVTXfXvkWlkaBQpz/somj/A8vQ9W361W0QAl0Fyj1bpBR/eUwf6tHTl9 dRtsDmwYC1RNMy6w2pM1M5eHKZcz27LdVV5DBfpSjLddBTsCwpjcHYQle1+H+/+Vq5uX ZmVXW/1dqirkUay3cFnQ292YK5Hwg2or68gXp1o2bZhp2Ej6U5CLi7b+PkcV5EWcqsev ZEwIa6Ajwd0PjAjUn+aiEYXe+0f7HQ7+vSf4DaRaCwvSWm8N7WyQFA05+PFt6XJkhPlJ uRV5DepGHHnquWV9TkhzPh2BowAzyQCZIvZgybOIdRuuT2xbUGyQGxB3dRKxKZZfzg+U Ke8Q== X-Gm-Message-State: AO0yUKVvLt5ejT8SNfGGDOAMgB8EzAXHkZXRZTHVmZ1Ae0e+zhVkZII8 kWia5ewt6t8Qt4IAlqMm8sk= X-Google-Smtp-Source: AK7set+IUGHJShjKJvPzKVjn5laFkB9INmoH/pywgKjBPFEsMUEnjib/wApyKlDYa7dUoMk9xjN2pQ== X-Received: by 2002:a1c:7718:0:b0:3e2:17c2:fc8 with SMTP id t24-20020a1c7718000000b003e217c20fc8mr1108155wmi.2.1676905381062; Mon, 20 Feb 2023 07:03:01 -0800 (PST) Received: from smtpclient.apple (212-39-89-88.ip.btc-net.bg. [212.39.89.88]) by smtp.gmail.com with ESMTPSA id a6-20020a05600c348600b003df5be8987esm9927349wmq.20.2023.02.20.07.02.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Feb 2023 07:02:59 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> Date: Mon, 20 Feb 2023 17:02:58 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <9B29FA77-97E4-4FD4-9D95-9772EC097625@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> To: Paul Eggert <eggert@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Paul, the following tests fail after your patch: FAIL: tests/split/filter.sh FAIL: tests/split/b-chunk.sh FAIL: tests/split/l-chunk.sh FAIL: tests/cp/sparse-perf.sh Before patch https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-05-95f4ee0577dd836de523f46999777fbbbe9d2772-ori.txt Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (gvalkov[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.128.45 listed in list.dnswl.org] 1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server [212.39.89.88 listed in dnsbl.sorbs.net] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.128.45 listed in wl.mailspike.net] X-Debbugs-Envelope-To: 61386 Cc: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>, 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.5 (/) Hi Paul, the following tests fail after your patch: FAIL: tests/split/filter.sh FAIL: tests/split/b-chunk.sh FAIL: tests/split/l-chunk.sh FAIL: tests/cp/sparse-perf.sh Before patch = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-05= -95f4ee0577dd836de523f46999777fbbbe9d2772-ori.txt After patch = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-05= -95f4ee0577dd836de523f46999777fbbbe9d2772-patch.txt Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 20 Feb 2023 10:21:39 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 20 05:21:39 2023 Received: from localhost ([127.0.0.1]:50559 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pU3Ig-0003Mn-TD for submit <at> debbugs.gnu.org; Mon, 20 Feb 2023 05:21:39 -0500 Received: from mail-wm1-f44.google.com ([209.85.128.44]:43814) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pU3Ie-0003Ma-Vl for 61386 <at> debbugs.gnu.org; Mon, 20 Feb 2023 05:21:37 -0500 Received: by mail-wm1-f44.google.com with SMTP id bg25-20020a05600c3c9900b003e21af96703so1932wmb.2 for <61386 <at> debbugs.gnu.org>; Mon, 20 Feb 2023 02:21:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=h0Vghq3eQAN9pmfYjHzxCnN2sDis3bOoN6VuGGpW4mk=; b=oJ28iw73Ua80kwVI+VM6YT2CLtK1ORlOYxI/qImvC+n+0opMkC/vTi6qrXjUmf81W9 bV9m3CasLfLYVkl66U+7JOctp9vL5lrkyuQE9mXAj5T4qeq+PHsj25ahJ16/7L2WE8uP p2IjiGvXXm5mxugrTbHB+yoDQ8PaR4zkjArKKQNnHjA1/1aPcoIbbe8Qz3L4k7leI8GH 9AALmUnVBwFsYUHIzLbcLBQ+CFC5Ihz1pwyK4uKnlYSHEtHjQigDkfhtPq+Eh3YJAwZy haMmMGYITFgXmwUPHwFRyyTVk9XPMfuK+HShC/lXoWDn7TwXBth8qyxwO+01psxREV0p 933A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=h0Vghq3eQAN9pmfYjHzxCnN2sDis3bOoN6VuGGpW4mk=; b=d680sR1O+NjJAWDA9zmXPKRAIMSPykv5NXbI8/xFlY6Y4oJDa6vr9VSgAEot0SdWX3 Z3b2QyPTmHB8PirtHheQUno4Qi7eJZqAedUciR5DlpA/vG8TAD5+25YqkMlErG7R8NVE spZWb16Xrytq0PuBVEEtGfFq6y7FgbS0jUVzxG3Rwr71uRKTlMr2CZF55BAa5caPY64j cqSZJnOrOfWXPcAx3Ulffxcak9wFSjs5vpXpoYj+qAGAJKQCEnYaeJkm1FaRChBEZ47G H3CQ5ic0YtP5cInx1X3F8d6ncSchf12+0aqeZaK9Mkw05/p2yZewWf4oRGzWy1c/Dvg9 Q8tg== X-Gm-Message-State: AO0yUKUYy0lRWCuVpHhq7Ili2JVTa6mSMg2TlQjDQQ65enS6TLZWXh/G NUVwgA/SAwx0Xp4QNSru14s= X-Google-Smtp-Source: AK7set83raIzDDIzHjpg5mOgpXegk+TKQs7RwVZ+dAntIDJrCRG+rZ2rNhw8tZHfC9BATAp1IbdxlQ== X-Received: by 2002:a05:600c:1d1a:b0:3e0:b1:c12d with SMTP id l26-20020a05600c1d1a00b003e000b1c12dmr418791wms.1.1676888490741; Mon, 20 Feb 2023 02:21:30 -0800 (PST) Received: from smtpclient.apple (212-5-158-16.ip.btc-net.bg. [212.5.158.16]) by smtp.gmail.com with ESMTPSA id o10-20020a1c750a000000b003dfe8c4c497sm9053493wmc.39.2023.02.20.02.21.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Feb 2023 02:21:29 -0800 (PST) From: George Valkov <gvalkov@HIDDEN> Message-Id: <177B5813-ABE6-4D74-8270-C133811484FF@HIDDEN> Content-Type: multipart/mixed; boundary="Apple-Mail=_E7BCDED0-4500-4E0D-AE11-C517446B8CA4" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Date: Mon, 20 Feb 2023 12:21:27 +0200 In-Reply-To: <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> To: Paul Eggert <eggert@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: > On 2023-02-19, at 8:08 AM, Paul Eggert wrote: > > George, given what you've written I suppose we should give up the idea of copying sparse files efficiently on macOS (and on FreeBSD 13.0-RELEASE, [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server [212.5.158.16 listed in dnsbl.sorbs.net] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (gvalkov[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.128.44 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.128.44 listed in wl.mailspike.net] X-Debbugs-Envelope-To: 61386 Cc: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>, 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.5 (/) --Apple-Mail=_E7BCDED0-4500-4E0D-AE11-C517446B8CA4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 2023-02-19, at 8:08 AM, Paul Eggert <eggert@HIDDEN> wrote: >=20 > George, given what you've written I suppose we should give up the idea = of copying sparse files efficiently on macOS (and on FreeBSD = 13.0-RELEASE, as it has a similar bug with SEEK_HOLE and SEEK_DATA), in = cases where fclonefileat does not work. I agree, Paul. I just ran some benchmarks: the time it takes to call = msync unconditionally hurts the performance: 502 us if the memory view = is already synced seems fine, however 28177-36585 us whenever a sync is = required is more than the time it takes to performing a full-copy = 7042-21528 us. > Please try the attached Gnulib patch; it uses your idea of disabling = SEEK_HOLE and SEEK_DATA on macOS, except on steroids because it disables = these two macros everywhere, for all Gnulib-using applications, and it = also disables them on FreeBSD < 14. For coreutils you need only this = patch's change to lib/unistd.in.h. If this works for you I suppose we = can install it into Gnulib and propagate that into coreutils etc. What is the correct way to apply your patch? I tried git apply 0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch=20 error: patch failed: ChangeLog:1 error: ChangeLog: patch does not apply error: doc/posix-functions/lseek.texi: No such file or directory error: lib/unistd.in.h: wrong type Then I made the changes to lib/unistd.in.h manually, and ran the tests: * before: corrupted * after: fixed Regarding FreeBSD, I have a FreeBSD 13.1-RELEASE VM running on Hyper-V, = so I compiled and ran my samples: m.c, d.c: the copy was correct, = however d.c reports that there are no segments to skip, which means that = the file created by m.c using mmap is not sparse even though it should = contain one blank segment and be sparse. Next I installed a fresh copy = of FreeBSD 13.0-RELEASE and observed the same behaviour. Both = installations use ZFS and the tests were ran as root. If you know what conditions are required to reproduce the issue on = FreeBSD, let me know and I will test it for you. Otherwise I am not sure = if we need to make any changes to FreeBSD, since I did not observe = anything wrong there. Is it ok if I continue testing on FreeBSD 13.1, because the 13.0 I just = installed lacks any configuration and is quite uncomfortable to use. It = also rejects my password over SSH, so I=E2=80=99m forced to use the = console. I can test pretty much anything that runs on Hyper-V. > Also, you reported the bug against macOS 12.6. Can you check whether = the same bug occurs on macOS 13? If it's still there I suppose the = attached patch will need to be updated since it guesses the bug is fixed = in macOS 13. Yes it does. I just ran my samples under macOS Internet recovery, which = reports as Ventura with kernel compiled on the 30th of January 2023. See = the attached log: --Apple-Mail=_E7BCDED0-4500-4E0D-AE11-C517446B8CA4 Content-Disposition: attachment; filename=ventura-internet-recovery.txt Content-Type: text/plain; x-unix-mode=0644; name="ventura-internet-recovery.txt" Content-Transfer-Encoding: quoted-printable uname -a Darwin gMac.lan 22.3.0 Darwin Kernel Version 22.3.0: Mon Jan 30 20:42:11 = PST 2023; root:xnu-8792.81.3~2/RELEASE_X86_64 x86_64 ./m=20 src 3 dst 4 00001000 skip total bytes copied 1a45640 / 1a46640 1a46640 100535 us /gfc cc1-mmap=20 16d835378ab973a114082a585cc76958bdbccec0 cc1-mmap ./d src 3 dst 4 c a46640 p 1a46640 h 1a46640 d 1000000 total bytes copied a46640 / 1a46640 7117 us ./gfc cc1-sparse=20 75e6f6cb303cb5d3909c6d7830c417fc6ed658c3 cc1-sparse ./n=20 dst 3 MS_ASYNC 1 MS_INVALIDATE 2 MS_SYNC 16 size bytes 1a46640 msync ok 43770 us ./d=20 src 3 dst 4 c 1000 p 1000 h 1000 d 0 c 1a44640 p 1a46640 h 1a46640 d 2000 total bytes copied 1a45640 / 1a46640 41322 us ./gfc cc1-sparse=20 16d835378ab973a114082a585cc76958bdbccec0 cc1-sparse ./m=20 src 3 dst 4 00001000 skip total bytes copied 1a45640 / 1a46640 1a46640 59198 us ./d=20 src 3 dst 4 c a46640 p 1a46640 h 1a46640 d 1000000 total bytes copied a46640 / 1a46640 12144 us ./gfc cc1-sparse=20 75e6f6cb303cb5d3909c6d7830c417fc6ed658c3 cc1-sparse ./n=20 dst 3 MS_ASYNC 1 MS_INVALIDATE 2 MS_SYNC 16 size bytes 1a46640 msync ok 27462 us ./d=20 src 3 dst 4 c 1000 p 1000 h 1000 d 0 c 1a44640 p 1a46640 h 1a46640 d 2000 total bytes copied 1a45640 / 1a46640 32125 us ./gfc cc1-sparse=20 16d835378ab973a114082a585cc76958bdbccec0 cc1-sparse --Apple-Mail=_E7BCDED0-4500-4E0D-AE11-C517446B8CA4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > I'll cc this email to bug-gnulib to give them a heads-up. (Gnulib = readers can see <https://bugs.gnu.org/61386> for context.) Thanks, I=E2=80=99ll check it. > Really, Apple should fix this serious data corruption bug in APFS and = macOS. > <0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch> I agree, I opened a new ticket with Apple on 2023-02-16. Both via = e-mail, and https://security.apple.com/reports/OE1928608366012 This was at the same time I reported about mmap and msync here. I got = this replay on the following day: > Thanks for the additional information. We are reviewing it and will = follow up if we have any questions or need additional details. Feel free to write them at product-security@HIDDEN and include this = line OE0928602070811 - please include this ID in replies to this thread. Introduce your self as part of GNU/coreutils and let them know it has = been 18 months since the issue was reported privately to them, so = we=E2=80=99ve gone public now. CC me <same as gmail> at abv dot bg. Georgi Valkov httpstorm.com nano RTOS --Apple-Mail=_E7BCDED0-4500-4E0D-AE11-C517446B8CA4--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 19 Feb 2023 16:31:48 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Feb 19 11:31:48 2023 Received: from localhost ([127.0.0.1]:49568 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pTmbL-0004R2-To for submit <at> debbugs.gnu.org; Sun, 19 Feb 2023 11:31:48 -0500 Received: from mail-wm1-f46.google.com ([209.85.128.46]:35750) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pTmbK-0004Qr-Ab for 61386 <at> debbugs.gnu.org; Sun, 19 Feb 2023 11:31:46 -0500 Received: by mail-wm1-f46.google.com with SMTP id p18-20020a05600c359200b003dc57ea0dfeso646378wmq.0 for <61386 <at> debbugs.gnu.org>; Sun, 19 Feb 2023 08:31:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=adqKi/PGk9Hv797oedcUQW6WA9VWv/y1wG0AerlrT9M=; b=dwkDcNLIF26ZkfBzCMTU0SWrXcVCnYWOsuG+J7N+GjpeW/E0EpCHQ/2GxSeUknOzTh 9rcACAYV2HQHmXq5XJOlkf02tOb9IYrIXdezeIPNZGVT1Ypgrmu4ztoGgWclOotjwTWU NmKC/nBBFj6vmihHb2mRsxmhAGs6DYrIcLwMs5AqIz8+byONYhlNKe43FOfFvqSnBqJm g/6SshKAzRkonaYHsiQt89FGc1c/KkDD9myODR8vJDKbl0YewSX2rlZUT3YWAqfwwFXf f5XboTmJDibjKfx4aiSUe3a5sk8IDz+ZAowGpBKuuc+FyAFQEV8ECfBb4ce73Xr5l012 t5Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=adqKi/PGk9Hv797oedcUQW6WA9VWv/y1wG0AerlrT9M=; b=zSZck205/4wa7mSCJQEL5CHjexH4+K5eIn6qK/RmW0H+ibHn0xrAtHxdEfDF4kWxyt /O3rlBa8OGcqne6abPmQPxdeoP8VQ9UtNtYIgckin/FWB2yc4RmHgVHFX4QnO1Hkzbvm +Gzud6DfH+1Ku/d8+lEbxhFiIt7X+ysWdwEHie+XCTdBUfuc1qS9YGJNxnsA2kHGSSdW wM/4q6NOhYxnFgV6hP+t+nrzKHMbnucRc7+0rEl/Ke/aASHui/5YmHZqPkVIA9HNmsKo DCrc3qfNlAryNZbnHvts5J0Te4Z1H/Y4AWpy7Ixza/xu4oCtmUUMc4/Dm2bSn+fWX9Sn k/vg== X-Gm-Message-State: AO0yUKW8CJfmFBQH+qH12NiPlYKo1R/gUL1lu+VUIJr0mxmCixz0zkRg kYQAz2x9rEBMCLIlI1wrmuk= X-Google-Smtp-Source: AK7set9t4KornNV1L/oF/x0HQScs3J1je7IS0CwcpZDfL2kQ9exym2vTqA+UGA3lUvfHGl9AQydvfA== X-Received: by 2002:a05:600c:3d8f:b0:3dd:1c46:b92 with SMTP id bi15-20020a05600c3d8f00b003dd1c460b92mr5158803wmb.16.1676824300147; Sun, 19 Feb 2023 08:31:40 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id he8-20020a05600c540800b003e208cec49bsm14069261wmb.3.2023.02.19.08.31.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 19 Feb 2023 08:31:39 -0800 (PST) Message-ID: <3071f905-0f0c-db7c-a032-d5974e32ef8e@HIDDEN> Date: Sun, 19 Feb 2023 16:31:37 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 19/02/2023 06:08, Paul Eggert wrote: > George, given what you've written I suppose we should give up the idea > of copying sparse files efficiently on macOS (and on FreeBSD > 13.0-RELEASE, as it has a similar bug with SEEK_HOLE and SEEK_DATA), in > cases where fclonefileat does not work. > > Please try the attached Gnulib patch; it uses your idea of disabling > SEEK_HOLE and SEEK_DATA on macOS, except on steroids because it disables > these two macros everywhere, for all Gnulib-using applications, and it > also disables them on FreeBSD < 14. For coreutils you need only this > patch's change to lib/unistd.in.h. If this works for you I suppose we > can install it into Gnulib and propagate that into coreutils etc. > > Also, you reported the bug against macOS 12.6. Can you check whether the > same bug occurs on macOS 13? If it's still there I suppose the attached > patch will need to be updated since it guesses the bug is fixed in macOS 13. > > I'll cc this email to bug-gnulib to give them a heads-up. (Gnulib > readers can see <https://bugs.gnu.org/61386> for context.) > > Really, Apple should fix this serious data corruption bug in APFS and macOS. Yes I concur. I'll adjust the coreutils tests to avoid using SEEK_DATA if not available in a more dynamic manner. thank you, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 19 Feb 2023 06:09:05 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Feb 19 01:09:05 2023 Received: from localhost ([127.0.0.1]:45466 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pTcsi-0008Sa-MN for submit <at> debbugs.gnu.org; Sun, 19 Feb 2023 01:09:05 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:52646) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pTcsg-0008S4-Ez for 61386 <at> debbugs.gnu.org; Sun, 19 Feb 2023 01:09:03 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 5A53F160040; Sat, 18 Feb 2023 22:08:56 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id z3f3RUG-R0Hp; Sat, 18 Feb 2023 22:08:55 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 112CC160044; Sat, 18 Feb 2023 22:08:55 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 112CC160044 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676786935; bh=70WqYBVcGY7ifnwTaSZFEICdljOsgoaS758HcIWJ1BU=; h=Content-Type:Message-ID:Date:MIME-Version:Subject:To:From; b=ocMcAlJJ3thGb11YMdd3RNJnKD2Yrv+caR6D6WW+ZPWJTy2WQo3+6Cgq+674TzJbI +Uv6UEskWnR3W+Q0BpE1S8E9mfOAMM74ZAjQzxWL23GM+0P7Wj4t8MCvZSf7ooQ4yk UeDseD9P5fS6aN96rFK52bYKIpI44EhLkEjWdDG8= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id vfZC40vkkOZO; Sat, 18 Feb 2023 22:08:54 -0800 (PST) Received: from [192.168.0.205] (ip72-206-2-24.fv.ks.cox.net [72.206.2.24]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 891F2160040; Sat, 18 Feb 2023 22:08:54 -0800 (PST) Content-Type: multipart/mixed; boundary="------------hpr00TrCyz0Zct10t15skc20" Message-ID: <55b39c6d-e0be-5abd-0242-d35143d4836f@HIDDEN> Date: Sun, 19 Feb 2023 00:08:53 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN> <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> From: Paul Eggert <eggert@HIDDEN> In-Reply-To: <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN> X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN>, 61386 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) This is a multi-part message in MIME format. --------------hpr00TrCyz0Zct10t15skc20 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit George, given what you've written I suppose we should give up the idea of copying sparse files efficiently on macOS (and on FreeBSD 13.0-RELEASE, as it has a similar bug with SEEK_HOLE and SEEK_DATA), in cases where fclonefileat does not work. Please try the attached Gnulib patch; it uses your idea of disabling SEEK_HOLE and SEEK_DATA on macOS, except on steroids because it disables these two macros everywhere, for all Gnulib-using applications, and it also disables them on FreeBSD < 14. For coreutils you need only this patch's change to lib/unistd.in.h. If this works for you I suppose we can install it into Gnulib and propagate that into coreutils etc. Also, you reported the bug against macOS 12.6. Can you check whether the same bug occurs on macOS 13? If it's still there I suppose the attached patch will need to be updated since it guesses the bug is fixed in macOS 13. I'll cc this email to bug-gnulib to give them a heads-up. (Gnulib readers can see <https://bugs.gnu.org/61386> for context.) Really, Apple should fix this serious data corruption bug in APFS and macOS. --------------hpr00TrCyz0Zct10t15skc20 Content-Type: text/x-patch; charset=UTF-8; name="0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch" Content-Disposition: attachment; filename="0001-lseek-avoid-SEEK_HOLE-bugs-in-FreeBSD-macOS.patch" Content-Transfer-Encoding: base64 RnJvbSAwMDI0Yjg0NjBkZmU1ZTEzM2Y5ZTBkNjNlZDRmNzUzMTRkZWEyZjgyIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBTdW4sIDE5IEZlYiAyMDIzIDAwOjA1OjI0IC0wNjAwClN1YmplY3Q6IFtQQVRD SF0gbHNlZWs6IGF2b2lkIFNFRUtfSE9MRSBidWdzIGluIEZyZWVCU0QsIG1hY09TCk1JTUUt VmVyc2lvbjogMS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD1VVEYtOApD b250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0CgpUaGlzIGF0dGVtcHRzIHRvIGZpeCA8 aHR0cHM6Ly9idWdzLmdudS5vcmcvNjEzODY+LCBhIGJ1ZyBpbiBHTlUgY3AKY2F1c2VkIGJ5 IGEgc2VyaW91cyBkYXRhIGNvcnJ1cHRpb24gYnVnIGluIEZyZWVCU0QgYW5kIG1hY09TLgoq IGRvYy9wb3NpeC1mdW5jdGlvbnMvbHNlZWsudGV4aTogTWVudGlvbiB0aGUgYnVnLgoqIGxp Yi91bmlzdGQuaW4uaCAoU0VFS19EQVRBLCBTRUVLX0hPTEUpOiBVbmRlZiBpbiBtYWNPUyA8 IDEzIGFuZApGcmVlQlNEIDwgMTQuICBGcmVlQlNEIGZpeGVkIHRoZSBidWcgc29tZXRpbWUg ZHVyaW5nIEZyZWVCU0QgMTMKPGh0dHBzOi8vYnVncy5mcmVlYnNkLm9yZy8yNTYyMDU+LCBz byB0aGUgIkZyZWVCU0QgPCAxNCIgaXMKY29uc2VydmF0aXZlLiAgSXTigJlzIHVua25vd24g d2hlbiAob3IgaW5kZWVkIHdoZXRoZXIpIEFwcGxlIGZpeGVkCm1hY09TIGJ1dCB0aGlzIHBh dGNoIGd1ZXNzZXMgbWFjT1MgMTMuCi0tLQogQ2hhbmdlTG9nICAgICAgICAgICAgICAgICAg ICAgIHwgMTIgKysrKysrKysrKysrCiBkb2MvcG9zaXgtZnVuY3Rpb25zL2xzZWVrLnRleGkg fCAgNSArKysrKwogbGliL3VuaXN0ZC5pbi5oICAgICAgICAgICAgICAgIHwgMTIgKysrKysr KysrKysrCiAzIGZpbGVzIGNoYW5nZWQsIDI5IGluc2VydGlvbnMoKykKCmRpZmYgLS1naXQg YS9DaGFuZ2VMb2cgYi9DaGFuZ2VMb2cKaW5kZXggYzFjYTYxMDU0OC4uYzhlOTcyNjkxNiAx MDA2NDQKLS0tIGEvQ2hhbmdlTG9nCisrKyBiL0NoYW5nZUxvZwpAQCAtMSwzICsxLDE1IEBA CisyMDIzLTAyLTE4ICBQYXVsIEVnZ2VydCAgPGVnZ2VydEBjcy51Y2xhLmVkdT4KKworCWxz ZWVrOiBhdm9pZCBTRUVLX0hPTEUgYnVncyBpbiBGcmVlQlNELCBtYWNPUworCVRoaXMgYXR0 ZW1wdHMgdG8gZml4IDxodHRwczovL2J1Z3MuZ251Lm9yZy82MTM4Nj4sIGEgYnVnIGluIEdO VSBjcAorCWNhdXNlZCBieSBhIHNlcmlvdXMgZGF0YSBjb3JydXB0aW9uIGJ1ZyBpbiBGcmVl QlNEIGFuZCBtYWNPUy4KKwkqIGRvYy9wb3NpeC1mdW5jdGlvbnMvbHNlZWsudGV4aTogTWVu dGlvbiB0aGUgYnVnLgorCSogbGliL3VuaXN0ZC5pbi5oIChTRUVLX0RBVEEsIFNFRUtfSE9M RSk6IFVuZGVmIGluIG1hY09TIDwgMTMgYW5kCisJRnJlZUJTRCA8IDE0LiAgRnJlZUJTRCBm aXhlZCB0aGUgYnVnIHNvbWV0aW1lIGR1cmluZyBGcmVlQlNEIDEzCisJPGh0dHBzOi8vYnVn cy5mcmVlYnNkLm9yZy8yNTYyMDU+LCBzbyB0aGUgIkZyZWVCU0QgPCAxNCIgaXMKKwljb25z ZXJ2YXRpdmUuICBJdOKAmXMgdW5rbm93biB3aGVuIChvciBpbmRlZWQgd2hldGhlcikgQXBw bGUgZml4ZWQKKwltYWNPUyBidXQgdGhpcyBwYXRjaCBndWVzc2VzIG1hY09TIDEzLgorCiAy MDIzLTAyLTE4ICBCcnVubyBIYWlibGUgIDxicnVub0BjbGlzcC5vcmc+CiAKIAljb25maWdt YWtlOiBBZGQgc3VwcG9ydCBmb3IgJGJ1aWxkX29zICE9ICRob3N0X29zLgpkaWZmIC0tZ2l0 IGEvZG9jL3Bvc2l4LWZ1bmN0aW9ucy9sc2Vlay50ZXhpIGIvZG9jL3Bvc2l4LWZ1bmN0aW9u cy9sc2Vlay50ZXhpCmluZGV4IDJmOGUyYjU4NzcuLjM0NzA1MjRiMTIgMTAwNjQ0Ci0tLSBh L2RvYy9wb3NpeC1mdW5jdGlvbnMvbHNlZWsudGV4aQorKysgYi9kb2MvcG9zaXgtZnVuY3Rp b25zL2xzZWVrLnRleGkKQEAgLTM3LDQgKzM3LDkgQEAgSVJJWCA2LjUuCiBAaXRlbQogU29t ZSBzeXN0ZW1zIGRvIG5vdCBzdXBwb3J0IEBjb2Rle1NFRUtfREFUQX0gYW5kIEBjb2Rle1NF RUtfSE9MRX06CiBBSVgsIEhQLVVYLCBNaWNyb3NvZnQgV2luZG93cywgTmV0QlNELCBPcGVu QlNELgorQGl0ZW0KK1NvbWUgc3lzdGVtcyBoYXZlIGEgYnVnZ3kgQGNvZGV7U0VFS19EQVRB fSBhbmQgQGNvZGV7U0VFS19IT0xFfSwKK2FuZCBHbnVsaWIgd29ya3MgYXJvdW5kIHRoZSBw cm9ibGVtIHZpYSBAY29kZXsjdW5kZWYgU0VFS19EQVRBfQorYW5kIEBjb2RleyN1bmRlZiBT RUVLX0hPTEV9OgorRnJlZUJTRCAxMywgbWFjT1MgMTIuCiBAZW5kIGl0ZW1pemUKZGlmZiAt LWdpdCBhL2xpYi91bmlzdGQuaW4uaCBiL2xpYi91bmlzdGQuaW4uaAppbmRleCBiZmM1MDFl NWE3Li40N2IzMmViNDQ1IDEwMDY0NAotLS0gYS9saWIvdW5pc3RkLmluLmgKKysrIGIvbGli L3VuaXN0ZC5pbi5oCkBAIC00MCw2ICs0MCwxOCBAQAogIyB1bmRlZiBfR0xfSU5DTFVESU5H X1VOSVNURF9ICiAjZW5kaWYKIAorLyogQXZvaWQgbHNlZWsgYnVncyBpbiBGcmVlQlNELCBt YWNPUyA8aHR0cHM6Ly9idWdzLmdudS5vcmcvNjEzODY+LiAgKi8KKyNpZiBkZWZpbmVkIF9f RnJlZUJTRF9fICYmIF9fRnJlZUJTRF9fIDwgMTQKKyMgdW5kZWYgU0VFS19EQVRBCisjIHVu ZGVmIFNFRUtfSE9MRQorI2VsaWYgKGRlZmluZWQgX19BUFBMRV9fICYmIGRlZmluZWQgX19N QUNIX18gXAorICAgICAgICYmICghZGVmaW5lZCBNQUNfT1NfWF9WRVJTSU9OX01JTl9SRVFV SVJFRCBcCisgICAgICAgICAgIHx8IE1BQ19PU19YX1ZFUlNJT05fTUlOX1JFUVVJUkVEIDwg MTMwMDAwKSkKKyMgaW5jbHVkZSA8c3lzL2ZjbnRsLmg+IC8qIEl0IGFsc28gZGVmaW5lcyB0 aGUgdHdvIG1hY3Jvcy4gICovCisjIHVuZGVmIFNFRUtfREFUQQorIyB1bmRlZiBTRUVLX0hP TEUKKyNlbmRpZgorCiAvKiBHZXQgYWxsIHBvc3NpYmxlIGRlY2xhcmF0aW9ucyBvZiBnZXRo b3N0bmFtZSgpLiAgKi8KICNpZiBAR05VTElCX0dFVEhPU1ROQU1FQCAmJiBAVU5JU1REX0hf SEFWRV9XSU5TT0NLMl9IQCBcCiAgICYmICFkZWZpbmVkIF9HTF9JTkNMVURJTkdfV0lOU09D SzJfSAotLSAKMi4zOS4yCgo= --------------hpr00TrCyz0Zct10t15skc20--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 16 Feb 2023 23:41:33 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 16 18:41:33 2023 Received: from localhost ([127.0.0.1]:37987 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pSnsb-00044v-Ep for submit <at> debbugs.gnu.org; Thu, 16 Feb 2023 18:41:33 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:54754) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pSnsY-00044c-VL for 61386 <at> debbugs.gnu.org; Thu, 16 Feb 2023 18:41:31 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id CBAFE160080; Thu, 16 Feb 2023 15:41:24 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id ngnQ2VlpKkio; Thu, 16 Feb 2023 15:41:24 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 1480F160085; Thu, 16 Feb 2023 15:41:24 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 1480F160085 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676590884; bh=Mo2cMpL68NLxOMpzCxetgYn3QvNpchhVzaaxORDCrY4=; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type: Content-Transfer-Encoding; b=EPvdlSXVUcfkbuA0chLDxZo2zyPMP8d5gZj0FEfaAapHXdAfdVTodlRrR9vI5rdRd +xRO0OF1Q5ar6yf2xjtdFzZTEPmbiK35gDRS1nD+BPPWxb9uSTLOKki1rQUnbzKrkx D/a489pkn+M7tW7+0bowfCTitiiZz+HQ3J8ShgGw= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 3ORs3P97_cws; Thu, 16 Feb 2023 15:41:23 -0800 (PST) Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id EDC2F160080; Thu, 16 Feb 2023 15:41:23 -0800 (PST) Message-ID: <25189d09-b160-234f-d388-159db7b99f22@HIDDEN> Date: Thu, 16 Feb 2023 15:41:23 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.2 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <45adda7f-4654-e186-7052-c14612415c22@HIDDEN> <354b1398-3c1c-9f83-1209-837825b9a85f@HIDDEN> <acc021fa-5983-2026-63fd-94073902b756@HIDDEN> <528eb43d-5ef7-2745-577d-8ab7ccc64407@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <528eb43d-5ef7-2745-577d-8ab7ccc64407@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, George Valkov <gvalkov@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) On 2/16/23 13:32, P=C3=A1draig Brady wrote: > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* any desired permissions that weren't cl= oned.=C2=A0 */ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 extra_permissions =3D desired_mode &= ~cloned_mode; That comment is a tad redundant, but I did add some commentary along=20 those lines and installed the patch into Savannah master. I haven't had time to look into the SEEK_HOLE stuff but hope to find time= .
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 16 Feb 2023 21:32:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 16 16:32:12 2023
Received: from localhost ([127.0.0.1]:37817 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pSlrQ-000060-1N
for submit <at> debbugs.gnu.org; Thu, 16 Feb 2023 16:32:12 -0500
Received: from mail-wm1-f47.google.com ([209.85.128.47]:51762)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <pixelbeat@HIDDEN>) id 1pSlrN-00005l-SK
for 61386 <at> debbugs.gnu.org; Thu, 16 Feb 2023 16:32:10 -0500
Received: by mail-wm1-f47.google.com with SMTP id l25so49565wms.1
for <61386 <at> debbugs.gnu.org>; Thu, 16 Feb 2023 13:32:09 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=content-transfer-encoding:in-reply-to:from:content-language
:references:cc:to:subject:user-agent:mime-version:date:message-id
:sender:from:to:cc:subject:date:message-id:reply-to;
bh=nOaZhNTUAjX4gMev5ZY/p648bKwqm9eh5y4OKvZ61sw=;
b=As5Sr/Rbs8vbV8ur6rRUqT3LtrH9BjbeCH4bRu4TWDR89B3djkZpUQ1U0ZVQaLnYoW
zqK9shZoaWUL6xxlydHPuuQFdA7z75c0WFIaKih5nhCHxZSTBz57bAYTLnvin/0XpdLP
x6tRhjkEZsJyGapgkRPOJvhoJPmz8WPYK5+NiHtX5+TJTkgpHZRCldYPqHb6L5Kp+dZP
vyAksjbkTYFe+IRllkAIM4elh/7kituYPPnV2SMK4m3c+cB/0HveKl3pde6uzxIdvCYz
4TvfW/KflXuhW0sjWwFNivdzzTWJNZyR5mbmULs4HMeaA4lhfc1XrQyk0fMK0oOqOY8p
lVsQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=content-transfer-encoding:in-reply-to:from:content-language
:references:cc:to:subject:user-agent:mime-version:date:message-id
:sender:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=nOaZhNTUAjX4gMev5ZY/p648bKwqm9eh5y4OKvZ61sw=;
b=th6en8/sINUBz/HwSB7woOqekaGfg2rF3Ps1LTouMY20QdHv9K3NETwzcdJcnY8C+f
MwVsIZJ2nHeQIbbR7Vg141xS3+webxyLTpIMDXLPU7CQkzVJgt7Q6D2sqpslk3J9PhP5
xmAfIo8quDGbPhjHz7tZoq3Gw7Mv71DJX6R8KJHRpTwmCB3nAU3b4wpiJgx4CYv+xjbf
8WACOUfu+PCoISil8n+AXM7GqPyvNHmWNOJIR5yyV3nv8c6N/EOiS8V2f24nIsnqxrvP
2IsCA2WcqSClV1x2fPh8EY7NuGh+ddyWcz3nevmcMC9i6j6DqtB4oz8sg5rm4xEPYlkb
Sr0A==
X-Gm-Message-State: AO0yUKXFYBqusvFLxNyCYiHCSYUSME1aIHYzXr+wiiJYxfkzt5jfVJi1
EM9vXOFBpQ5CE4cOGA9BdJ0=
X-Google-Smtp-Source: AK7set/cYEAc+iu90LFP/I/IIel0cSQri0Zfk95u+9AmW1hYyesxw50twV7cA4cpuRFzKQ131TIh2A==
X-Received: by 2002:a05:600c:3b85:b0:3df:fcbd:3159 with SMTP id
n5-20020a05600c3b8500b003dffcbd3159mr3016458wms.3.1676583123809;
Thu, 16 Feb 2023 13:32:03 -0800 (PST)
Received: from [192.168.1.9]
(95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175])
by smtp.googlemail.com with ESMTPSA id
he8-20020a05600c540800b003e208cec49bsm4860391wmb.3.2023.02.16.13.32.02
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Thu, 16 Feb 2023 13:32:03 -0800 (PST)
Message-ID: <528eb43d-5ef7-2745-577d-8ab7ccc64407@HIDDEN>
Date: Thu, 16 Feb 2023 21:32:01 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101
Thunderbird/109.0
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
To: Paul Eggert <eggert@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
<f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
<63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN>
<f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN>
<1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN>
<45adda7f-4654-e186-7052-c14612415c22@HIDDEN>
<354b1398-3c1c-9f83-1209-837825b9a85f@HIDDEN>
<acc021fa-5983-2026-63fd-94073902b756@HIDDEN>
Content-Language: en-US
From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
In-Reply-To: <acc021fa-5983-2026-63fd-94073902b756@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.9 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org, George Valkov <gvalkov@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.9 (-)
On 15/02/2023 10:56, Paul Eggert wrote:
> Attached is an updated proposal for the fclonefileat patch.
>
> CVE-2021-30995 confirmed my suspicion that coreutils 9.1 and the current
> bleeding-edge coreutils on Savannah both have an exploitable security
> bug on macOS. Although I hope this patch fixes the bug, it could use
> more pairs of eyes, given that Apple had so many problems fixing its own
> security bug involving fclonefileat, and given that the coreutils code
> has so many flags and conditions.
The logic looks good to me, and all tests pass here on macOS 13.2
I see the added `if (! (cloned_mode & ~desired_mode))` restriction I presume
for security reasons, so that we only _add_ permission bits to those copied by fclonefileat(),
and never create an overly accessible file and then remove permission bits.
That wasn't obvious to me at least so I'd suggest adding comments from the diff below.
For the record this restriction will mean we can't reflink in some cases.
For example the following would fail with ENOSUP on macOS:
touch file;
chmod g+w file
umask 0022
cp --reflink=always file file.copy
Using -p in the above cp command would work though.
thanks,
Pádraig.
diff --git a/src/copy.c b/src/copy.c
index c5f2cc186..7c866e4fb 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -1284,6 +1284,7 @@ copy_reg (char const *src_name, char const *dst_name,
: x->set_mode ? x->mode
: ((x->explicit_no_preserve_mode ? MODE_RW_UGO : dst_mode)
& ~ cached_umask ()));
+ /* If fclonefileat() would not set undesired extra permissions. */
if (! (cloned_mode & ~desired_mode))
{
int fc_flags
@@ -1318,7 +1319,9 @@ copy_reg (char const *src_name, char const *dst_name,
}
}
+ /* any desired permissions that weren't cloned. */
extra_permissions = desired_mode & ~cloned_mode;
+
if (!extra_permissions
&& (!x->preserve_mode || (fc_flags & CLONE_ACL)
|| !fd_has_acl (source_desc)))
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 16 Feb 2023 19:41:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 16 14:41:53 2023
Received: from localhost ([127.0.0.1]:37699 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pSk8e-0005Dm-Mc
for submit <at> debbugs.gnu.org; Thu, 16 Feb 2023 14:41:53 -0500
Received: from mail-wm1-f53.google.com ([209.85.128.53]:51858)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pSk8c-0005DU-87
for 61386 <at> debbugs.gnu.org; Thu, 16 Feb 2023 14:41:50 -0500
Received: by mail-wm1-f53.google.com with SMTP id n36so1649667wmr.1
for <61386 <at> debbugs.gnu.org>; Thu, 16 Feb 2023 11:41:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20210112; t=1676576504;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:from:to:cc:subject:date:message-id:reply-to;
bh=10AbNFEEcO2JRkVlLeVb+vr6FO8TMZsOEQMfqVDDH34=;
b=UEHw9Py1Pammaoii7UqQuDZXJIK113DPs9mbL43yTxQMI5PyMulFGk6eaFY2+SbpnD
Q+CRdrOr1OebPkY5tqYWbqtx9dyW4TGJKrTRz4NTA/OFByyVNyTGjZFJA2Hr/pLuuPVN
HKRBKqPtl12MNMHda4Dth/XsaFFCXOugIYSwE8iKRTH3AN/iWArG2Eqm2KYDFEKdY2or
/VfEvAbgpi9NWLWPtRURVCB9LkIB0Oo1wPztZob4NqdI0IF/7bNiqw24mPSFZiqwdv3P
EsUAL2fFlXo9hOTBro1ycJhbc0e8iXYydTevEQjG4HyuWJ8B5gpYgyhorA2gn7zUalhw
6hkw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112; t=1676576504;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=10AbNFEEcO2JRkVlLeVb+vr6FO8TMZsOEQMfqVDDH34=;
b=2vwUHCCtWV22BBHNIGrj6sG9UvKssryKMT8wkoY6TZ7ro3JdEjkUvEMcrOPGSAYWDK
Lc6sX0Q99XCKIHWuO9RrPE3bJ6rP5z8JM/yA2tXX85NjrgACA3kz/FciRi0d3xykqkJL
S3+9zfve/U2PiFudKz9vsV1iBQiOrM95Q9/DHQPr/n7TuCUNnax5RsYn4Hqqc0vNoN0l
HD0VJwNciT8WYqLBvEQTq8vBUssUptePu+V74mZX40CAZV88T2T9abJVGmgSFFdfAKkz
CjG3G18EexN4PiWDIPJWECTCS2e4Nm2998Ja91CYo6gdNdSl/H2qrQ3Rm8PTak6fSW8B
rAuA==
X-Gm-Message-State: AO0yUKU/3I3tvFwr/Nm8vM6G2sPilxt5YKigsXj22NVC0hKkqyxzaYS6
BlYpLfZnnEmiGQQ7snXuevQ=
X-Google-Smtp-Source: AK7set8k2OmLyTXKAPQ6O05cgVbW278SP07W+XA9p+bmfNAq6UlivLFG6yWlKeFjI+oNlHOFzoTcJg==
X-Received: by 2002:a7b:cd08:0:b0:3e2:1d9d:f10 with SMTP id
f8-20020a7bcd08000000b003e21d9d0f10mr840824wmj.4.1676576503942;
Thu, 16 Feb 2023 11:41:43 -0800 (PST)
Received: from smtpclient.apple (212-39-89-241.ip.btc-net.bg. [212.39.89.241])
by smtp.gmail.com with ESMTPSA id
ay1-20020a05600c1e0100b003e1fee8baacsm6312066wmb.25.2023.02.16.11.41.42
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Thu, 16 Feb 2023 11:41:42 -0800 (PST)
From: George Valkov <gvalkov@HIDDEN>
Message-Id: <9139A642-34D8-42A8-887F-CE027AAAB2F7@HIDDEN>
Content-Type: multipart/mixed;
boundary="Apple-Mail=_ED3FF750-836C-493B-B886-6EDB626F3B01"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Date: Thu, 16 Feb 2023 21:41:41 +0200
In-Reply-To: <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN>
To: Paul Eggert <eggert@HIDDEN>,
=?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
<f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
<63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN>
<f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN>
<1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN>
<E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN>
<3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN>
<4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--Apple-Mail=_ED3FF750-836C-493B-B886-6EDB626F3B01
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
Gentlemen, I found a solution, as well as the reason why =
lseek(SEEK_DATA) returns invalid data.
I found a way to craft a file that gets corrupted when we try to make a =
sparse copy. It=E2=80=99s based on mmap:
1. Open src and dst.
2. Obtain the size of src, and truncate dst to the same size.
3. Use mmap to map dst to RAM.
4. Read src and memcpy all non-blank pages to dst.
5. Use munmap and then close src and dst.
6. If src contained any blank pages, dst is now sparse.
7. Since we did not call msync, the changes are still cached in RAM and =
lseek(SEEK_DATA) is not aware that some of the pages contain valid data.
Workaround:
1. open(dst, O_RDONLY)
2. Obtain the file size =3D lseek(dst, 0, SEEK_END)
3. Use mmap to map dst to RAM.
4. Call msync.
5. Use munmap and then close dst.
6. The memory view of dst is now synchronised with the underlaying =
file-system.
7. lseek(SEEK_DATA) returns valid data.
Proof of Concept:
1. Start by extracting cc1
=
https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/cc1.tgz=
2. The extracted file is not sparse, we need to craft a sparse copy =
cc1-mmap without synchronising the filesystem with the memory view
gcc m.c && ./a.out
src 3 dst 4
00001000 skip
total bytes copied 1a45640 / 1a46640 1a46640
3. Now sparse copy cc1-mmap to cc1-sparse
gcc d.c && ./a.out
src 3 dst 4
c a46640 p 1a46640 h 1a46640 d 1000000
total bytes copied a46640 / 1a46640
4. cc1-sparse is corrupted
sha1sum cc1*
16d835378ab973a114082a585cc76958bdbccec0 cc1
16d835378ab973a114082a585cc76958bdbccec0 cc1-mmap
75e6f6cb303cb5d3909c6d7830c417fc6ed658c3 cc1-sparse
5. Use msync to update the filesystem
gcc n.c && ./a.out
dst 3 MS_ASYNC 1 MS_INVALIDATE 2 MS_SYNC 16
size bytes 1a46640 msync ok
6. Make sparse copy again cc1-mmap to cc1-sparse
gcc d.c && ./a.out
src 3 dst 4
c 1000 p 1000 h 1000 d 0
c 1a44640 p 1a46640 h 1a46640 d 2000
total bytes copied 1a45640 / 1a46640
7. cc1-sparse is good
sha1sum cc1*
16d835378ab973a114082a585cc76958bdbccec0 cc1
16d835378ab973a114082a585cc76958bdbccec0 cc1-mmap
16d835378ab973a114082a585cc76958bdbccec0 cc1-sparse
PoC samples
--Apple-Mail=_ED3FF750-836C-493B-B886-6EDB626F3B01
Content-Disposition: attachment;
filename=d.c
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="d.c"
Content-Transfer-Encoding: 7bit
#include <errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <unistd.h>
#include <sys/clonefile.h>
#define MAX_SIZE (1024 * 1024 * 100)
char msg[MAX_SIZE];
int main(int argc, char ** argv)
{
// sparse copy test
// block size i 4 KB
// c d67000 p 1367000 h 1367000 d 600000
// total bytes copied d67000 / 1a46640
int src = open("cc1-mmap", O_RDONLY);
int dst = open("cc1-sparse", O_CREAT | O_TRUNC | O_RDWR, 0700);
printf("src %i dst %i\n", src, dst);
//- printf("SET %i CUR %i END %i HOLE %i DATA %i\n", SEEK_SET, SEEK_CUR, SEEK_END, SEEK_HOLE, SEEK_DATA);
off_t a = 0;
off_t b = 0;
off_t d = 0;
off_t e = 0;
off_t h = 0;
off_t p = 0;
off_t s = -1;
off_t t = 0;
ssize_t c = 0;
ssize_t i = 0;
ssize_t r = 0;
ssize_t w = 0;
int ea = 0;
int eb = 0;
int ed = 0;
int ee = 0;
int eh = 0;
int ep = 0;
do
{
if (++i >= 10)
{
printf("LOOP %zi\n", i);
break;
}
errno = 0;
d = lseek(src, d, SEEK_DATA);
ed = errno;
h = lseek(src, d, SEEK_HOLE);
eh = errno;
a = lseek(src, d, SEEK_SET);
ea = errno;
b = lseek(dst, d, SEEK_SET);
eb = errno;
c = h - d;
if ((a == -1) || (b == -1) || (d == -1) || (h == -1))
{
int handled = 0;
if ((d == -1) && (ed == ENXIO))
{
p = lseek(src, 0, SEEK_END);
ep = errno;
if (p == -1)
{
printf(
"lseek(SEEK_END) failed p %lli %2i %s\n",
p, ep, strerror(ep)
);
}
else
{
e = ftruncate(dst, p);
ee = errno;
if (e == -1)
{
printf(
"ftruncate(%lli) failed %lli %2i %s\n",
p, e, ee, strerror(ee)
);
}
else
{
handled = 1;
}
}
}
if (!handled)
{
printf(
"lseek failed"
" ENXIO %i EBADF %i EINVAL %i EOVERFLOW %i ESPIPE %i\n"
" p %llx\n"
" d %llx %2i %s\n"
" h %llx %2i %s\n"
" a %llx %2i %s\n"
" b %llx %2i %s\n",
ENXIO, EBADF, EINVAL, EOVERFLOW, ESPIPE,
p,
d, ed, strerror(ed),
h, eh, strerror(eh),
a, ea, strerror(ea),
b, eb, strerror(eb)
);
}
break;
}
if (c < 0)
{
printf("negative size c %zi = h %lli - d %lli\n", c, h, d);
continue;
}
if (s == d)
{
printf("EOF %llu %llu\n", s, d);
break;
}
s = d;
if (c > sizeof(msg))
{
printf("msg too small %zu %zi\n", sizeof(msg), c);
break;
}
r = read(src, msg, c);
w = write(dst, msg, c);
if ((r != c) || (w != c))
{
printf("expected %zi got %zi %zi\n", c, r, w);
break;
}
p = lseek(src, 0, SEEK_CUR);
t += c;
printf("c %zx p %llx h %llx d %llx\n", c, p, h, d);
d = p;
} while (1);
close(src);
close(dst);
printf("total bytes copied %llx / %llx\n", t, p);
return 0;
}
--Apple-Mail=_ED3FF750-836C-493B-B886-6EDB626F3B01
Content-Disposition: attachment;
filename=m.c
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="m.c"
Content-Transfer-Encoding: 7bit
#include <errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <unistd.h>
#include <sys/clonefile.h>
#include <sys/mman.h>
#define MAX_SIZE (1024 * 1024 * 100)
#define BLOCK_SIZE (4096)
char msg[MAX_SIZE];
char zero[BLOCK_SIZE];
int main(int argc, char ** argv)
{
// sparse mmap test
// block size i 4 KB
int src = open("cc1", O_RDONLY);
int dst = open("cc1-mmap", O_CREAT | O_TRUNC | O_RDWR, 0700);
printf("src %i dst %i\n", src, dst);
//- printf("SET %i CUR %i END %i HOLE %i DATA %i\n", SEEK_SET, SEEK_CUR, SEEK_END, SEEK_HOLE, SEEK_DATA);
const char * p = msg;
const char * n = msg;
const char * e = msg;
char * d = NULL;
off_t a = 0;
off_t b = 0;
off_t s = 0;
off_t t = 0;
ssize_t c = 0;
ssize_t i = 0;
ssize_t r = 0;
ssize_t w = 0;
int ea = 0;
int eb = 0;
int ed = 0;
int es = 0;
int et = 0;
errno = 0;
s = lseek(src, 0, SEEK_END);
es = errno;
if (s == -1)
{
printf(
"lseek(SEEK_END) failed s %lli %2i %s\n",
s, es, strerror(es)
);
return 1;
}
if (s > sizeof(msg))
{
printf("msg too small %zu %lli\n", sizeof(msg), s);
return 2;
}
t = ftruncate(dst, s);
et = errno;
if (t == -1)
{
printf(
"ftruncate(%lli) failed %lli %2i %s\n",
s, t, et, strerror(et)
);
return 3;
}
a = lseek(src, 0, SEEK_SET);
ea = errno;
if (a == -1)
{
printf(
"lseek(SEEK_SET) failed a %lli %2i %s\n",
a, ea, strerror(ea)
);
return 4;
}
// read all data
r = read(src, msg, s);
if (r != s)
{
printf("expected %zi read %zi\n", c, r);
return 5;
}
memset(zero, 0, sizeof(zero));
d = mmap(NULL, s, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, dst, 0);
ed = errno;
if (!d)
{
printf("mmap failed %p %i %s\n", d, ed, strerror(ed));
return 6;
}
for (e = msg + s; p < e; p = n, d += BLOCK_SIZE)
{
errno = 0;
n = p + BLOCK_SIZE;
if (n > e)
{
n = e;
}
c = n - p;
if (memcmp(zero, p, c))
{
// write valid sector
//- printf("%08zx copy\n", p - msg);
memcpy(d, p, c);
i += c;
}
else
{
// skip empty sector
printf("%08zx skip\n", p - msg);
}
}
b = lseek(src, 0, SEEK_CUR);
munmap(d, s);
close(src);
close(dst);
printf("total bytes copied %zx / %llx %llx\n", i, b, s);
return 0;
}
--Apple-Mail=_ED3FF750-836C-493B-B886-6EDB626F3B01
Content-Disposition: attachment;
filename=n.c
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="n.c"
Content-Transfer-Encoding: 7bit
#include <errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <unistd.h>
#include <sys/clonefile.h>
#include <sys/mman.h>
int main(int argc, char ** argv)
{
// sparse mmap sync
// block size i 4 KB
int dst = open("cc1-mmap", O_RDONLY, 0700);
printf("dst %i MS_ASYNC %i MS_INVALIDATE %i MS_SYNC %i\n", dst, MS_ASYNC, MS_INVALIDATE, MS_SYNC);
char * d = NULL;
int a = 0;
int b = 0;
off_t s = 0;
int ea = 0;
int eb = 0;
int ed = 0;
int es = 0;
errno = 0;
s = lseek(dst, 0, SEEK_END);
es = errno;
if (s == -1)
{
printf(
"lseek(SEEK_END) failed s %lli %2i %s\n",
s, es, strerror(es)
);
return 1;
}
d = mmap(NULL, s, PROT_READ, MAP_FILE | MAP_SHARED, dst, 0);
ed = errno;
if (!d)
{
printf("mmap failed %p %i %s\n", d, ed, strerror(ed));
return 2;
}
// MS_ASYNC 1 MS_INVALIDATE 2 MS_SYNC 16
a = msync(d, s, 0);
ea = errno;
if (a == -1)
{
printf("msync failed s %i %2i %s\n", a, ea, strerror(ea));
return 3;
}
b = munmap(d, s);
eb = errno;
close(dst);
printf("size bytes %llx msync ok\n", s);
return 0;
}
--Apple-Mail=_ED3FF750-836C-493B-B886-6EDB626F3B01
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=us-ascii
Cheers!
Georgi Valkov
httpstorm.com
nano RTOS
--Apple-Mail=_ED3FF750-836C-493B-B886-6EDB626F3B01--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 15 Feb 2023 22:53:41 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 15 17:53:41 2023 Received: from localhost ([127.0.0.1]:34387 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pSQei-0008Th-Ue for submit <at> debbugs.gnu.org; Wed, 15 Feb 2023 17:53:41 -0500 Received: from mail-wm1-f51.google.com ([209.85.128.51]:34697) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pSQeh-0008TV-G8 for 61386 <at> debbugs.gnu.org; Wed, 15 Feb 2023 17:53:40 -0500 Received: by mail-wm1-f51.google.com with SMTP id f23-20020a05600c491700b003dff4480a17so3106314wmp.1 for <61386 <at> debbugs.gnu.org>; Wed, 15 Feb 2023 14:53:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1676501613; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=GOsb4Dxf95JXIEtjwsMbIvAZ1r7dPr79MC/y+idQ+Eo=; b=m67hyweDPrNz/nFbjL1eN1Yw28dTd6GC5wkN236AwPb7Gtm6lbD/AqUjt3NcSM3tPu GoeibwkIoEcyPklsGiYIOjvQseg93YSevjwutCLhsYhxuYeYKcNiuUzfeUzT3b8Xa9aI 36fsOaqFQo1hCvDEnOn+a1RqarlX22GA+VT9ig15o28ecOy80LVoqfYu1p3OG2fAGX9R CCBcf67M8uP1w25MGdhnC/1izAsfgksl/vIBIWKth7iuNIv9LtHRqBn3TIZv2zIyf7TW vzcOtroCQg0iXkGqjyiCewcxqNY7IsgRwWu7zNM3yHHuuO5kal5yyFUrhBve9KCqAtZV vH/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1676501613; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GOsb4Dxf95JXIEtjwsMbIvAZ1r7dPr79MC/y+idQ+Eo=; b=mQT7THscPtGt5LeDswZoxGyjndcW9IXfynPWk3IKkm46ohPeCbui6fKPVUB9ctffyF Kv3Pe/hU/k9tERO8HCeng2AjeiM9J3NJ11rvyY1GKKOnKTsT7JrBdydpXvRwtgdbhFmI wuJnhcjkfDZ6f8KtwGyLCpgOdFZrYYFk1ZWzfmREtPZuzjwFJPz+0ytLb4e66M3TbpTv LElOWebgAKJpCTgRRTtQ4qGd2XtLKAf6vRiFxqeOh1DZMAcz5EHmYo0Vfl7H0JrntKye e+zhbEuYadwLeJNtruZ600Rvn9d8WSt5wVR9VGUv1Iq8e/Z5vhEDHqi5zMSI2H68zC1g bpDA== X-Gm-Message-State: AO0yUKUDtiUi66X0Nk7Os4GVwD7E65fgmC/d2+NFHwOSjpLATxBFeLWp FVE7I4Ampa4szZAU+7xmv0c= X-Google-Smtp-Source: AK7set9UyO7qn+7mrmeCPEbFdB2A0jYX8feb1g6UZxMff1sWNQprz7THraISoAXWtYb3kjxeAWKLsA== X-Received: by 2002:a05:600c:3c9e:b0:3dc:3b1a:5d2d with SMTP id bg30-20020a05600c3c9e00b003dc3b1a5d2dmr3643254wmb.0.1676501613490; Wed, 15 Feb 2023 14:53:33 -0800 (PST) Received: from smtpclient.apple (212-39-89-241.ip.btc-net.bg. [212.39.89.241]) by smtp.gmail.com with ESMTPSA id iz17-20020a05600c555100b003dc4b4dea31sm3275215wmb.27.2023.02.15.14.53.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Feb 2023 14:53:32 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <fac9d09e-312c-969a-440b-24db8054945c@HIDDEN> Date: Thu, 16 Feb 2023 00:53:30 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <7603E830-ADB2-4DD7-907A-A4F1C3D75622@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <45adda7f-4654-e186-7052-c14612415c22@HIDDEN> <354b1398-3c1c-9f83-1209-837825b9a85f@HIDDEN> <acc021fa-5983-2026-63fd-94073902b756@HIDDEN> <EC5AAB70-963A-44DA-ADDA-545D0DDE4753@HIDDEN> <fac9d09e-312c-969a-440b-24db8054945c@HIDDEN> To: Paul Eggert <eggert@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On 2023-02-15, at 9:48 PM, Paul Eggert <eggert@HIDDEN> wrote: >=20 > On 2023-02-15 07:26, George Valkov wrote: >> I tested your patch: both overwrite existing and clone to new produce = a working copy. Here are the test results: >> = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-04= -cf80f988eeb97cc3f8c65ae58e735d36f865277b-clone.txt >=20 > I see some test failures there, involving cp. Do you get the same set = of test failures without the patch? Yes. Note that P=C3=A1draig disabled two tests which fail after the = patch: tests/cp/thru-dangling.sh, tests/seek-data-capable (the actual test that = failed was related to sparse, I think sparse-extents.sh) = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-01= -cf80f988eeb97cc3f8c65ae58e735d36f865277b-ori.txt = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-04= -cf80f988eeb97cc3f8c65ae58e735d36f865277b-clone.txt >> In the case when a dangling symlink is involved and depending on the = arguments passed to cp, would it be possible to use CLONE_NOFOLLOW with = fclonefileat or fall-back to a normal copy? Does dangling refer to = source or destination? >=20 > Dangling refers to the destination. The latest proposed patch does use = CLONE_NOFOLLOW, which means fclonefileat should not follow symlinks to = the destination. (This behavior is not documented, unfortunately, but = it's the only behavior that makes sense.) Ok, thank you for clarifying! Good luck! Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 15 Feb 2023 22:41:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 15 17:41:03 2023
Received: from localhost ([127.0.0.1]:34367 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pSQSU-00085v-40
for submit <at> debbugs.gnu.org; Wed, 15 Feb 2023 17:41:03 -0500
Received: from mail-ej1-f53.google.com ([209.85.218.53]:44015)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pSQSQ-00085L-QK
for 61386 <at> debbugs.gnu.org; Wed, 15 Feb 2023 17:41:00 -0500
Received: by mail-ej1-f53.google.com with SMTP id k16so692490ejv.10
for <61386 <at> debbugs.gnu.org>; Wed, 15 Feb 2023 14:40:58 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:from:to:cc:subject:date:message-id:reply-to;
bh=SO+xGbJkbenfTm/oFS5EOwH/QqjjXmLfAPLAxJ1zcUw=;
b=WmiX48oTjjsZIiRdT5oeDh5+f0XKLVLRIM5EPSE2ga+xvk1GIkahyudZan+juBy4oq
MobiO4afdyMXBBzWK3vPfBb4RlmuseuadUWk6xjcqN0gHnxR2QbdKUrSR/a4DoAnSCU2
BPAFkMDXt+7DdSTQ0Y0FxvkIAp95/k8EBedPUc+D4zAdOhNfxvmVovhtsbcDRepODjS9
VUqgEpqYa7zpVqrk3/IjMGcspDgTQLYapD3VSNisXUzcUPoi2PiApKyDARiih3wl6HXi
nb8QT+i91JrLWY3Thh5YjRNaD9nnSq2/szEmhNXXats2ks0tSsRystL+fwdcX24azbcy
cosg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=SO+xGbJkbenfTm/oFS5EOwH/QqjjXmLfAPLAxJ1zcUw=;
b=NMF7V+zE7HIBSvW3ZZ4YYeTuKXxL6zjOUBV/xjTZMdfu1O5PSl49YuFpBqgucZi05G
/77XnE7oD7CgfDT0vsC/gpJ3XcxJrDXgl3fzQkwCne/z0Uf07JgNhlyKsK8zZsarQ9I4
U2e0zZSzAknW5S7pW4Qb7fhNk5W8MDqaErJWI3G86F7wIcXpSuUHUUQ3ZlKBEVpRjCtk
5jSwrB2P3HVFNyed5AeesLdxbuUSAsK3Put7+dniw82teNy5rZ8aIK+iMyaWGDoImQO0
sG7IPgRmYPB/nDlWr1sekYEDkNOmudlJajahkPJOFjAb+QYgjpQIIdZbLSiA2hlz6gN4
BG/g==
X-Gm-Message-State: AO0yUKXP5rBGhFvBDZlCVFtfyybAAvFdp/L0/pit0WFfsq7S7nkP8Rsp
+03AzHYPNYBuF/NP+An97fQ=
X-Google-Smtp-Source: AK7set+ZQ09H+h3RVBvy/lqLFjlF3Fi8y7VByQa49qyU3fvxIQqk2ckXkJ1b9v9vliaVDN/4yqXZpg==
X-Received: by 2002:a17:906:209e:b0:8b1:3b2d:ba37 with SMTP id
30-20020a170906209e00b008b13b2dba37mr3213158ejq.7.1676500852321;
Wed, 15 Feb 2023 14:40:52 -0800 (PST)
Received: from smtpclient.apple (212-5-158-145.ip.btc-net.bg. [212.5.158.145])
by smtp.gmail.com with ESMTPSA id
i25-20020a170906851900b008b1347773ccsm2458646ejx.144.2023.02.15.14.40.50
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Wed, 15 Feb 2023 14:40:50 -0800 (PST)
From: George Valkov <gvalkov@HIDDEN>
Message-Id: <4E376F75-7440-4FEB-8934-3748F6AE06F4@HIDDEN>
Content-Type: multipart/mixed;
boundary="Apple-Mail=_4EF4D2CC-EE74-4D00-9418-5CC293358930"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Date: Thu, 16 Feb 2023 00:40:49 +0200
In-Reply-To: <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN>
To: Paul Eggert <eggert@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
<f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
<63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN>
<f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN>
<1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN>
<E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN>
<3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 1.5 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: > On 2023-02-15, at 9:26 PM,
Paul Eggert wrote: > > On 2023-02-15
06:05, George Valkov wrote: >> gcc d.c && ./a.out >> src 3 dst 4 >> c 14053376
p 20344832 h 20344832 d 6291456 >> total bytes copied [...]
Content analysis details: (1.5 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (gvalkov[at]gmail.com)
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
[212.5.158.145 listed in dnsbl.sorbs.net]
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust [209.85.218.53 listed in list.dnswl.org]
0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3)
[209.85.218.53 listed in wl.mailspike.net]
0.0 RCVD_IN_MSPIKE_WL Mailspike good senders
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)
--Apple-Mail=_4EF4D2CC-EE74-4D00-9418-5CC293358930
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
> On 2023-02-15, at 9:26 PM, Paul Eggert <eggert@HIDDEN> wrote:
>=20
> On 2023-02-15 06:05, George Valkov wrote:
>> gcc d.c && ./a.out
>> src 3 dst 4
>> c 14053376 p 20344832 h 20344832 d 6291456
>> total bytes copied 14053376 / 27551296
>=20
> Thanks, this is due to a known incompatibility in macOS lseek that =
coreutils is supposed to work around. See =
<https://www.gnu.org/software/gnulib/manual/html_node/lseek.html>, which =
says, "On some platforms, lseek (fd, offset, SEEK_DATA) returns a value =
greater than offset even when offset addresses data: macOS 12".
Hi Paul! There are two possible solutions here:
1. Do not use lseek(SEEK_DATA) on macOS.
2. Update the cached list of valid sectors which is used by lseek. See =
=C2=A7=C2=A7=C2=A7 below. There could be some API to achieve this. A =
workaround is to fclonefileat to a temporary file and then remove it. =
Internally fclonefileat updates that cache. We might be able to find how =
it=E2=80=99s done.
manpage
If whence is SEEK_DATA, the offset is set to the start of the next =
non-hole file region greater than or equal to the supplied offset.
I believe lseek operates correctly, however the underlaying cache it =
uses is not current after certain conditions occur.
> I guess that somehow, the way you're building coreutils defeats the =
workaround. If so, we'll need to change how coreutils is built in your =
environment, or fix coreutils 'configure' so that the workaround isn't =
defeated in your environment.
Here are the build commands I use:
git clone https://github.com/coreutils/coreutils.git
cd
coreutils
git submodule foreach git pull origin master
./bootstrap
./configure
make -j 16
Here is a tarball of coreutils after building:
=
https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/coreuti=
ls-cf80f988eeb97cc3f8c65ae58e735d36f865277b.tgz
> Although in <https://bugs.gnu.org/61386#128> P=C3=A1draig was dubious =
about this guess, his reasoning that the bug is likely specific to APFS =
rather than an API mismatch could be wrong, as I think HFS doesn't =
support SEEK_DATA at all or reports trivial answers, so coreutils is not =
likely to run into the problem on HFS even if the bug is an API issue.
> Here are some things we can do to test this guess.
>=20
> 1. Please try the attached program e.c in place of your d.c program. =
e.c is like d.c, except it attempts to use the coreutils workaround. =
What symptoms do you observe? If e.c works then it's almost surely a =
problem in how coreutils is built (compiler options or whatnot), not in =
the coreutils workaround. If e.c does not work it's likely that the =
Gnulib workaround does not suffice on your macOS platform, in which case =
we need to improve the workaround by hacking further on e.c and porting =
the result back to Gnulib. (There are other possibilities.)
It produces the same corrupted copy as the original sample:
gcc e.c && ./a.out=20
__APPLE__ 1 __MACH__ 1 SEEK_DATA 4
src 3 dst 4
c 14053376 p 20344832 h 20344832 d 6291456
total bytes copied 14053376 / 27551296
In hex
c d67000 p 1367000 h 1367000 d 600000
total bytes copied d67000 / 1a46640
419494dd4527ebb22c7bf2b388316a4beb5c73d2 cc1
7b447132f56f3b549ef62a4780945e82d26f7d2c cc1-sparse
The after I use clone to update the cached list of valid sectors for cc1
./coreutils-2023-02-15/src/cp cc1 cc1-clone
It returns a bunch of errors, but produces a valid copy. There is no =
point in using this workaround on macOS, it does not offer any =
improvements. I would recommend making sure that lseek sees an updated =
list of valid sectors.
gcc e.c && ./a.out=20
__APPLE__ 1 __MACH__ 1 SEEK_DATA 4
src 3 dst 4
c 1000 p 1000 h 1000 d 0
c 1a44640 p 1a46640 h 1a46640 d 2000
lseek failed ENXIO 6 EBADF 9 EINVAL 22 EOVERFLOW 84 ESPIPE 29
p 1a46640
d 1a46640 6 Device not configured
h ffffffffffffffff 6 Device not configured
a 1a46640 6 Device not configured
b 1a46640 6 Device not configured
total bytes copied 1a45640 / 1a46640
419494dd4527ebb22c7bf2b388316a4beb5c73d2 cc1
419494dd4527ebb22c7bf2b388316a4beb5c73d2 cc1-clone
419494dd4527ebb22c7bf2b388316a4beb5c73d2 cc1-sparse
> 2. Please verify that coreutils cp is using the Gnulib workaround. In =
the src directory, the shell command "nm -o *.o | grep lseek" should =
output only lines containing "rpl_lseek"; there shouldn't be any lines =
saying just "lseek". Also, please run the command "objdump -d =
lib/libcoreutils_a-lseek.o" and verify that the replacement lseek is =
actually doing something nontrivial (you should get maybe three dozen =
lines of assembly language; if it's much less then this is the problem).
nm -o *.o | grep lseek
cat.o: U _rpl_lseek
copy.o: U _rpl_lseek
dd.o: U _rpl_lseek
head.o: U _rpl_lseek
selinux.o: no symbols
shred.o: U _rpl_lseek
shuf.o: U _rpl_lseek
split.o: U _rpl_lseek
tac.o: U _rpl_lseek
tail.o: U _rpl_lseek
tail.o: 0000000000002d80 t _xlseek
truncate.o: U _rpl_lseek
wc.o: U _rpl_lseek
objdump -d lib/libcoreutils_a-lseek.o
lib/libcoreutils_a-lseek.o: file format mach-o 64-bit x86-64
Disassembly of section __TEXT,__text:
0000000000000000 <_rpl_lseek>:
0: 55 pushq %rbp
1: 48 89 e5 movq %rsp, %rbp
4: 41 57 pushq %r15
6: 41 56 pushq %r14
8: 53 pushq %rbx
9: 50 pushq %rax
a: 49 89 f7 movq %rsi, %r15
d: 41 89 fe movl %edi, %r14d
10: 83 fa 04 cmpl $4, %edx
13: 75 23 jne 0x38 <_rpl_lseek+0x38>
15: 44 89 f7 movl %r14d, %edi
18: 4c 89 fe movq %r15, %rsi
1b: ba 03 00 00 00 movl $3, %edx
20: e8 00 00 00 00 callq 0x25 <_rpl_lseek+0x25>
25: 48 89 c3 movq %rax, %rbx
28: 48 85 c0 testq %rax, %rax
2b: 78 20 js 0x4d <_rpl_lseek+0x4d>
2d: 31 d2 xorl %edx, %edx
2f: 4c 39 fb cmpq %r15, %rbx
32: 0f 94 c2 sete %dl
35: c1 e2 02 shll $2, %edx
38: 44 89 f7 movl %r14d, %edi
3b: 4c 89 fe movq %r15, %rsi
3e: 48 83 c4 08 addq $8, %rsp
42: 5b popq %rbx
43: 41 5e popq %r14
45: 41 5f popq %r15
47: 5d popq %rbp
48: e9 00 00 00 00 jmp 0x4d <_rpl_lseek+0x4d>
4d: e8 00 00 00 00 callq 0x52 <_rpl_lseek+0x52>
52: 83 38 06 cmpl $6, (%rax)
55: 49 0f 44 df cmoveq %r15, %rbx
59: 48 89 d8 movq %rbx, %rax
5c: 48 83 c4 08 addq $8, %rsp
60: 5b popq %rbx
61: 41 5e popq %r14
63: 41 5f popq %r15
65: 5d popq %rbp
66: c3 retq
> 3. Please confirm that _DARWIN_C_SOURCE is defined to 1 in =
lib/config.h.
Yes
Lib/config.h : line 3640
/* Enable general extensions on macOS. */
#ifndef _DARWIN_C_SOURCE
# define _DARWIN_C_SOURCE 1
#endif
> 4. What is the output of the following commands, in the coreutils =
build directory?
>=20
> rm lib/libcoreutils_a-lseek.o
> make V=3D1 lib/libcoreutils_a-lseek.o
> gcc -E -Ilib lib/lseek.c<e.c>
rm lib/libcoreutils_a-lseek.o
make V=3D1 lib/libcoreutils_a-lseek.o
gcc -I. -I./lib -Ilib -I./lib -Isrc -I./src -I/usr/local/include =
-Wno-cast-qual -Wno-conversion -Wno-float-equal -Wno-sign-compare =
-Wno-undef -Wno-unused-function -Wno-unused-parameter =
-Wno-float-conversion -Wimplicit-fallthrough -Wno-pedantic =
-Wno-sign-conversion -Wno-type-limits -g -O2 -MT =
lib/libcoreutils_a-lseek.o -MD -MP -MF =
lib/.deps/libcoreutils_a-lseek.Tpo -c -o lib/libcoreutils_a-lseek.o =
`test -f 'lib/lseek.c' || echo './'`lib/lseek.c
mv -f lib/.deps/libcoreutils_a-lseek.Tpo =
lib/.deps/libcoreutils_a-lseek.Po
gcc -E -Ilib lib/lseek.c < e.c > epp.txt
=
https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/epp.txt=
=C2=A7=C2=A7=C2=A7
I updated d.c to print in hex and use O_TRUNC on dst. I wrote another =
sample to read the entire source and write non-blank sectors creating a =
sparse file.
Copy cc1 to B
gcc f.c && ./a.out=20
src 3 dst 4
00001000 skip
total bytes copied 1a45640 / 1a46640 1a46640
Copy B to cc1-sparse
src 3 dst 4
c 1000 p 1000 h 1000 d 0
c 1a44640 p 1a46640 h 1a46640 d 2000
total bytes copied 1a45640 / 1a46640
c6daa8af9a75ab24e03252e63c6a855979f04095 cc1
c6daa8af9a75ab24e03252e63c6a855979f04095 B
c6daa8af9a75ab24e03252e63c6a855979f04095 cc1-sparse
I also tried playing with smaller values for BLOCK_SIZE down to 1. The =
output from d.c will always be the same because cc1 has only one sector =
that is a hole 0x1000-0x2000. The resulting sparse file B is then copied =
correctly by d.c, which suggests that lseek(SEEK_DATA) operates =
correctly when the file-system is in a good state, however it is likely =
that it returns cached data that could become invalid under certain =
conditions. If I unmount the sparse disk image and mount it again or use =
fclonefileat on cc1, the cache is updated and lseek returns good =
results. It would be interesting to trace the creation of cc1 during =
gcc/initial/compile.
Copy cc1 to cc1-sparse before the cached list of valid sectors is =
updated
src 3 dst 4
c d67000 p 1367000 h 1367000 d 600000
total bytes copied d67000 / 1a46640
7b447132f56f3b549ef62a4780945e82d26f7d2c cc1-sparse
Update the cached list of valid sectors for cc1
./coreutils-2023-02-15/src/cp cc1 cc1-clone
Copy cc1 to cc1-sparse after the cached list of valid sectors is updated
gcc d.c && ./a.out=20
src 3 dst 4
c 1000 p 1000 h 1000 d 0
c 1a44640 p 1a46640 h 1a46640 d 2000
total bytes copied 1a45640 / 1a46640
c6daa8af9a75ab24e03252e63c6a855979f04095 cc1-sparse
For a normal file, the list of valid sectors is up to date
gcc d.c && ./a.out=20
src 3 dst 4
c 229 p 229 h 229 d 0
total bytes copied 229 / 229
The sector size on the internal NVME disk is 4 KB (0x1000).
--Apple-Mail=_4EF4D2CC-EE74-4D00-9418-5CC293358930
Content-Disposition: attachment;
filename=d.c
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="d.c"
Content-Transfer-Encoding: 7bit
#include <errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <unistd.h>
#include <sys/clonefile.h>
#define MAX_SIZE (1024 * 1024 * 100)
char msg[MAX_SIZE];
int main(int argc, char ** argv)
{
// sparse copy test
// block size i 4 KB
// c d67000 p 1367000 h 1367000 d 600000
// total bytes copied d67000 / 1a46640
int src = open("cc1", O_RDONLY);
int dst = open("cc1-sparse", O_CREAT | O_TRUNC | O_RDWR, 0700);
printf("src %i dst %i\n", src, dst);
//- printf("SET %i CUR %i END %i HOLE %i DATA %i\n", SEEK_SET, SEEK_CUR, SEEK_END, SEEK_HOLE, SEEK_DATA);
off_t a = 0;
off_t b = 0;
off_t d = 0;
off_t e = 0;
off_t h = 0;
off_t p = 0;
off_t s = -1;
off_t t = 0;
ssize_t c = 0;
ssize_t i = 0;
ssize_t r = 0;
ssize_t w = 0;
int ea = 0;
int eb = 0;
int ed = 0;
int ee = 0;
int eh = 0;
int ep = 0;
do
{
if (++i >= 10)
{
printf("LOOP %zi\n", i);
break;
}
errno = 0;
d = lseek(src, d, SEEK_DATA);
ed = errno;
h = lseek(src, d, SEEK_HOLE);
eh = errno;
a = lseek(src, d, SEEK_SET);
ea = errno;
b = lseek(dst, d, SEEK_SET);
eb = errno;
c = h - d;
if ((a == -1) || (b == -1) || (d == -1) || (h == -1))
{
int handled = 0;
if ((d == -1) && (ed == ENXIO))
{
p = lseek(src, 0, SEEK_END);
ep = errno;
if (p == -1)
{
printf(
"lseek(SEEK_END) failed p %lli %2i %s\n",
p, ep, strerror(ep)
);
}
else
{
e = ftruncate(dst, p);
ee = errno;
if (e == -1)
{
printf(
"ftruncate(%lli) failed %lli %2i %s\n",
p, e, ee, strerror(ee)
);
}
else
{
handled = 1;
}
}
}
if (!handled)
{
printf(
"lseek failed"
" ENXIO %i EBADF %i EINVAL %i EOVERFLOW %i ESPIPE %i\n"
" p %lli\n"
" d %lli %2i %s\n"
" h %lli %2i %s\n"
" a %lli %2i %s\n"
" b %lli %2i %s\n",
ENXIO, EBADF, EINVAL, EOVERFLOW, ESPIPE,
p,
d, ed, strerror(ed),
h, eh, strerror(eh),
a, ea, strerror(ea),
b, eb, strerror(eb)
);
}
break;
}
if (c < 0)
{
printf("negative size c %zi = h %lli - d %lli\n", c, h, d);
continue;
}
if (s == d)
{
printf("EOF %llu %llu\n", s, d);
break;
}
s = d;
if (c > sizeof(msg))
{
printf("msg too small %zu %zi\n", sizeof(msg), c);
break;
}
r = read(src, msg, c);
w = write(dst, msg, c);
if ((r != c) || (w != c))
{
printf("expected %zi got %zi %zi\n", c, r, w);
break;
}
p = lseek(src, 0, SEEK_CUR);
t += c;
printf("c %zx p %llx h %llx d %llx\n", c, p, h, d);
d = p;
} while (1);
close(src);
close(dst);
printf("total bytes copied %llx / %llx\n", t, p);
return 0;
}
--Apple-Mail=_4EF4D2CC-EE74-4D00-9418-5CC293358930
Content-Disposition: attachment;
filename=e.c
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="e.c"
Content-Transfer-Encoding: 7bit
#define _DARWIN_C_SOURCE 1
#define _GNU_SOURCE 1
#include <errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <unistd.h>
#include <sys/clonefile.h>
off_t
rpl_lseek (int fd, off_t offset, int whence)
{
#if defined __APPLE__ && defined __MACH__ && defined SEEK_DATA
if (whence == SEEK_DATA)
{
/* If OFFSET points to data, macOS lseek+SEEK_DATA returns the
start S of the first data region that begins *after* OFFSET,
where the region from OFFSET to S consists of possibly-empty
data followed by a possibly-empty hole. To work around this
portability glitch, check whether OFFSET is within data by
using lseek+SEEK_HOLE, and if so return to OFFSET by using
lseek+SEEK_SET. Also, contrary to the macOS documentation,
lseek+SEEK_HOLE can fail with ENXIO if there are no holes on
or after OFFSET. What a mess! */
off_t next_hole = lseek (fd, offset, SEEK_HOLE);
if (next_hole < 0)
return errno == ENXIO ? offset : next_hole;
if (next_hole != offset)
whence = SEEK_SET;
}
#endif
return lseek (fd, offset, whence);
}
#define lseek rpl_lseek
#define MAX_SIZE (1024 * 1024 * 100)
char msg[MAX_SIZE];
int main(int argc, char ** argv)
{
// sparse copy test
int src = open("cc1", O_RDONLY);
int dst = open("cc1-sparse", O_CREAT | O_TRUNC | O_RDWR, 0700);
printf("__APPLE__ %u __MACH__ %u SEEK_DATA %u\n", __APPLE__, __MACH__, SEEK_DATA);
printf("src %i dst %i\n", src, dst);
//- printf("SET %i CUR %i END %i HOLE %i DATA %i\n", SEEK_SET, SEEK_CUR, SEEK_END, SEEK_HOLE, SEEK_DATA);
long long a = 0;
long long b = 0;
long long d = 0;
long long e = 0;
long long h = 0;
long long p = 0;
long long s = -1;
long long t = 0;
ssize_t c = 0;
ssize_t i = 0;
ssize_t r = 0;
ssize_t w = 0;
int ea = 0;
int eb = 0;
int ed = 0;
int ee = 0;
int eh = 0;
int ep = 0;
do
{
if (++i >= 10)
{
printf("LOOP %zi\n", i);
break;
}
errno = 0;
d = lseek(src, d, SEEK_DATA);
ed = errno;
h = lseek(src, d, SEEK_HOLE);
eh = errno;
a = lseek(src, d, SEEK_SET);
ea = errno;
b = lseek(dst, d, SEEK_SET);
eb = errno;
c = h - d;
if ((a == -1) || (b == -1) || (d == -1) || (h == -1))
{
int handled = 0;
if ((d == -1) && (ed == ENXIO))
{
p = lseek(src, 0, SEEK_END);
ep = errno;
if (p == -1)
{
printf(
"lseek(SEEK_END) failed p %lli %2i %s\n",
p, ep, strerror(ep)
);
}
else
{
e = ftruncate(dst, p);
ee = errno;
if (e == -1)
{
printf(
"ftruncate(%lli) failed %lli %2i %s\n",
p, e, ee, strerror(ee)
);
}
else
{
handled = 1;
}
}
}
if (!handled)
{
printf(
"lseek failed"
" ENXIO %i EBADF %i EINVAL %i EOVERFLOW %i ESPIPE %i\n"
" p %llx\n"
" d %llx %2i %s\n"
" h %llx %2i %s\n"
" a %llx %2i %s\n"
" b %llx %2i %s\n",
ENXIO, EBADF, EINVAL, EOVERFLOW, ESPIPE,
p,
d, ed, strerror(ed),
h, eh, strerror(eh),
a, ea, strerror(ea),
b, eb, strerror(eb)
);
}
break;
}
if (c < 0)
{
printf("negative size c %zi = h %lli - d %lli\n", c, h, d);
continue;
}
if (s == d)
{
printf("EOF %llu %llu\n", s, d);
break;
}
s = d;
if (c > sizeof(msg))
{
printf("msg too small %zu %zi\n", sizeof(msg), c);
break;
}
r = read(src, msg, c);
w = write(dst, msg, c);
if ((r != c) || (w != c))
{
printf("expected %zi got %zi %zi\n", c, r, w);
break;
}
p = lseek(src, 0, SEEK_CUR);
t += c;
printf("c %zx p %llx h %llx d %llx\n", c, p, h, d);
d = p;
} while (1);
close(src);
close(dst);
printf("total bytes copied %llx / %llx\n", t, p);
return 0;
}
--Apple-Mail=_4EF4D2CC-EE74-4D00-9418-5CC293358930
Content-Disposition: attachment;
filename=f.c
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="f.c"
Content-Transfer-Encoding: 7bit
#include <errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <unistd.h>
#include <sys/clonefile.h>
#define MAX_SIZE (1024 * 1024 * 100)
#define BLOCK_SIZE (4096)
char msg[MAX_SIZE];
char zero[BLOCK_SIZE];
int main(int argc, char ** argv)
{
// sparse fill test
// block size i 4 KB
int src = open("cc1", O_RDONLY);
int dst = open("B", O_CREAT | O_TRUNC | O_RDWR, 0700);
printf("src %i dst %i\n", src, dst);
//- printf("SET %i CUR %i END %i HOLE %i DATA %i\n", SEEK_SET, SEEK_CUR, SEEK_END, SEEK_HOLE, SEEK_DATA);
const char * p = msg;
const char * n = msg;
const char * e = msg;
off_t a = 0;
off_t b = 0;
off_t s = 0;
off_t t = 0;
ssize_t c = 0;
ssize_t i = 0;
ssize_t r = 0;
ssize_t w = 0;
int ea = 0;
int eb = 0;
int es = 0;
int et = 0;
errno = 0;
s = lseek(src, 0, SEEK_END);
es = errno;
if (s == -1)
{
printf(
"lseek(SEEK_END) failed s %lli %2i %s\n",
s, es, strerror(es)
);
return 1;
}
if (s > sizeof(msg))
{
printf("msg too small %zu %lli\n", sizeof(msg), s);
return 2;
}
t = ftruncate(dst, s);
et = errno;
if (t == -1)
{
printf(
"ftruncate(%lli) failed %lli %2i %s\n",
s, t, et, strerror(et)
);
return 3;
}
a = lseek(src, 0, SEEK_SET);
ea = errno;
if (a == -1)
{
printf(
"lseek(SEEK_SET) failed a %lli %2i %s\n",
a, ea, strerror(ea)
);
return 4;
}
// read all data
r = read(src, msg, s);
if (r != s)
{
printf("expected %zi read %zi\n", c, r);
return 5;
}
memset(zero, 0, sizeof(zero));
for (e = msg + s; p < e; p = n)
{
errno = 0;
n = p + BLOCK_SIZE;
if (n > e)
{
n = e;
}
c = n - p;
if (memcmp(zero, p, c))
{
// write valid sector
//- printf("%08zx copy\n", p - msg);
w = write(dst, p, c);
if (w != c)
{
printf("expected %zi wrote %zi\n", c, w);
break;
}
i += c;
}
else
{
// skip empty sector
printf("%08zx skip\n", p - msg);
b = lseek(dst, c, SEEK_CUR);
eb = errno;
if (b == -1)
{
printf(
"lseek(SEEK_CUR) failed b %lli %2i %s\n",
b, eb, strerror(eb)
);
break;
}
}
}
b = lseek(src, 0, SEEK_CUR);
close(src);
close(dst);
printf("total bytes copied %zx / %llx %llx\n", i, b, s);
return 0;
}
--Apple-Mail=_4EF4D2CC-EE74-4D00-9418-5CC293358930
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=us-ascii
Georgi Valkov
httpstorm.com
nano RTOS
--Apple-Mail=_4EF4D2CC-EE74-4D00-9418-5CC293358930--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 15 Feb 2023 19:48:43 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 15 14:48:43 2023 Received: from localhost ([127.0.0.1]:34101 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pSNli-0002oz-Lw for submit <at> debbugs.gnu.org; Wed, 15 Feb 2023 14:48:43 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:35376) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pSNlg-0002oj-Ox for 61386 <at> debbugs.gnu.org; Wed, 15 Feb 2023 14:48:41 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id CC36616006A; Wed, 15 Feb 2023 11:48:33 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Xg-Vshie57GZ; Wed, 15 Feb 2023 11:48:33 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id DF9EF160066; Wed, 15 Feb 2023 11:48:32 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu DF9EF160066 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676490512; bh=QgoHyI/pketRArPt5GU//L+mGq2yYQybxWh9M3QYcZE=; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type: Content-Transfer-Encoding; b=OBXq7nqM4DB7bkRluVDKFrHueVCqRjir230zbelpD7GAsT9+DItHJ2ZiObWYl7kZb rEmQb3GrbEF3DLizqTdbop+pTyRxIxpMbry9G/eLqhXSykCHobbEw8EKcQ3M2fRU/1 WULe+UeeDA3U20wcvByTemHbQSY8LWu0A46XPgEU= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id EdTS8ByrXJxG; Wed, 15 Feb 2023 11:48:32 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id B517E160041; Wed, 15 Feb 2023 11:48:32 -0800 (PST) Message-ID: <fac9d09e-312c-969a-440b-24db8054945c@HIDDEN> Date: Wed, 15 Feb 2023 11:48:32 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <45adda7f-4654-e186-7052-c14612415c22@HIDDEN> <354b1398-3c1c-9f83-1209-837825b9a85f@HIDDEN> <acc021fa-5983-2026-63fd-94073902b756@HIDDEN> <EC5AAB70-963A-44DA-ADDA-545D0DDE4753@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <EC5AAB70-963A-44DA-ADDA-545D0DDE4753@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) On 2023-02-15 07:26, George Valkov wrote: > I tested your patch: both overwrite existing and clone to new produce a working copy. Here are the test results: > https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-04-cf80f988eeb97cc3f8c65ae58e735d36f865277b-clone.txt I see some test failures there, involving cp. Do you get the same set of test failures without the patch? > In the case when a dangling symlink is involved and depending on the arguments passed to cp, would it be possible to use CLONE_NOFOLLOW with fclonefileat or fall-back to a normal copy? Does dangling refer to source or destination? Dangling refers to the destination. The latest proposed patch does use CLONE_NOFOLLOW, which means fclonefileat should not follow symlinks to the destination. (This behavior is not documented, unfortunately, but it's the only behavior that makes sense.)
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 15 Feb 2023 19:26:49 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 15 14:26:49 2023 Received: from localhost ([127.0.0.1]:34060 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pSNQW-0002B1-Mi for submit <at> debbugs.gnu.org; Wed, 15 Feb 2023 14:26:49 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:33086) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pSNQT-0002Ak-4e for 61386 <at> debbugs.gnu.org; Wed, 15 Feb 2023 14:26:46 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 06BE4160066; Wed, 15 Feb 2023 11:26:39 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id rgyW7vPnIdcW; Wed, 15 Feb 2023 11:26:37 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 78C5D16006A; Wed, 15 Feb 2023 11:26:37 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 78C5D16006A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676489197; bh=c+CERfN+K+y5ppF/oimYMuf+/B7a9EXy9fS7b6QmvcY=; h=Content-Type:Message-ID:Date:MIME-Version:To:From:Subject; b=mJFKSDgk25aXCKd4DNK4AX6qglp/K3XeinV4mrnk8PA+ZwQA/eOIeuioSPkoeFdVn AaJMtSWKf8snTb4IAIgeMipYUaeSReYVA6XBw67jFNihcdYuYER4LXoyEO7Z1LOi5A TpLcfIEdQTKxeThSS+AQW84JNcOVhPHcxIV8/NO0= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id VBA58bVs_CRx; Wed, 15 Feb 2023 11:26:37 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 4A334160066; Wed, 15 Feb 2023 11:26:37 -0800 (PST) Content-Type: multipart/mixed; boundary="------------cUpCM8TfyyM0Bh05rV13ojaW" Message-ID: <3caf257f-e085-929f-eaad-f05bc65d3a98@HIDDEN> Date: Wed, 15 Feb 2023 11:26:36 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS In-Reply-To: <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN> X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) This is a multi-part message in MIME format. --------------cUpCM8TfyyM0Bh05rV13ojaW Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable On 2023-02-15 06:05, George Valkov wrote: > gcc d.c && ./a.out > src 3 dst 4 > c 14053376 p 20344832 h 20344832 d 6291456 > total bytes copied 14053376 / 27551296 Thanks, this is due to a known incompatibility in macOS lseek that=20 coreutils is supposed to work around. See=20 <https://www.gnu.org/software/gnulib/manual/html_node/lseek.html>, which=20 says, "On some platforms, lseek (fd, offset, SEEK_DATA) returns a value=20 greater than offset even when offset addresses data: macOS 12". I guess that somehow, the way you're building coreutils defeats the=20 workaround. If so, we'll need to change how coreutils is built in your=20 environment, or fix coreutils 'configure' so that the workaround isn't=20 defeated in your environment. Although in=20 <https://bugs.gnu.org/61386#128> P=C3=A1draig was dubious about this gues= s,=20 his reasoning that the bug is likely specific to APFS rather than an API=20 mismatch could be wrong, as I think HFS doesn't support SEEK_DATA at all=20 or reports trivial answers, so coreutils is not likely to run into the=20 problem on HFS even if the bug is an API issue. Here are some things we can do to test this guess. 1. Please try the attached program e.c in place of your d.c program. e.c=20 is like d.c, except it attempts to use the coreutils workaround. What=20 symptoms do you observe? If e.c works then it's almost surely a problem=20 in how coreutils is built (compiler options or whatnot), not in the=20 coreutils workaround. If e.c does not work it's likely that the Gnulib=20 workaround does not suffice on your macOS platform, in which case we=20 need to improve the workaround by hacking further on e.c and porting the=20 result back to Gnulib. (There are other possibilities.) 2. Please verify that coreutils cp is using the Gnulib workaround. In=20 the src directory, the shell command "nm -o *.o | grep lseek" should=20 output only lines containing "rpl_lseek"; there shouldn't be any lines=20 saying just "lseek". Also, please run the command "objdump -d=20 lib/libcoreutils_a-lseek.o" and verify that the replacement lseek is=20 actually doing something nontrivial (you should get maybe three dozen=20 lines of assembly language; if it's much less then this is the problem). 3. Please confirm that _DARWIN_C_SOURCE is defined to 1 in lib/config.h. 4. What is the output of the following commands, in the coreutils build=20 directory? rm lib/libcoreutils_a-lseek.o make V=3D1 lib/libcoreutils_a-lseek.o gcc -E -Ilib lib/lseek.c --------------cUpCM8TfyyM0Bh05rV13ojaW Content-Type: text/x-csrc; charset=UTF-8; name="e.c" Content-Disposition: attachment; filename="e.c" Content-Transfer-Encoding: base64 I2RlZmluZSBfREFSV0lOX0NfU09VUkNFIDEKI2RlZmluZSBfR05VX1NPVVJDRSAxCiNpbmNs dWRlIDxlcnJuby5oPgojaW5jbHVkZSA8c3RkaW8uaD4KI2luY2x1ZGUgPGZjbnRsLmg+CiNp bmNsdWRlIDxzdHJpbmcuaD4KI2luY2x1ZGUgPHVuaXN0ZC5oPgojaW5jbHVkZSA8c3lzL2Ns b25lZmlsZS5oPgoKb2ZmX3QKcnBsX2xzZWVrIChpbnQgZmQsIG9mZl90IG9mZnNldCwgaW50 IHdoZW5jZSkKewojaWYgZGVmaW5lZCBfX0FQUExFX18gJiYgZGVmaW5lZCBfX01BQ0hfXyAm JiBkZWZpbmVkIFNFRUtfREFUQQogIGlmICh3aGVuY2UgPT0gU0VFS19EQVRBKQogICAgewog ICAgICAvKiBJZiBPRkZTRVQgcG9pbnRzIHRvIGRhdGEsIG1hY09TIGxzZWVrK1NFRUtfREFU QSByZXR1cm5zIHRoZQogICAgICAgICBzdGFydCBTIG9mIHRoZSBmaXJzdCBkYXRhIHJlZ2lv biB0aGF0IGJlZ2lucyAqYWZ0ZXIqIE9GRlNFVCwKICAgICAgICAgd2hlcmUgdGhlIHJlZ2lv biBmcm9tIE9GRlNFVCB0byBTIGNvbnNpc3RzIG9mIHBvc3NpYmx5LWVtcHR5CiAgICAgICAg IGRhdGEgZm9sbG93ZWQgYnkgYSBwb3NzaWJseS1lbXB0eSBob2xlLiAgVG8gd29yayBhcm91 bmQgdGhpcwogICAgICAgICBwb3J0YWJpbGl0eSBnbGl0Y2gsIGNoZWNrIHdoZXRoZXIgT0ZG U0VUIGlzIHdpdGhpbiBkYXRhIGJ5CiAgICAgICAgIHVzaW5nIGxzZWVrK1NFRUtfSE9MRSwg YW5kIGlmIHNvIHJldHVybiB0byBPRkZTRVQgYnkgdXNpbmcKICAgICAgICAgbHNlZWsrU0VF S19TRVQuICBBbHNvLCBjb250cmFyeSB0byB0aGUgbWFjT1MgZG9jdW1lbnRhdGlvbiwKICAg ICAgICAgbHNlZWsrU0VFS19IT0xFIGNhbiBmYWlsIHdpdGggRU5YSU8gaWYgdGhlcmUgYXJl IG5vIGhvbGVzIG9uCiAgICAgICAgIG9yIGFmdGVyIE9GRlNFVC4gIFdoYXQgYSBtZXNzISAg Ki8KICAgICAgb2ZmX3QgbmV4dF9ob2xlID0gbHNlZWsgKGZkLCBvZmZzZXQsIFNFRUtfSE9M RSk7CiAgICAgIGlmIChuZXh0X2hvbGUgPCAwKQogICAgICAgIHJldHVybiBlcnJubyA9PSBF TlhJTyA/IG9mZnNldCA6IG5leHRfaG9sZTsKICAgICAgaWYgKG5leHRfaG9sZSAhPSBvZmZz ZXQpCiAgICAgICAgd2hlbmNlID0gU0VFS19TRVQ7CiAgICB9CiNlbmRpZgogIHJldHVybiBs c2VlayAoZmQsIG9mZnNldCwgd2hlbmNlKTsKfQojZGVmaW5lIGxzZWVrIHJwbF9sc2VlawoK I2RlZmluZSBNQVhfU0laRSAoMTAyNCAqIDEwMjQgKiAxMDApCmNoYXIgbXNnW01BWF9TSVpF XTsKCmludCBtYWluKGludCBhcmdjLCBjaGFyICoqIGFyZ3YpCnsKCS8vIHNwYXJzZSBjb3B5 IHRlc3QKCWludCBzcmMgPSBvcGVuKCJjYzEiLCBPX1JET05MWSk7CglpbnQgZHN0ID0gb3Bl bigiY2MxLXNwYXJzZSIsIE9fQ1JFQVQgfCBPX1JEV1IsIDA3MDApOwoJcHJpbnRmKCJzcmMg JWkgIGRzdCAlaVxuIiwgc3JjLCBkc3QpOwoJLy8tIHByaW50ZigiU0VUICVpICBDVVIgJWkg IEVORCAlaSAgSE9MRSAlaSAgREFUQSAlaVxuIiwgU0VFS19TRVQsIFNFRUtfQ1VSLCBTRUVL X0VORCwgU0VFS19IT0xFLCBTRUVLX0RBVEEpOwoKCWxvbmcgbG9uZyBhID0gMDsKCWxvbmcg bG9uZyBiID0gMDsKCWxvbmcgbG9uZyBkID0gMDsKCWxvbmcgbG9uZyBlID0gMDsKCWxvbmcg bG9uZyBoID0gMDsKCWxvbmcgbG9uZyBwID0gMDsKCWxvbmcgbG9uZyBzID0gLTE7Cglsb25n IGxvbmcgdCA9IDA7Cglzc2l6ZV90IGMgPSAwOwoJc3NpemVfdCBpID0gMDsKCXNzaXplX3Qg ciA9IDA7Cglzc2l6ZV90IHcgPSAwOwoJaW50IGVhID0gMDsKCWludCBlYiA9IDA7CglpbnQg ZWQgPSAwOwoJaW50IGVlID0gMDsKCWludCBlaCA9IDA7CglpbnQgZXAgPSAwOwoKCWRvCgl7 CgkJaWYgKCsraSA+PSAxMCkKCQl7CgkJCXByaW50ZigiTE9PUCAlemlcbiIsIGkpOwoJCQli cmVhazsKCQl9CgoJCWVycm5vID0gMDsKCQlkID0gbHNlZWsoc3JjLCBkLCBTRUVLX0RBVEEp OwoJCWVkID0gZXJybm87CgkJaCA9IGxzZWVrKHNyYywgZCwgU0VFS19IT0xFKTsKCQllaCA9 IGVycm5vOwoJCWEgPSBsc2VlayhzcmMsIGQsIFNFRUtfU0VUKTsKCQllYSA9IGVycm5vOwoJ CWIgPSBsc2Vlayhkc3QsIGQsIFNFRUtfU0VUKTsKCQllYiA9IGVycm5vOwoJCWMgPSBoIC0g ZDsKCgkJaWYgKChhID09IC0xKSB8fCAoYiA9PSAtMSkgfHwgKGQgPT0gLTEpIHx8IChoID09 IC0xKSkKCQl7CgkJCWludCBoYW5kbGVkID0gMDsKCgkJCWlmICgoZCA9PSAtMSkgJiYgKGVk ID09IEVOWElPKSkKCQkJewoJCQkJcCA9IGxzZWVrKHNyYywgMCwgU0VFS19FTkQpOwoJCQkJ ZXAgPSBlcnJubzsKCgkJCQlpZiAocCA9PSAtMSkKCQkJCXsKCQkJCQlwcmludGYoCgkJCQkJ CSJsc2VlayhTRUVLX0VORCkgZmFpbGVkICBwICVsbGkgJTJpICVzXG4iLAoJCQkJCQlwLCBl cCwgc3RyZXJyb3IoZXApCgkJCQkJKTsKCQkJCX0KCQkJCWVsc2UKCQkJCXsKCQkJCQllID0g ZnRydW5jYXRlKGRzdCwgcCk7CgkJCQkJZWUgPSBlcnJubzsKCgkJCQkJaWYgKGUgPT0gLTEp CgkJCQkJewoJCQkJCQlwcmludGYoCgkJCQkJCQkiZnRydW5jYXRlKCVsbGkpIGZhaWxlZCAg JWxsaSAlMmkgJXNcbiIsCgkJCQkJCQlwLCBlLCBlZSwgc3RyZXJyb3IoZWUpCgkJCQkJCSk7 CgkJCQkJfQoJCQkJCWVsc2UKCQkJCQl7CgkJCQkJCWhhbmRsZWQgPSAxOwoJCQkJCX0KCQkJ CX0KCQkJfQoKCQkJaWYgKCFoYW5kbGVkKQoJCQl7CgkJCQlwcmludGYoCgkJCQkJImxzZWVr IGZhaWxlZCIKCQkJCQkiICBFTlhJTyAlaSAgRUJBREYgJWkgIEVJTlZBTCAlaSAgRU9WRVJG TE9XICVpICBFU1BJUEUgJWlcbiIKCQkJCQkiICBwICVsbGlcbiIKCQkJCQkiICBkICVsbGkg JTJpICVzXG4iCgkJCQkJIiAgaCAlbGxpICUyaSAlc1xuIgoJCQkJCSIgIGEgJWxsaSAlMmkg JXNcbiIKCQkJCQkiICBiICVsbGkgJTJpICVzXG4iLAoJCQkJCUVOWElPLCBFQkFERiwgRUlO VkFMLCBFT1ZFUkZMT1csIEVTUElQRSwKCQkJCQlwLAoJCQkJCWQsIGVkLCBzdHJlcnJvcihl ZCksCgkJCQkJaCwgZWgsIHN0cmVycm9yKGVoKSwKCQkJCQlhLCBlYSwgc3RyZXJyb3IoZWEp LAoJCQkJCWIsIGViLCBzdHJlcnJvcihlYikKCQkJCSk7CgkJCX0KCgkJCWJyZWFrOwoJCX0K CgkJaWYgKGMgPCAwKQoJCXsKCQkJcHJpbnRmKCJuZWdhdGl2ZSBzaXplIGMgJXppID0gaCAl bGxpIC0gZCAlbGxpXG4iLCBjLCBoLCBkKTsKCQkJY29udGludWU7CgkJfQoKCQlpZiAocyA9 PSBkKQoJCXsKCQkJcHJpbnRmKCJFT0YgJWxsdSAlbGx1XG4iLCBzLCBkKTsKCQkJYnJlYWs7 CgkJfQoKCQlzID0gZDsKCgkJaWYgKGMgPiBzaXplb2YobXNnKSkKCQl7CgkJCXByaW50Zigi bXNnIHRvbyBzbWFsbCAlenUgJXppXG4iLCBzaXplb2YobXNnKSwgYyk7CgkJCWJyZWFrOwoJ CX0KCgkJciA9IHJlYWQoc3JjLCBtc2csIGMpOwoJCXcgPSB3cml0ZShkc3QsIG1zZywgYyk7 CgoJCWlmICgociAhPSBjKSB8fCAodyAhPSBjKSkKCQl7CgkJCXByaW50ZigiZXhwZWN0ZWQg JXppICBnb3QgJXppICV6aVxuIiwgYywgciwgdyk7CgkJCWJyZWFrOwoJCX0KCgkJcCA9IGxz ZWVrKHNyYywgMCwgU0VFS19DVVIpOwoJCXQgKz0gYzsKCQlwcmludGYoImMgJXppICBwICVs bHUgIGggJWxsdSAgZCAlbGx1XG4iLCBjLCBwLCBoLCBkKTsKCQlkID0gcDsKCX0gd2hpbGUg KDEpOwoKCWNsb3NlKHNyYyk7CgljbG9zZShkc3QpOwoKCXByaW50ZigidG90YWwgYnl0ZXMg Y29waWVkICVsbHUgLyAlbGxpXG4iLCB0LCBwKTsKCglyZXR1cm4gMDsKfQo= --------------cUpCM8TfyyM0Bh05rV13ojaW--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 15 Feb 2023 15:26:33 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 15 10:26:32 2023 Received: from localhost ([127.0.0.1]:33637 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pSJg0-0007Fa-Hz for submit <at> debbugs.gnu.org; Wed, 15 Feb 2023 10:26:32 -0500 Received: from mail-wm1-f41.google.com ([209.85.128.41]:39667) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pSJfy-0007FN-8c for 61386 <at> debbugs.gnu.org; Wed, 15 Feb 2023 10:26:31 -0500 Received: by mail-wm1-f41.google.com with SMTP id hg24-20020a05600c539800b003e1f5f2a29cso1830801wmb.4 for <61386 <at> debbugs.gnu.org>; Wed, 15 Feb 2023 07:26:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1676474784; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=RqVhE+p3qd4ljiEBMvqb4yN+TLLDyRdzdc0ATA213eM=; b=T7sIMcN95hLAD6vA7Vq2SGfWQIbl2juTqcCtq6Ud/ZKdRItme+W6OKE6VfFqKCiALD tcm3sM7FjRyZLrh9IeRg9kbBtwXDaNQ22Yi3zJtOx0qJYAS9/pty5pq0HK+eEgpMapW+ 8GfANaoavzPMJ1F6QRb9Tc+6w44M5oFkrRMQ0Qzbt8q36ahe/zb+v6yoSP3qWuXm552A jkcR4lef7VomWamo5vgdy2v5rp09GFYVnuwZQkcFFFybVDOiAxTPe640YyATuqqstWkk gFKBKSWI1E7OYLrPsiwTgYwtaHTaJxQj2gDibJNLQT3Trn4iZtGSq+6EqfD81qmoOFDl hxKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1676474784; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RqVhE+p3qd4ljiEBMvqb4yN+TLLDyRdzdc0ATA213eM=; b=eUsmY53UALJKDCqZGzTvG3oEHjItXtOg2H74Cx/FUBruTJGD/hoRLJElRdznLptso3 K+eJKlk5knA5XftFhtLZ2xF4K3Lr6wQwJox/xX2b6/PCNbu4O7ahQYd0CqOk5Q05gGBj fd0/kMft0P9oF3w/DEmZY0JqX7B+jWKhQpO//ddUkaTws9CyndolmUCIsbPEiKp7qD4t 73dSOaHaTJsy2p/+tZIyC9eIPcwQ12kYyVO98LxTwhW11IPGLh2xf1QKlFB2B8RxmNBh wOnQB1MQ/DaU2xK3rfM8NZPFu9c16Y8s0kk/ZT41U1cOrGDF/96w42mbvXVnrkwVXmMB S9qQ== X-Gm-Message-State: AO0yUKWJWFUbXYynTUOX54qyD1bH045DI67qsOJlIuhbokSfs/Oxl83k uzUad8xwYwnM/o4uDTim6FU= X-Google-Smtp-Source: AK7set9yAAIoaBl6sOmu6ooSwtSa+EGhhC+/Mq1KC1o+oYeyhe2LC6AP2nDLS8TiwlKkjnSVhX4giQ== X-Received: by 2002:a05:600c:b87:b0:3e2:415:f09f with SMTP id fl7-20020a05600c0b8700b003e20415f09fmr1861605wmb.3.1676474784152; Wed, 15 Feb 2023 07:26:24 -0800 (PST) Received: from smtpclient.apple (212-39-89-241.ip.btc-net.bg. [212.39.89.241]) by smtp.gmail.com with ESMTPSA id g17-20020a056000119100b002c54e26bca5sm11303057wrx.49.2023.02.15.07.26.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Feb 2023 07:26:23 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <acc021fa-5983-2026-63fd-94073902b756@HIDDEN> Date: Wed, 15 Feb 2023 17:26:21 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <EC5AAB70-963A-44DA-ADDA-545D0DDE4753@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <45adda7f-4654-e186-7052-c14612415c22@HIDDEN> <354b1398-3c1c-9f83-1209-837825b9a85f@HIDDEN> <acc021fa-5983-2026-63fd-94073902b756@HIDDEN> To: Paul Eggert <eggert@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On 2023-02-15, at 12:56 PM, Paul Eggert <eggert@HIDDEN> wrote: >=20 > The disabling SEEK_HOLE on Apple is OK if a release is imminent. = Otherwise I'd like to get to the bottom of it first. This can be done by = having George use dtrace or (if that's not possible) adding debugging = printfs. Hi Paul! Sure, send me some code, describe the tests you need and I will = come back with the results. Since we are tracing regular programs and = not system components, dtruss can be used. Here is a trace of the sparse = copy sample I wrote earlier, when it tries to copy cc1 to cc1-sparse, = resulting in a corrupted copy: = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/dtruss-= sparse-copy.txt > Attached is an updated proposal for the fclonefileat patch. >=20 > CVE-2021-30995 confirmed my suspicion that coreutils 9.1 and the = current bleeding-edge coreutils on Savannah both have an exploitable = security bug on macOS. Although I hope this patch fixes the bug, it = could use more pairs of eyes, given that Apple had so many problems = fixing its own security bug involving fclonefileat, and given that the = coreutils code has so many flags and conditions. > <0001-cp-fclonefileat-security-fix-CLONE_ACL-fixups.patch> I tested your patch: both overwrite existing and clone to new produce a = working copy. Here are the test results: = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-04= -cf80f988eeb97cc3f8c65ae58e735d36f865277b-clone.txt In the case when a dangling symlink is involved and depending on the = arguments passed to cp, would it be possible to use CLONE_NOFOLLOW with = fclonefileat or fall-back to a normal copy? Does dangling refer to = source or destination? Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 15 Feb 2023 14:05:36 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 15 09:05:36 2023
Received: from localhost ([127.0.0.1]:58316 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pSIPg-0004AZ-2D
for submit <at> debbugs.gnu.org; Wed, 15 Feb 2023 09:05:36 -0500
Received: from mail-wm1-f51.google.com ([209.85.128.51]:34381)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pSIPb-0004AC-3g
for 61386 <at> debbugs.gnu.org; Wed, 15 Feb 2023 09:05:34 -0500
Received: by mail-wm1-f51.google.com with SMTP id
f23-20020a05600c491700b003dff4480a17so2464216wmp.1
for <61386 <at> debbugs.gnu.org>; Wed, 15 Feb 2023 06:05:31 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:from:to:cc:subject:date:message-id:reply-to;
bh=VPmyjikxgNiuo2x2EzRopD09BR9weupcW4xYHg1KyLM=;
b=NVbkpCPGmxUSoC/mNpchO+1xJYFPZkmKUta2Jg6V83eNbFAnGA05qh05bGkVG+02pG
HRClajPRIWHYMQdmGIl2vsjIp3h6yiXuadIefJVvl9pHaWzPOIu+COOEVTUhjCWqWyqH
z3ZxDt++X4gxBsOB9/0cOPmfzysulWUz+ro1oXBp9Ep8l5zsdlz3syWY3a6RIKMXi/9o
BGItLE3S7yMSMtbU2v5pMPBQGDUL8Jh5dpcb1lOyEP/EaLX3j2FOfF72i7QX+v4AYLau
ixroVZC9xzC952q7wryElpkukm2a3hgd53PjUgBke6m4/eVvZlx02O9zbNJ4cNVDEUtd
au9w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=VPmyjikxgNiuo2x2EzRopD09BR9weupcW4xYHg1KyLM=;
b=gKFIeCLggdLcGxA/BGVv/wUuwjE3ZD5jSzrIKUOofT43WVxTPAMJc4ry/Vjao7Ou4Z
/neVjfllhuNPzDUj0xwBw9vCLOfNJsLkk4AV02ufscaj7v1jjQIW/LN6giL4tZfA0tX3
VbILUjad5gY49SaDT+uHf2fE/7qB22QKFVCLVEdxLmkGWyRo6mtpXJadQmwvIRre8dDN
/nJqjJJBSRfR6jNgOP8RAI+vhWjdxzfwRJNq2QJNjGuhjS7EEpzswayADhDOd0mbUf15
S/Dw7lmJlk4ip4N2x/eSnlRJLL+dcDF39e3dqGrwH5tCTOCh7lJBOIsauO1vvl80Rlar
Vc/A==
X-Gm-Message-State: AO0yUKXUpzxVGy5q27DC1B2Gy3Imta882/f44+EkWBA4yEqMS+QkQ1M9
qwdDPztjB6HJ6ltEpWaNBNM=
X-Google-Smtp-Source: AK7set8DG0xw44+IC3bkRMyqPfpXimNIv2QluhmDDwvxd9/iLaQYY9Evtm3H9CSHIoGuqXBvNaXaDA==
X-Received: by 2002:a05:600c:3c8d:b0:3e1:bfc:f57b with SMTP id
bg13-20020a05600c3c8d00b003e10bfcf57bmr2528534wmb.2.1676469924846;
Wed, 15 Feb 2023 06:05:24 -0800 (PST)
Received: from smtpclient.apple (212-39-89-241.ip.btc-net.bg. [212.39.89.241])
by smtp.gmail.com with ESMTPSA id
r19-20020a05600c459300b003dc47d458cdsm2492041wmo.15.2023.02.15.06.05.23
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Wed, 15 Feb 2023 06:05:23 -0800 (PST)
From: George Valkov <gvalkov@HIDDEN>
Message-Id: <E138F1AF-24F6-43B3-92F6-8E7AE3657110@HIDDEN>
Content-Type: multipart/mixed;
boundary="Apple-Mail=_4300AA33-564C-4CA0-AF33-AB97DAF99D37"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Date: Wed, 15 Feb 2023 16:05:22 +0200
In-Reply-To: <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>,
Paul Eggert <eggert@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
<f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
<63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN>
<f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN>
<1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--Apple-Mail=_4300AA33-564C-4CA0-AF33-AB97DAF99D37
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
> On 2023-02-14, at 2:12 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 14/02/2023 06:12, Paul Eggert wrote:
>> On 2023-02-13 09:16, George Valkov wrote:
>>> 1. We apply my original patch to disable sparse-copy on macOS. =
Otherwise since fclonefileat is not used whenever we overwrite a file, =
corruption will still occur.
>> I'm not entirely sold on this patch, because I still don't understand
>> what's happening. The original bug report in
>> <https://github.com/openwrt/openwrt/pull/11960> basically says only =
"it
>> doesn't work", and I'd like to know more.
>> Part of the reason I'm hesitating is that there are multiple ways of
>> interpreting what SEEK_HOLE and SEEK_DATA mean, and perhaps it's just
>> that Apple has come up with yet another way to interpret it, which we
>> need to know about.
I created a sparse copy sample that traces all calls. It relays only on =
the manpage for lseek. I should note that I am not familiar with the =
implementation used by coreutils, which mages it a good ground for =
independent research on the topic. The code is attached as d.c Here are =
my observations:
1. Given a non-sparse file A, it produces an exact copy B.
gcc d.c && ./a.out
src 3 dst 4
c 553 p 553 h 553 d 0
total bytes copied 553 / 553
a2d8f888d5c88f6eef83a3bf4ef2434a85a64792 A
a2d8f888d5c88f6eef83a3bf4ef2434a85a64792 B
2. Given cc1, it produces a corrupted copy cc1-sparse, that matches the =
checksum of the copy cc-ori created by coreutils/src/cp. So unless you =
can find a flaw in my implementation, we can safely assume that =
SEEK_DATA skips valid data blocks and hence it should not be used on =
macOS because it is broken:
./coreutils/src/cp cc1 cc1-ori
gcc d.c && ./a.out
src 3 dst 4
c 14053376 p 20344832 h 20344832 d 6291456
total bytes copied 14053376 / 27551296
e8eb21ec118ff3a8fce3ad85d5f9a72d47a257c6 cc1
7b447132f56f3b549ef62a4780945e82d26f7d2c cc1-ori
7b447132f56f3b549ef62a4780945e82d26f7d2c cc1-sparse
-rwxr-xr-x 1 g staff 27551296 Feb 15 09:59 cc1*
-rwxr-xr-x 1 g staff 27551296 Feb 15 14:29 cc1-ori*
-rwx------ 1 g staff 27551296 Feb 15 14:29 cc1-sparse*
Further research: it would be interesting to find how my sparse copy =
sample, as well as coreutils/src/cp handle custom crafted sparse files. =
My first idea would be a file with the same size as cc1 (27 551 296 =
bytes) with a couple of data blocks in the middle. My second idea is to =
read the entire cc1, then call ftruncate to set the size of the copy and =
write only those bytes that are not 0.
> I agree it would be good to know more.
> However given it works on HFS but fails on APFS suggests a file system =
specific issue,
> rather than an API mismatch issue (over what we're already catering =
for on macOS).
> I suspect it's a similar issue to the one that openzfs had:
> https://github.com/openzfs/zfs/issues/11900
According to Wikipedia:
> Apple's HFS+ does not provide support for sparse files, but in OS X, =
the virtual file system layer supports storing them in any supported =
file system, including HFS+.
https://en.wikipedia.org/wiki/Sparse_file
Hence the reason we don=E2=80=99t observe any issues on HFS+ is most =
likely that the files we try to copy from it are not sparse in first =
place.
> Given how closed / uncommunicative Apple are in general
> and specifically for this already reported bug,
> I'm inclined to disable SEEK_DATA on __APPLE__ for the upcoming =
release.
>=20
> Also we've mitigated the impact somewhat by enabling fclonefileat() in =
more cases.
I agree. We can try to send them one last message with a link to this =
group, and be done with it if they don=E2=80=99t relay. I just found a =
feedback ticket from 2022-09-16, where they replayed asking me to test =
macOS 13 Beta 8. At that time Finder was still broken, and now it =
isn=E2=80=99t. So they might have at least partially addressed the =
issue.
https://feedbackassistant.apple.com/feedback/11522527
>> Another reason to hesitate is that GNU coreutils is not the only core
>> program that uses SEEK_HOLE and SEEK_DATA. If this really is a =
generic
>> Apple problem we need to know which Apple releases have it, so that =
we
>> can program around it at the Gnulib level instead of mucking about =
with
>> each individual program.
We need to start somewhere. Fixing cp and install will certainly be =
helpful to users, until you address the issue globally.
It might be possible to install VMs with various versions of macOS, =
though I suspect it will take time.
P=C3=A1draig, did your try cp on
=
build_dir/toolchain-arm_cortex-a9+vfpv3-d16_gcc-12.2.0_musl_eabi/gcc-12.2.=
0-initial/gcc/cc1
after building gcc in OpenWRT build root?
make toolchain/gcc/initial/{clean,compile} -j 16
Here is a step by step article how to setup the build environment:
=
https://httpstorm.com/share/.openwrt/help/OpenWRT%20build%20on%20macOS.htm=
> I've attached the 3 latest patches I'm considering in this area.
> I presume you're OK with your amended fclonefileat() improvement one?
Thank you! Yes, I=E2=80=99m all for the performance benefits of using =
fclonefileat. I applied your patches on top of =
cf80f988eeb97cc3f8c65ae58e735d36f865277b:
Both rewriting an existing file and cloning to a new one produce a =
correct copies of a sparse source. Here are the test results:
=
https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-03=
-cf80f988eeb97cc3f8c65ae58e735d36f865277b-clone.txt
I wrote a sparse copy sample
--Apple-Mail=_4300AA33-564C-4CA0-AF33-AB97DAF99D37
Content-Disposition: attachment;
filename=d.c
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="d.c"
Content-Transfer-Encoding: 7bit
#include <errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <unistd.h>
#include <sys/clonefile.h>
#define MAX_SIZE (1024 * 1024 * 100)
char msg[MAX_SIZE];
int main(int argc, char ** argv)
{
// sparse copy test
int src = open("cc1", O_RDONLY);
int dst = open("cc1-sparse", O_CREAT | O_RDWR, 0700);
printf("src %i dst %i\n", src, dst);
//- printf("SET %i CUR %i END %i HOLE %i DATA %i\n", SEEK_SET, SEEK_CUR, SEEK_END, SEEK_HOLE, SEEK_DATA);
off_t a = 0;
off_t b = 0;
off_t d = 0;
off_t e = 0;
off_t h = 0;
off_t p = 0;
off_t s = -1;
off_t t = 0;
ssize_t c = 0;
ssize_t i = 0;
ssize_t r = 0;
ssize_t w = 0;
int ea = 0;
int eb = 0;
int ed = 0;
int ee = 0;
int eh = 0;
int ep = 0;
do
{
if (++i >= 10)
{
printf("LOOP %zi\n", i);
break;
}
errno = 0;
d = lseek(src, d, SEEK_DATA);
ed = errno;
h = lseek(src, d, SEEK_HOLE);
eh = errno;
a = lseek(src, d, SEEK_SET);
ea = errno;
b = lseek(dst, d, SEEK_SET);
eb = errno;
c = h - d;
if ((a == -1) || (b == -1) || (d == -1) || (h == -1))
{
int handled = 0;
if ((d == -1) && (ed == ENXIO))
{
p = lseek(src, 0, SEEK_END);
ep = errno;
if (p == -1)
{
printf(
"lseek(SEEK_END) failed p %lli %2i %s\n",
p, ep, strerror(ep)
);
}
else
{
e = ftruncate(dst, p);
ee = errno;
if (e == -1)
{
printf(
"ftruncate(%lli) failed %lli %2i %s\n",
p, e, ee, strerror(ee)
);
}
else
{
handled = 1;
}
}
}
if (!handled)
{
printf(
"lseek failed"
" ENXIO %i EBADF %i EINVAL %i EOVERFLOW %i ESPIPE %i\n"
" p %lli\n"
" d %lli %2i %s\n"
" h %lli %2i %s\n"
" a %lli %2i %s\n"
" b %lli %2i %s\n",
ENXIO, EBADF, EINVAL, EOVERFLOW, ESPIPE,
p,
d, ed, strerror(ed),
h, eh, strerror(eh),
a, ea, strerror(ea),
b, eb, strerror(eb)
);
}
break;
}
if (c < 0)
{
printf("negative size c %zi = h %lli - d %lli\n", c, h, d);
continue;
}
if (s == d)
{
printf("EOF %llu %llu\n", s, d);
break;
}
s = d;
if (c > sizeof(msg))
{
printf("msg too small %zu %zi\n", sizeof(msg), c);
break;
}
r = read(src, msg, c);
w = write(dst, msg, c);
if ((r != c) || (w != c))
{
printf("expected %zi got %zi %zi\n", c, r, w);
break;
}
p = lseek(src, 0, SEEK_CUR);
t += c;
printf("c %zi p %llu h %llu d %llu\n", c, p, h, d);
d = p;
} while (1);
close(src);
close(dst);
printf("total bytes copied %llu / %lli\n", t, p);
return 0;
}
--Apple-Mail=_4300AA33-564C-4CA0-AF33-AB97DAF99D37
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=us-ascii
Cheers!
Georgi Valkov
httpstorm.com
nano RTOS
--Apple-Mail=_4300AA33-564C-4CA0-AF33-AB97DAF99D37--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 15 Feb 2023 10:56:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 15 05:56:47 2023 Received: from localhost ([127.0.0.1]:57948 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pSFSx-0007wQ-59 for submit <at> debbugs.gnu.org; Wed, 15 Feb 2023 05:56:47 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:40746) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pSFSu-0007wC-Tq for 61386 <at> debbugs.gnu.org; Wed, 15 Feb 2023 05:56:46 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id CD825160041; Wed, 15 Feb 2023 02:56:37 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id vT0D6t5MFGon; Wed, 15 Feb 2023 02:56:36 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 5EA42160076; Wed, 15 Feb 2023 02:56:36 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 5EA42160076 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676458596; bh=UZz/Nj5QeKX2T2m9b2lu0Oz2HlxpVNMjGDXIgrIF/S8=; h=Content-Type:Message-ID:Date:MIME-Version:To:From:Subject; b=kQ4WhSJJcEH6W7wv2ZepuRd+wzus64/k0SYqWxSBgljm+bpY1flERWMVayiR7F4vH 7sPW9y0T7E7cS0puXqBM1eAsIp1sh0rJtkcn4Qkxum6Fz6NbjFV9ANznUXXQElJhue ZO0swjfVEiqMv5DWAmQs1RJHpSt3ayzHiwAgVIIs= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id G3MYNopk4KGZ; Wed, 15 Feb 2023 02:56:36 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 2CBA2160041; Wed, 15 Feb 2023 02:56:36 -0800 (PST) Content-Type: multipart/mixed; boundary="------------d3sasI5F9S5zzz6EZc8WbvfB" Message-ID: <acc021fa-5983-2026-63fd-94073902b756@HIDDEN> Date: Wed, 15 Feb 2023 02:56:35 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Content-Language: en-US To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <45adda7f-4654-e186-7052-c14612415c22@HIDDEN> <354b1398-3c1c-9f83-1209-837825b9a85f@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS In-Reply-To: <354b1398-3c1c-9f83-1209-837825b9a85f@HIDDEN> X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, George Valkov <gvalkov@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) This is a multi-part message in MIME format. --------------d3sasI5F9S5zzz6EZc8WbvfB Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Attached is an updated proposal for the fclonefileat patch. CVE-2021-30995 confirmed my suspicion that coreutils 9.1 and the current bleeding-edge coreutils on Savannah both have an exploitable security bug on macOS. Although I hope this patch fixes the bug, it could use more pairs of eyes, given that Apple had so many problems fixing its own security bug involving fclonefileat, and given that the coreutils code has so many flags and conditions. --------------d3sasI5F9S5zzz6EZc8WbvfB Content-Type: text/x-patch; charset=UTF-8; name="0001-cp-fclonefileat-security-fix-CLONE_ACL-fixups.patch" Content-Disposition: attachment; filename="0001-cp-fclonefileat-security-fix-CLONE_ACL-fixups.patch" Content-Transfer-Encoding: base64 RnJvbSA5YmI5ODAxNmViM2EzNjljM2FlMTk1YjcxZmNjMTAxMGU1YWE2ZTUzIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBGcmksIDEwIEZlYiAyMDIzIDEzOjM0OjU0IC0wODAwClN1YmplY3Q6IFtQQVRD SF0gY3A6IGZjbG9uZWZpbGVhdCBzZWN1cml0eSBmaXggKyBDTE9ORV9BQ0wgKyBmaXh1cHMK Ciogc3JjL2NvcHkuYzogU29tZSBjaGFuZ2VzIGlmIEhBVkVfRkNMT05FRklMRUFUICYmICFV U0VfWEFUVFIuCihmZF9oYXNfYWNsKTogTmV3IGZ1bmN0aW9uLgooQ0xPTkVfQUNMKTogRGVm YXVsdCB0byAwLgooY29weV9yZWcpOiBVc2UgQ0xPTkVfTk9GT0xMT1cgdG8gYXZvaWQgcmFj ZXMgbGlrZSBDVkUtMjAyMS0zMDk5NQo8aHR0cHM6Ly93d3cudHJlbmRtaWNyby5jb20vZW5f dXMvcmVzZWFyY2gvMjIvYS8KYW5hbHl6aW5nLWFuLW9sZC1idWctYW5kLWRpc2NvdmVyaW5n LWN2ZS0yMDIxLTMwOTk1LS5odG1sPi4KVXNlIENMT05FX0FDTCBpZiBhdmFpbGFibGUgYW5k IHdvcmtpbmcsIGZhbGxpbmcgYmFjayB0byBjbG9uaW5nCndpdGhvdXQgaXQgaWYgaXQgZmFp bHMgZHVlIHRvIEVJTlZBTC4KSWYgdGhlIG9ubHkgcHJvYmxlbSB3aXRoIGZjbG9uZWZpbGVh dCBpcyB0aGF0IGl0IHdvdWxkIGNyZWF0ZSB0aGUKZmlsZSB3aXRoIHRoZSB3cm9uZyB0aW1l c3RhbXAsIG9yIHdpdGggdG9vIGZldyBwZXJtaXNzaW9ucywKZG8gdGhhdCBidXQgZml4IHRo ZSB0aW1lc3RhbXAgYW5kIHBlcm1pc3Npb25zIGFmdGVyd2FyZHMsCnJhdGhlciB0aGFuIGZh bGxpbmcgYmFjayBvbiBhIHRyYWRpdGlvbmFsIGNvcHkuCi0tLQogTkVXUyAgICAgICB8ICAg NyArKysrCiBzcmMvY29weS5jIHwgMTA5ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKystLS0tLS0tCiAyIGZpbGVzIGNoYW5nZWQsIDEwMyBpbnNlcnRp b25zKCspLCAxMyBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9ORVdTIGIvTkVXUwppbmRl eCAzZDBlZGUxNTAuLmQ2NmVhMGY2ZiAxMDA2NDQKLS0tIGEvTkVXUworKysgYi9ORVdTCkBA IC0xMCw2ICsxMCwxMCBAQCBHTlUgY29yZXV0aWxzIE5FV1MgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAtKi0gb3V0bGluZSAtKi0KICAgdG90YWwgbGluZSBpbiB0aGlz IGNhc2UuCiAgIFtidWcgaW50cm9kdWNlZCB3aXRoIHRoZSAtLXRvdGFsIG9wdGlvbiBpbiBj b3JldXRpbHMtOC4yNl0KIAorICAnY3AgLXAnIG5vIGxvbmdlciBoYXMgYSBzZWN1cml0eSBo b2xlIHdoZW4gY2xvbmluZyBpbnRvIGEgZGFuZ2xpbmcKKyAgc3ltYm9saWMgbGluayBvbiBt YWNPUyAxMC4xMiBhbmQgbGF0ZXIuCisgIFtidWcgaW50cm9kdWNlZCBpbiBjb3JldXRpbHMt OS4xXQorCiAgICdjcCAtcnggLyAvbW50JyBubyBsb25nZXIgY29tcGxhaW5zICJjYW5ub3Qg Y3JlYXRlIGRpcmVjdG9yeSAvbW50LyIuCiAgIFtidWcgaW50cm9kdWNlZCBpbiBjb3JldXRp bHMtOS4xXQogCkBAIC0xMjQsNiArMTI4LDkgQEAgR05VIGNvcmV1dGlscyBORVdTICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLSotIG91dGxpbmUgLSotCiAgIGFuZCBw b3NzaWJseSBlbXBsb3lpbmcgY29weSBvZmZsb2FkaW5nIG9yIHJlZmxpbmtpbmcsCiAgIGZv ciB0aGUgbm9uIHNwYXJzZSBwb3J0aW9uIG9mIHN1Y2ggc3BhcnNlIGZpbGVzLgogCisgIE9u IG1hY09TLCBjcCBjcmVhdGVzIGEgY29weS1vbi13cml0ZSBjbG9uZSBpbiBtb3JlIGNhc2Vz LgorICBQcmV2aW91c2x5IGNwIHdvdWxkIG9ubHkgZG8gdGhpcyB3aGVuIHByZXNlcnZpbmcg bW9kZSBhbmQgdGltZXN0YW1wcy4KKwogICBkYXRlIC0tZGVidWcgbm93IGRpYWdub3NlcyBp ZiBtdWx0aXBsZSAtLWRhdGUgb3IgLS1zZXQgb3B0aW9ucyBhcmUKICAgc3BlY2lmaWVkLCBh cyBvbmx5IHRoZSBsYXN0IHNwZWNpZmllZCBpcyBzaWduaWZpY2FudCBpbiB0aGF0IGNhc2Uu CiAKZGlmZiAtLWdpdCBhL3NyYy9jb3B5LmMgYi9zcmMvY29weS5jCmluZGV4IGRmYmI1NTdk ZS4uYzVmMmNjMTg2IDEwMDY0NAotLS0gYS9zcmMvY29weS5jCisrKyBiL3NyYy9jb3B5LmMK QEAgLTEwNzYsNiArMTA3NiwyNSBAQCBpbmZlcl9zY2FudHlwZSAoaW50IGZkLCBzdHJ1Y3Qg c3RhdCBjb25zdCAqc2IsCiAgIHJldHVybiBaRVJPX1NDQU5UWVBFOwogfQogCisjaWYgSEFW RV9GQ0xPTkVGSUxFQVQgJiYgIVVTRV9YQVRUUgorIyBpbmNsdWRlIDxzeXMvYWNsLmg+Cisv KiBSZXR1cm4gdHJ1ZSBpZiBGRCBoYXMgYSBub250cml2aWFsIEFDTC4gICovCitzdGF0aWMg Ym9vbAorZmRfaGFzX2FjbCAoaW50IGZkKQoreworICAvKiBFdmVyeSBwbGF0Zm9ybSB3aXRo IGZjbG9uZWZpbGVhdCAobWFjT1MgMTAuMTIgb3IgbGF0ZXIpIGFsc28gaGFzCisgICAgIGFj bF9nZXRfZmRfbnAuICAqLworICBib29sIGhhc19hY2wgPSBmYWxzZTsKKyAgYWNsX3QgYWNs ID0gYWNsX2dldF9mZF9ucCAoZmQsIEFDTF9UWVBFX0VYVEVOREVEKTsKKyAgaWYgKGFjbCkK KyAgICB7CisgICAgICBhY2xfZW50cnlfdCBhY2U7CisgICAgICBoYXNfYWNsID0gMCA8PSBh Y2xfZ2V0X2VudHJ5IChhY2wsIEFDTF9GSVJTVF9FTlRSWSwgJmFjZSk7CisgICAgICBhY2xf ZnJlZSAoYWNsKTsKKyAgICB9CisgIHJldHVybiBoYXNfYWNsOworfQorI2VuZGlmCiAKIC8q IEhhbmRsZSBmYWlsdXJlIGZyb20gRklDTE9ORSBvciBmY2xvbmVmaWxlYXQuCiAgICBSZXR1 cm4gRkFMU0UgaWYgaXQncyBhIHRlcm1pbmFsIGZhaWx1cmUgZm9yIHRoaXMgZmlsZS4gICov CkBAIC0xMjQ0LDI0ICsxMjYzLDgyIEBAIGNvcHlfcmVnIChjaGFyIGNvbnN0ICpzcmNfbmFt ZSwgY2hhciBjb25zdCAqZHN0X25hbWUsCiAgIGlmICgqbmV3X2RzdCkKICAgICB7CiAjaWYg SEFWRV9GQ0xPTkVGSUxFQVQgJiYgIVVTRV9YQVRUUgotLyogQ0xPTkVfTk9PV05FUkNPUFkg b25seSBhdmFpbGFibGUgb24gbWFjb3MgPj0gMTAuMTMuICAqLworIyBpZm5kZWYgQ0xPTkVf QUNMCisjICBkZWZpbmUgQ0xPTkVfQUNMIDAgLyogQWRkZWQgaW4gbWFjT1MgMTIuNi4gICov CisjIGVuZGlmCiAjIGlmbmRlZiBDTE9ORV9OT09XTkVSQ09QWQotIyAgZGVmaW5lIENMT05F X05PT1dORVJDT1BZIDAKKyMgIGRlZmluZSBDTE9ORV9OT09XTkVSQ09QWSAwIC8qIEFkZGVk IGluIG1hY09TIDEwLjEzLiAgKi8KICMgZW5kaWYKLSAgICAgIGludCBmY19mbGFncyA9IHgt PnByZXNlcnZlX293bmVyc2hpcCA/IDAgOiBDTE9ORV9OT09XTkVSQ09QWTsKKyAgICAgIC8q IFRyeSBmY2xvbmVmaWxlYXQgaWYgY29weWluZyBkYXRhIGluIHJlZmxpbmsgbW9kZS4KKyAg ICAgICAgIFVzZSBDTE9ORV9OT0ZPTExPVyB0byBhdm9pZCBzZWN1cml0eSBpc3N1ZXMgdGhh dCBjb3VsZCBvY2N1cgorICAgICAgICAgaWYgd3JpdGluZyB0aHJvdWdoIGRhbmdsaW5nIHN5 bWxpbmtzLiAgQWx0aG91Z2ggdGhlIGNpcmNhCisgICAgICAgICAyMDIzIG1hY09TIGRvY3Vt ZW50YXRpb24gZG9lc24ndCBzYXkgc28sIENMT05FX05PRk9MTE9XCisgICAgICAgICBhZmZl Y3RzIHRoZSBkZXN0aW5hdGlvbiBmaWxlIHRvby4gICovCiAgICAgICBpZiAoZGF0YV9jb3B5 X3JlcXVpcmVkICYmIHgtPnJlZmxpbmtfbW9kZQotICAgICAgICAgICYmIHgtPnByZXNlcnZl X21vZGUgJiYgeC0+cHJlc2VydmVfdGltZXN0YW1wcwotICAgICAgICAgICYmICh4LT5wcmVz ZXJ2ZV9vd25lcnNoaXAgfHwgQ0xPTkVfTk9PV05FUkNPUFkpKQorICAgICAgICAgICYmIChD TE9ORV9OT09XTkVSQ09QWSB8fCB4LT5wcmVzZXJ2ZV9vd25lcnNoaXApKQogICAgICAgICB7 Ci0gICAgICAgICAgaWYgKGZjbG9uZWZpbGVhdCAoc291cmNlX2Rlc2MsIGRzdF9kaXJmZCwg ZHN0X3JlbG5hbWUsIGZjX2ZsYWdzKSA9PSAwKQotICAgICAgICAgICAgZ290byBjbG9zZV9z cmNfZGVzYzsKLSAgICAgICAgICBlbHNlIGlmICghIGhhbmRsZV9jbG9uZV9mYWlsIChkc3Rf ZGlyZmQsIGRzdF9yZWxuYW1lLCBzcmNfbmFtZSwKLSAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBkc3RfbmFtZSwKLSAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAtMSwgZmFsc2UgLyogV2UgZGlkbid0IGNyZWF0ZSBkc3QgICovLAot ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHgtPnJlZmxpbmtfbW9k ZSkpCisgICAgICAgICAgbW9kZV90IGNsb25lZF9tb2RlX2JpdHMgPSBTX0lTVlRYIHwgU19J UldYVUdPOworICAgICAgICAgIG1vZGVfdCBjbG9uZWRfbW9kZSA9IHNyY19tb2RlICYgY2xv bmVkX21vZGVfYml0czsKKyAgICAgICAgICBtb2RlX3QgZGVzaXJlZF9tb2RlCisgICAgICAg ICAgICA9ICh4LT5wcmVzZXJ2ZV9tb2RlID8gc3JjX21vZGUgJiBDSE1PRF9NT0RFX0JJVFMK KyAgICAgICAgICAgICAgIDogeC0+c2V0X21vZGUgPyB4LT5tb2RlCisgICAgICAgICAgICAg ICA6ICgoeC0+ZXhwbGljaXRfbm9fcHJlc2VydmVfbW9kZSA/IE1PREVfUldfVUdPIDogZHN0 X21vZGUpCisgICAgICAgICAgICAgICAgICAmIH4gY2FjaGVkX3VtYXNrICgpKSk7CisgICAg ICAgICAgaWYgKCEgKGNsb25lZF9tb2RlICYgfmRlc2lyZWRfbW9kZSkpCiAgICAgICAgICAg ICB7Ci0gICAgICAgICAgICAgIHJldHVybl92YWwgPSBmYWxzZTsKLSAgICAgICAgICAgICAg Z290byBjbG9zZV9zcmNfZGVzYzsKKyAgICAgICAgICAgICAgaW50IGZjX2ZsYWdzCisgICAg ICAgICAgICAgICAgPSAoQ0xPTkVfTk9GT0xMT1cKKyAgICAgICAgICAgICAgICAgICB8ICh4 LT5wcmVzZXJ2ZV9tb2RlID8gQ0xPTkVfQUNMIDogMCkKKyAgICAgICAgICAgICAgICAgICB8 ICh4LT5wcmVzZXJ2ZV9vd25lcnNoaXAgPyAwIDogQ0xPTkVfTk9PV05FUkNPUFkpKTsKKyAg ICAgICAgICAgICAgaW50IHMgPSBmY2xvbmVmaWxlYXQgKHNvdXJjZV9kZXNjLCBkc3RfZGly ZmQsIGRzdF9yZWxuYW1lLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ZmNfZmxhZ3MpOworICAgICAgICAgICAgICBpZiAocyAhPSAwICYmIChmY19mbGFncyAmIENM T05FX0FDTCkgJiYgZXJybm8gPT0gRUlOVkFMKQorICAgICAgICAgICAgICAgIHsKKyAgICAg ICAgICAgICAgICAgIGZjX2ZsYWdzICY9IH5DTE9ORV9BQ0w7CisgICAgICAgICAgICAgICAg ICBzID0gZmNsb25lZmlsZWF0IChzb3VyY2VfZGVzYywgZHN0X2RpcmZkLCBkc3RfcmVsbmFt ZSwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZjX2ZsYWdzKTsKKyAg ICAgICAgICAgICAgICB9CisgICAgICAgICAgICAgIGlmIChzID09IDApCisgICAgICAgICAg ICAgICAgeworICAgICAgICAgICAgICAgICAgLyogVXBkYXRlIHRoZSBjbG9uZSdzIHRpbWVz dGFtcHMgYW5kIHBlcm1pc3Npb25zCisgICAgICAgICAgICAgICAgICAgICBhcyBuZWVkZWQu ICAqLworCisgICAgICAgICAgICAgICAgICBpZiAoIXgtPnByZXNlcnZlX3RpbWVzdGFtcHMp CisgICAgICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3Qg dGltZXNwZWMgdGltZXNwZWNbMl07CisgICAgICAgICAgICAgICAgICAgICAgdGltZXNwZWNb MF0udHZfbnNlYyA9IHRpbWVzcGVjWzFdLnR2X25zZWMgPSBVVElNRV9OT1c7CisgICAgICAg ICAgICAgICAgICAgICAgaWYgKHV0aW1lbnNhdCAoZHN0X2RpcmZkLCBkc3RfcmVsbmFtZSwg dGltZXNwZWMsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgQVRfU1lN TElOS19OT0ZPTExPVykKKyAgICAgICAgICAgICAgICAgICAgICAgICAgIT0gMCkKKyAgICAg ICAgICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICAgICAgICAgICAgZXJyb3Ig KDAsIGVycm5vLCBfKCJ1cGRhdGluZyB0aW1lcyBmb3IgJXMiKSwKKyAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIHF1b3RlYWYgKGRzdF9uYW1lKSk7CisgICAgICAgICAgICAg ICAgICAgICAgICAgIHJldHVybl92YWwgPSBmYWxzZTsKKyAgICAgICAgICAgICAgICAgICAg ICAgICAgZ290byBjbG9zZV9zcmNfZGVzYzsKKyAgICAgICAgICAgICAgICAgICAgICAgIH0K KyAgICAgICAgICAgICAgICAgICAgfQorCisgICAgICAgICAgICAgICAgICBleHRyYV9wZXJt aXNzaW9ucyA9IGRlc2lyZWRfbW9kZSAmIH5jbG9uZWRfbW9kZTsKKyAgICAgICAgICAgICAg ICAgIGlmICghZXh0cmFfcGVybWlzc2lvbnMKKyAgICAgICAgICAgICAgICAgICAgICAmJiAo IXgtPnByZXNlcnZlX21vZGUgfHwgKGZjX2ZsYWdzICYgQ0xPTkVfQUNMKQorICAgICAgICAg ICAgICAgICAgICAgICAgICB8fCAhZmRfaGFzX2FjbCAoc291cmNlX2Rlc2MpKSkKKyAgICAg ICAgICAgICAgICAgICAgeworICAgICAgICAgICAgICAgICAgICAgIC8qIFBlcm1pc3Npb25z IGFyZSBhbHJlYWR5IE9LLiAgKi8KKyAgICAgICAgICAgICAgICAgICAgICBnb3RvIGNsb3Nl X3NyY19kZXNjOworICAgICAgICAgICAgICAgICAgICB9CisKKyAgICAgICAgICAgICAgICAg IG9taXR0ZWRfcGVybWlzc2lvbnMgPSAwOworICAgICAgICAgICAgICAgICAgZGVzdF9kZXNj ID0gLTE7CisgICAgICAgICAgICAgICAgICBnb3RvIHNldF9kZXN0X21vZGU7CisgICAgICAg ICAgICAgICAgfQorICAgICAgICAgICAgICBpZiAoISBoYW5kbGVfY2xvbmVfZmFpbCAoZHN0 X2RpcmZkLCBkc3RfcmVsbmFtZSwgc3JjX25hbWUsCisgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBkc3RfbmFtZSwKKyAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIC0xLCBmYWxzZSAvKiBXZSBkaWRuJ3QgY3JlYXRlIGRzdCAgKi8sCisg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB4LT5yZWZsaW5rX21vZGUp KQorICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICAgIHJldHVybl92YWwgPSBm YWxzZTsKKyAgICAgICAgICAgICAgICAgIGdvdG8gY2xvc2Vfc3JjX2Rlc2M7CisgICAgICAg ICAgICAgICAgfQogICAgICAgICAgICAgfQogICAgICAgICB9CiAjZW5kaWYKQEAgLTE0ODUs NiArMTU2Miw5IEBAIGNvcHlfcmVnIChjaGFyIGNvbnN0ICpzcmNfbmFtZSwgY2hhciBjb25z dCAqZHN0X25hbWUsCiAKICAgc2V0X2F1dGhvciAoZHN0X25hbWUsIGRlc3RfZGVzYywgc3Jj X3NiKTsKIAorI2lmIEhBVkVfRkNMT05FRklMRUFUICYmICFVU0VfWEFUVFIKK3NldF9kZXN0 X21vZGU6CisjZW5kaWYKICAgaWYgKHgtPnByZXNlcnZlX21vZGUgfHwgeC0+bW92ZV9tb2Rl KQogICAgIHsKICAgICAgIGlmIChjb3B5X2FjbCAoc3JjX25hbWUsIHNvdXJjZV9kZXNjLCBk c3RfbmFtZSwgZGVzdF9kZXNjLCBzcmNfbW9kZSkgIT0gMApAQCAtMTUxNiw2ICsxNTk2LDkg QEAgY29weV9yZWcgKGNoYXIgY29uc3QgKnNyY19uYW1lLCBjaGFyIGNvbnN0ICpkc3RfbmFt ZSwKICAgICAgICAgfQogICAgIH0KIAorICBpZiAoZGVzdF9kZXNjIDwgMCkKKyAgICBnb3Rv IGNsb3NlX3NyY19kZXNjOworCiBjbG9zZV9zcmNfYW5kX2RzdF9kZXNjOgogICBpZiAoY2xv c2UgKGRlc3RfZGVzYykgPCAwKQogICAgIHsKLS0gCjIuMzkuMQoK --------------d3sasI5F9S5zzz6EZc8WbvfB--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 15 Feb 2023 09:44:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 15 04:44:01 2023 Received: from localhost ([127.0.0.1]:57788 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pSEKX-0005rb-FG for submit <at> debbugs.gnu.org; Wed, 15 Feb 2023 04:44:01 -0500 Received: from mail-wr1-f53.google.com ([209.85.221.53]:42634) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pSEKV-0005rD-7O for 61386 <at> debbugs.gnu.org; Wed, 15 Feb 2023 04:43:59 -0500 Received: by mail-wr1-f53.google.com with SMTP id o15so15094130wrc.9 for <61386 <at> debbugs.gnu.org>; Wed, 15 Feb 2023 01:43:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=RPmWhFGlsP/Y1K+NqMETQZMN5cT+U9Sn00jdY4ohnFA=; b=BFrTncHK01f2NQR+KLLj9YjXdnEAMGvF3D/mdAqR8VKqpZ737VUWew+m7k8DqVeQ/i ixB/hRa1J8wp2+hHpJAH9JzV9Lwai5YOxLvhAjIOEsAlmaxjxox+KJJAPkNc7eFEmpHF 0wMpw4Xc9yYbtpFNpRmrjcVXZGbqsqv6mMF6l0NQUyttSPXuebY5N50y43DPeVZx8s5m FKoUgu5DOUZy71CwoxUzh7kALl8tC/9YdODuTHqWz8XX+y97Okga0nlW53kGOO5cQn7H e6BOYEN7X3/0gQyCqdspy+fSvHjwLSEW3zobjbJZS52LsIj+bvhglIfKeTxq6fgnahVQ 1g3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RPmWhFGlsP/Y1K+NqMETQZMN5cT+U9Sn00jdY4ohnFA=; b=fj84PqUNaj51fvQ3eSavM4S74VHuCbxq/3vyf1jNjAS2dxTJUGRNJlMr5cdOn9JRZ6 U2p07CAqVBlN6lajrxX3PjCLo6p+aHwBueUzuCdNWwubtBheUy1ICJSMUBDLMafLeDlr pAukYDF5cdNZj/axrNffHjgLXigLSNHtJOdryjZLslhyL6Fxmij1pYyDwuxa77RDWtUK K/zD6KP6xVVUNZYfknITTfI/0McUV87a6xelo4YO8WtT84oB5jJenLuZDzBS5YsGbmfo PnouKH0hqi1T05N26YO9A8yM9LVEdk713dwI+pTFUssrkJzvtADBmc7yQjU7lWcEp1c2 7uXg== X-Gm-Message-State: AO0yUKXg754oThBtCjLBYYC6pbHTuE68cAbbK1zkXowxkwE9UrJzL+pz 5YNH8pLjIM/CgewfXNATumM= X-Google-Smtp-Source: AK7set8QAyUK8wzjdjyS0SjIEIk8AcDgkdf3XLWXQ1Qa0FCdzZNmTetZepzKdBgvJ03UAdE4UmISXQ== X-Received: by 2002:adf:eece:0:b0:2c5:561e:80a0 with SMTP id a14-20020adfeece000000b002c5561e80a0mr882728wrp.45.1676454233217; Wed, 15 Feb 2023 01:43:53 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id y1-20020a5d6201000000b002c3ea5ebc73sm14956015wru.101.2023.02.15.01.43.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 15 Feb 2023 01:43:52 -0800 (PST) Message-ID: <354b1398-3c1c-9f83-1209-837825b9a85f@HIDDEN> Date: Wed, 15 Feb 2023 09:43:51 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> <45adda7f-4654-e186-7052-c14612415c22@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <45adda7f-4654-e186-7052-c14612415c22@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 14/02/2023 19:02, Paul Eggert wrote: > On 2023-02-14 04:12, Pádraig Brady wrote: > >> I suspect it's a similar issue to the one that openzfs had: >> https://github.com/openzfs/zfs/issues/11900 > > Yes, quite likely. > > >> Given how closed / uncommunicative Apple are in general >> and specifically for this already reported bug, >> I'm inclined to disable SEEK_DATA on __APPLE__ for the upcoming release. > > If there's an imminent release that's the only way to go. However, if we > do that we should also disable fclonefileat since the latest edition of > the patch isn't right either. It messes up the modes in some cases and > there's a good reason we don't otherwise allow writing through dangling > symlinks (see the length email discussions when that was put in) so that > would have to be addressed too. > > I'll see if I can come up with an improved version of the fclonefileat > patch that would handle these issues. > > >> I've attached the 3 latest patches I'm considering in this area. >> I presume you're OK with your amended fclonefileat() improvement one? > > Not yet, because it doesn't work correctly in some cases (e.g., > x->explicit_no_preserve_mode) and it mishandles the dangling symlink cases. > > The dangling symlink patch might be OK; depends on how fclonefileat > patch ends up. It's OK to copy through a dangling symlink with > fclonefileat only if we don't need to fix up permissions or timestamps > afterwards (e.g., cp -p). > > The disabling SEEK_HOLE on Apple is OK if a release is imminent. > Otherwise I'd like to get to the bottom of it first. This can be done by > having George use dtrace or (if that's not possible) adding debugging > printfs. Cool. The release is 1 - 2 weeks away anyway. thanks for looking at this Paul. Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 14 Feb 2023 19:02:26 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 14 14:02:26 2023 Received: from localhost ([127.0.0.1]:56984 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pS0ZN-0001uv-T3 for submit <at> debbugs.gnu.org; Tue, 14 Feb 2023 14:02:26 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:45762) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pS0ZM-0001ug-3V for 61386 <at> debbugs.gnu.org; Tue, 14 Feb 2023 14:02:25 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 20D7A160054; Tue, 14 Feb 2023 11:02:17 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id LD8PATw1RAmq; Tue, 14 Feb 2023 11:02:16 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 3625F16007F; Tue, 14 Feb 2023 11:02:16 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 3625F16007F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676401336; bh=izi883pH/KXNkLp4aQ4qcnha5wwD38G5xLdec+On3WA=; h=Message-ID:Date:MIME-Version:To:From:Subject:Content-Type: Content-Transfer-Encoding; b=KTeF25pV+rwPJree8EbyvKHnaNCLuqGmKoI2WHtNDkTvb3hjIAO63onGvfDxvpdBC xZhaYyoP+G/meIhXzHI9K4+ac41q2h6yEKlajKiiSUjBq2n49xOMVAPYnM9m49s1wa DfzYd1zCZmBwELpvuAD32OjmXK/TXIeXMxZV68xY= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id iuYwdBoLMXNB; Tue, 14 Feb 2023 11:02:16 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 0E0D7160054; Tue, 14 Feb 2023 11:02:16 -0800 (PST) Message-ID: <45adda7f-4654-e186-7052-c14612415c22@HIDDEN> Date: Tue, 14 Feb 2023 11:02:15 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Content-Language: en-US To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS In-Reply-To: <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) On 2023-02-14 04:12, P=C3=A1draig Brady wrote: > I suspect it's a similar issue to the one that openzfs had: > https://github.com/openzfs/zfs/issues/11900 Yes, quite likely. > Given how closed / uncommunicative Apple are in general > and specifically for this already reported bug, > I'm inclined to disable SEEK_DATA on __APPLE__ for the upcoming release= . If there's an imminent release that's the only way to go. However, if we=20 do that we should also disable fclonefileat since the latest edition of=20 the patch isn't right either. It messes up the modes in some cases and=20 there's a good reason we don't otherwise allow writing through dangling=20 symlinks (see the length email discussions when that was put in) so that=20 would have to be addressed too. I'll see if I can come up with an improved version of the fclonefileat=20 patch that would handle these issues. > I've attached the 3 latest patches I'm considering in this area. > I presume you're OK with your amended fclonefileat() improvement one? Not yet, because it doesn't work correctly in some cases (e.g.,=20 x->explicit_no_preserve_mode) and it mishandles the dangling symlink case= s. The dangling symlink patch might be OK; depends on how fclonefileat=20 patch ends up. It's OK to copy through a dangling symlink with=20 fclonefileat only if we don't need to fix up permissions or timestamps=20 afterwards (e.g., cp -p). The disabling SEEK_HOLE on Apple is OK if a release is imminent.=20 Otherwise I'd like to get to the bottom of it first. This can be done by=20 having George use dtrace or (if that's not possible) adding debugging=20 printfs.
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 14 Feb 2023 12:12:57 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 14 07:12:57 2023 Received: from localhost ([127.0.0.1]:52776 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pRuB6-0004WI-8B for submit <at> debbugs.gnu.org; Tue, 14 Feb 2023 07:12:57 -0500 Received: from mail-wm1-f54.google.com ([209.85.128.54]:38793) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pRuB3-0004W2-HD for 61386 <at> debbugs.gnu.org; Tue, 14 Feb 2023 07:12:54 -0500 Received: by mail-wm1-f54.google.com with SMTP id f47-20020a05600c492f00b003dc584a7b7eso13498236wmp.3 for <61386 <at> debbugs.gnu.org>; Tue, 14 Feb 2023 04:12:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:sender:from:to:cc:subject :date:message-id:reply-to; bh=F786mdGnoYkmPwKI9UmdWdpr9YBNOkeMIYa+jiV97nE=; b=mqZ+KJwQqPERGkRyygU2NWoMerkGUsq9EKnhRblPkljDQFQW94O56a1opz5Olt2CKO VZfkWUcGop7z/MX0jvz6O4XgDepJoffnDMVl4iinqMjYwfraRdrp+SqJb/cJDFU/yAcI FJNY6kjeqDAVFpICIPfRVVoSCP1Y4/pNVT29SMqxn9YuIvmo09p/pMmQdtD7L7EZM1Mj uJUIp3gHlelrP7ZPdCL5/7eELA1doXZMccukh9u6pGXx/JXTLV5sTqGMVpivRGFVbfSb TY7qC6Knudhx5UgPrjLV06oVwuWc57Xqnoggs/EjcOO/5VyQ2rkfdJm3wSLzS3QRHNA5 dvPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=F786mdGnoYkmPwKI9UmdWdpr9YBNOkeMIYa+jiV97nE=; b=m52vnI6e1AaXHbVN+9ndbrY3Pbye/DDVECeGpTyfOWl6vANZIdCRNGscB/5Fxt/dc6 Ijrb/bUKptBipWLOoiclM6FiOXratA72vwwgMyKAhdqEN1mzb6sdfnn295z3HH7zuwrB R7bPqEZmhOlz2RWfI1C+BpkZtclzL30kO3+6Lt+gcUtORQmZR0LnJ9+uytbHu2QqkwZJ PpXjioNGF0Jhqm1VxsJpHmEmwGdzu1UHO/cYg+jQuJ8Guzmpw3PwwHyHROes9JAh9655 whFvoFBI0KLdEXJAYpCHhDgD7/XQqRp/zCK9vbWr4MN+KSKfGbor0F4R8iuf7PnUuM3a AZMA== X-Gm-Message-State: AO0yUKUK3kBpcRI8WAW7W6G8wqIGK7P9c9bTWNljnFAG3K3gsR5tVgEf F7MSqfHrwblWXo81IBHIk5I= X-Google-Smtp-Source: AK7set9ATeW8b8hxq0ErBLDeg7lZ421HvhudOC7O0eTu9C9I9IoVqCTPvsZirVylK9M5IfJzSnk3dg== X-Received: by 2002:a05:600c:3085:b0:3d9:e5f9:984c with SMTP id g5-20020a05600c308500b003d9e5f9984cmr1745330wmn.2.1676376767403; Tue, 14 Feb 2023 04:12:47 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id q19-20020a05600c46d300b003dc434b39c7sm2859012wmo.0.2023.02.14.04.12.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Feb 2023 04:12:46 -0800 (PST) Content-Type: multipart/mixed; boundary="------------NbBDFh0nhDWzIDEcWLdDM2kl" Message-ID: <1f4bb896-45cc-9372-5cee-3882bb03529e@HIDDEN> Date: Tue, 14 Feb 2023 12:12:45 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) This is a multi-part message in MIME format. --------------NbBDFh0nhDWzIDEcWLdDM2kl Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 14/02/2023 06:12, Paul Eggert wrote: > On 2023-02-13 09:16, George Valkov wrote: >> 1. We apply my original patch to disable sparse-copy on macOS. Otherwise since fclonefileat is not used whenever we overwrite a file, corruption will still occur. > > I'm not entirely sold on this patch, because I still don't understand > what's happening. The original bug report in > <https://github.com/openwrt/openwrt/pull/11960> basically says only "it > doesn't work", and I'd like to know more. > > Part of the reason I'm hesitating is that there are multiple ways of > interpreting what SEEK_HOLE and SEEK_DATA mean, and perhaps it's just > that Apple has come up with yet another way to interpret it, which we > need to know about. I agree it would be good to know more. However given it works on HFS but fails on APFS suggests a file system specific issue, rather than an API mismatch issue (over what we're already catering for on macOS). I suspect it's a similar issue to the one that openzfs had: https://github.com/openzfs/zfs/issues/11900 Given how closed / uncommunicative Apple are in general and specifically for this already reported bug, I'm inclined to disable SEEK_DATA on __APPLE__ for the upcoming release. Also we've mitigated the impact somewhat by enabling fclonefileat() in more cases. > Another reason to hesitate is that GNU coreutils is not the only core > program that uses SEEK_HOLE and SEEK_DATA. If this really is a generic > Apple problem we need to know which Apple releases have it, so that we > can program around it at the Gnulib level instead of mucking about with > each individual program. Yes that would be best if possible. I've attached the 3 latest patches I'm considering in this area. I presume you're OK with your amended fclonefileat() improvement one? thanks, Pádraig --------------NbBDFh0nhDWzIDEcWLdDM2kl Content-Type: text/x-patch; charset=UTF-8; name="0001-copy-improve-use-of-fclonefileat.patch" Content-Disposition: attachment; filename="0001-copy-improve-use-of-fclonefileat.patch" Content-Transfer-Encoding: base64 RnJvbSAwYTU5MDg1M2IxNTlkNGVkYTkwYjFkM2E4M2ZjMTg1OTM1NjcyN2FjIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBGcmksIDEwIEZlYiAyMDIzIDEzOjM0OjU0IC0wODAwClN1YmplY3Q6IFtQQVRD SCAxLzNdIGNvcHk6IGltcHJvdmUgdXNlIG9mIGZjbG9uZWZpbGVhdAoKKiBzcmMvY29weS5j IChjb3B5X3JlZyk6IFVzZSBDTE9ORV9BQ0wgaWYgYXZhaWxhYmxlIGFuZCB3b3JraW5nLgpJ ZiB0aGUgb25seSBwcm9ibGVtIHdpdGggZmNsb25lZmlsZWF0IGlzIHRoYXQgaXQgd291bGQg Y3JlYXRlIHRoZQpmaWxlIHdpdGggdGhlIHdyb25nIHRpbWVzdGFtcCwgb3Igd2l0aCB0b28g ZmV3IHBlcm1pc3Npb25zLApkbyB0aGF0IGJ1dCBmaXggdGhlIHRpbWVzdGFtcCBhbmQgcGVy bWlzc2lvbnMgYWZ0ZXJ3YXJkcywKcmF0aGVyIHRoYW4gZmFsbGluZyBiYWNrIG9uIGEgdHJh ZGl0aW9uYWwgY29weS4KKENMT05FX0FDTCkgW0hBVkVfRkNMT05FRklMRUFUICYmICFVU0Vf WEFUVFJdOiBEZWZhdWx0IHRvIDAuCi0tLQogTkVXUyAgICAgICB8ICAzICsrKwogc3JjL2Nv cHkuYyB8IDUzICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr Ky0tLS0tLS0tCiAyIGZpbGVzIGNoYW5nZWQsIDQ4IGluc2VydGlvbnMoKyksIDggZGVsZXRp b25zKC0pCgpkaWZmIC0tZ2l0IGEvTkVXUyBiL05FV1MKaW5kZXggM2QwZWRlMTUwLi5jYmM2 MTRjMmUgMTAwNjQ0Ci0tLSBhL05FV1MKKysrIGIvTkVXUwpAQCAtMTI0LDYgKzEyNCw5IEBA IEdOVSBjb3JldXRpbHMgTkVXUyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IC0qLSBvdXRsaW5lIC0qLQogICBhbmQgcG9zc2libHkgZW1wbG95aW5nIGNvcHkgb2ZmbG9h ZGluZyBvciByZWZsaW5raW5nLAogICBmb3IgdGhlIG5vbiBzcGFyc2UgcG9ydGlvbiBvZiBz dWNoIHNwYXJzZSBmaWxlcy4KIAorICBPbiBtYWNPUywgY3AgY3JlYXRlcyBhIGNvcHktb24t d3JpdGUgY2xvbmUgaW4gbW9yZSBjYXNlcy4KKyAgUHJldmlvdXNseSBjcCB3b3VsZCBvbmx5 IGRvIHRoaXMgd2hlbiBwcmVzZXJ2aW5nIG1vZGUgYW5kIHRpbWVzdGFtcHMuCisKICAgZGF0 ZSAtLWRlYnVnIG5vdyBkaWFnbm9zZXMgaWYgbXVsdGlwbGUgLS1kYXRlIG9yIC0tc2V0IG9w dGlvbnMgYXJlCiAgIHNwZWNpZmllZCwgYXMgb25seSB0aGUgbGFzdCBzcGVjaWZpZWQgaXMg c2lnbmlmaWNhbnQgaW4gdGhhdCBjYXNlLgogCmRpZmYgLS1naXQgYS9zcmMvY29weS5jIGIv c3JjL2NvcHkuYwppbmRleCBkZmJiNTU3ZGUuLjgzNzBmNTViZCAxMDA2NDQKLS0tIGEvc3Jj L2NvcHkuYworKysgYi9zcmMvY29weS5jCkBAIC0xMTQ2LDYgKzExNDYsNyBAQCBjb3B5X3Jl ZyAoY2hhciBjb25zdCAqc3JjX25hbWUsIGNoYXIgY29uc3QgKmRzdF9uYW1lLAogICB1bmlv biBzY2FuX2luZmVyZW5jZSBzY2FuX2luZmVyZW5jZTsKICAgYm9vbCByZXR1cm5fdmFsID0g dHJ1ZTsKICAgYm9vbCBkYXRhX2NvcHlfcmVxdWlyZWQgPSB4LT5kYXRhX2NvcHlfcmVxdWly ZWQ7CisgIGJvb2wgbW9kZV9hbHJlYWR5X3ByZXNlcnZlZCA9IGZhbHNlOwogICBib29sIHBy ZXNlcnZlX3hhdHRyID0gVVNFX1hBVFRSICYgeC0+cHJlc2VydmVfeGF0dHI7CiAKICAgc291 cmNlX2Rlc2MgPSBvcGVuIChzcmNfbmFtZSwKQEAgLTEyNDQsMTcgKzEyNDUsNDggQEAgY29w eV9yZWcgKGNoYXIgY29uc3QgKnNyY19uYW1lLCBjaGFyIGNvbnN0ICpkc3RfbmFtZSwKICAg aWYgKCpuZXdfZHN0KQogICAgIHsKICNpZiBIQVZFX0ZDTE9ORUZJTEVBVCAmJiAhVVNFX1hB VFRSCi0vKiBDTE9ORV9OT09XTkVSQ09QWSBvbmx5IGF2YWlsYWJsZSBvbiBtYWNvcyA+PSAx MC4xMy4gICovCisjIGlmbmRlZiBDTE9ORV9BQ0wKKyMgIGRlZmluZSBDTE9ORV9BQ0wgMCAv KiBBZGRlZCBpbiBtYWNPUyAxMi42LiAgKi8KKyMgZW5kaWYKICMgaWZuZGVmIENMT05FX05P T1dORVJDT1BZCi0jICBkZWZpbmUgQ0xPTkVfTk9PV05FUkNPUFkgMAorIyAgZGVmaW5lIENM T05FX05PT1dORVJDT1BZIDAgLyogQWRkZWQgaW4gbWFjT1MgMTAuMTMuICAqLwogIyBlbmRp ZgotICAgICAgaW50IGZjX2ZsYWdzID0geC0+cHJlc2VydmVfb3duZXJzaGlwID8gMCA6IENM T05FX05PT1dORVJDT1BZOworICAgICAgbW9kZV90IGNsb25lZF9tb2RlX2JpdHMgPSBTX0lT VlRYIHwgU19JUldYVUdPOworICAgICAgbW9kZV90IGNsb25lZF9tb2RlID0gc3JjX21vZGUg JiBjbG9uZWRfbW9kZV9iaXRzOworICAgICAgaW50IGZjX2ZsYWdzID0gKCh4LT5wcmVzZXJ2 ZV9tb2RlID8gQ0xPTkVfQUNMIDogMCkKKyAgICAgICAgICAgICAgICAgICAgICB8ICh4LT5w cmVzZXJ2ZV9vd25lcnNoaXAgPyAwIDogQ0xPTkVfTk9PV05FUkNPUFkpKTsKICAgICAgIGlm IChkYXRhX2NvcHlfcmVxdWlyZWQgJiYgeC0+cmVmbGlua19tb2RlCi0gICAgICAgICAgJiYg eC0+cHJlc2VydmVfbW9kZSAmJiB4LT5wcmVzZXJ2ZV90aW1lc3RhbXBzCisgICAgICAgICAg JiYgKHgtPnByZXNlcnZlX21vZGUgfHwgISAoY2xvbmVkX21vZGUgJiB+ZHN0X21vZGUpKQog ICAgICAgICAgICYmICh4LT5wcmVzZXJ2ZV9vd25lcnNoaXAgfHwgQ0xPTkVfTk9PV05FUkNP UFkpKQogICAgICAgICB7Ci0gICAgICAgICAgaWYgKGZjbG9uZWZpbGVhdCAoc291cmNlX2Rl c2MsIGRzdF9kaXJmZCwgZHN0X3JlbG5hbWUsIGZjX2ZsYWdzKSA9PSAwKQotICAgICAgICAg ICAgZ290byBjbG9zZV9zcmNfZGVzYzsKKyAgICAgICAgICBpbnQgcyA9IGZjbG9uZWZpbGVh dCAoc291cmNlX2Rlc2MsIGRzdF9kaXJmZCwgZHN0X3JlbG5hbWUsIGZjX2ZsYWdzKTsKKyAg ICAgICAgICBpZiAocyAhPSAwICYmIChmY19mbGFncyAmIENMT05FX0FDTCkgIT0gMCAmJiBl cnJubyA9PSBFSU5WQUwpCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgIGZjX2ZsYWdz ICY9IH5DTE9ORV9BQ0w7CisgICAgICAgICAgICAgIHMgPSBmY2xvbmVmaWxlYXQgKHNvdXJj ZV9kZXNjLCBkc3RfZGlyZmQsIGRzdF9yZWxuYW1lLCBmY19mbGFncyk7CisgICAgICAgICAg ICB9CisgICAgICAgICAgaWYgKHMgPT0gMCkKKyAgICAgICAgICAgIHsKKyAgICAgICAgICAg ICAgaWYgKCF4LT5wcmVzZXJ2ZV90aW1lc3RhbXBzKQorICAgICAgICAgICAgICAgIHsKKyAg ICAgICAgICAgICAgICAgIHN0cnVjdCB0aW1lc3BlYyB0aW1lc3BlY1syXTsKKyAgICAgICAg ICAgICAgICAgIHRpbWVzcGVjWzBdLnR2X25zZWMgPSB0aW1lc3BlY1sxXS50dl9uc2VjID0g VVRJTUVfTk9XOworICAgICAgICAgICAgICAgICAgaWYgKHV0aW1lbnNhdCAoZHN0X2RpcmZk LCBkc3RfcmVsbmFtZSwgdGltZXNwZWMsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICBBVF9TWU1MSU5LX05PRk9MTE9XKQorICAgICAgICAgICAgICAgICAgICAgICE9IDAp CisgICAgICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICAgICAgICBlcnJvciAo MCwgZXJybm8sIF8oInVwZGF0aW5nIHRpbWVzIGZvciAlcyIpLAorICAgICAgICAgICAgICAg ICAgICAgICAgICAgICBxdW90ZWFmIChkc3RfbmFtZSkpOworICAgICAgICAgICAgICAgICAg ICAgIHJldHVybl92YWwgPSBmYWxzZTsKKyAgICAgICAgICAgICAgICAgICAgfQorICAgICAg ICAgICAgICAgIH0KKyAgICAgICAgICAgICAgbW9kZV9hbHJlYWR5X3ByZXNlcnZlZCA9IChm Y19mbGFncyAmIENMT05FX0FDTCkgIT0gMAorICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgJiYgY2xvbmVkX21vZGUgPT0gc3JjX21vZGU7CisgICAgICAgICAgICAg IGRlc3RfZGVzYyA9IC0xOworICAgICAgICAgICAgICBvbWl0dGVkX3Blcm1pc3Npb25zID0g ZHN0X21vZGUgJiB+Y2xvbmVkX21vZGU7CisgICAgICAgICAgICAgIGV4dHJhX3Blcm1pc3Np b25zID0gZHN0X21vZGUgJiB+IGNhY2hlZF91bWFzayAoKTsKKyAgICAgICAgICAgICAgZ290 byBzZXRfZGVzdF9tb2RlOworICAgICAgICAgICAgfQogICAgICAgICAgIGVsc2UgaWYgKCEg aGFuZGxlX2Nsb25lX2ZhaWwgKGRzdF9kaXJmZCwgZHN0X3JlbG5hbWUsIHNyY19uYW1lLAog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRzdF9uYW1lLAogICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC0xLCBmYWxzZSAvKiBXZSBk aWRuJ3QgY3JlYXRlIGRzdCAgKi8sCkBAIC0xNDg1LDkgKzE1MTcsMTQgQEAgY29weV9yZWcg KGNoYXIgY29uc3QgKnNyY19uYW1lLCBjaGFyIGNvbnN0ICpkc3RfbmFtZSwKIAogICBzZXRf YXV0aG9yIChkc3RfbmFtZSwgZGVzdF9kZXNjLCBzcmNfc2IpOwogCisjaWYgSEFWRV9GQ0xP TkVGSUxFQVQgJiYgIVVTRV9YQVRUUgorc2V0X2Rlc3RfbW9kZToKKyNlbmRpZgogICBpZiAo eC0+cHJlc2VydmVfbW9kZSB8fCB4LT5tb3ZlX21vZGUpCiAgICAgewotICAgICAgaWYgKGNv cHlfYWNsIChzcmNfbmFtZSwgc291cmNlX2Rlc2MsIGRzdF9uYW1lLCBkZXN0X2Rlc2MsIHNy Y19tb2RlKSAhPSAwCisgICAgICBpZiAoIW1vZGVfYWxyZWFkeV9wcmVzZXJ2ZWQKKyAgICAg ICAgICAmJiAoY29weV9hY2wgKHNyY19uYW1lLCBzb3VyY2VfZGVzYywgZHN0X25hbWUsIGRl c3RfZGVzYywgc3JjX21vZGUpCisgICAgICAgICAgICAgICE9IDApCiAgICAgICAgICAgJiYg eC0+cmVxdWlyZV9wcmVzZXJ2ZSkKICAgICAgICAgcmV0dXJuX3ZhbCA9IGZhbHNlOwogICAg IH0KQEAgLTE1MTcsNyArMTU1NCw3IEBAIGNvcHlfcmVnIChjaGFyIGNvbnN0ICpzcmNfbmFt ZSwgY2hhciBjb25zdCAqZHN0X25hbWUsCiAgICAgfQogCiBjbG9zZV9zcmNfYW5kX2RzdF9k ZXNjOgotICBpZiAoY2xvc2UgKGRlc3RfZGVzYykgPCAwKQorICBpZiAoMCA8PSBkZXN0X2Rl c2MgJiYgY2xvc2UgKGRlc3RfZGVzYykgPCAwKQogICAgIHsKICAgICAgIGVycm9yICgwLCBl cnJubywgXygiZmFpbGVkIHRvIGNsb3NlICVzIiksIHF1b3RlYWYgKGRzdF9uYW1lKSk7CiAg ICAgICByZXR1cm5fdmFsID0gZmFsc2U7Ci0tIAoyLjI2LjIKCg== --------------NbBDFh0nhDWzIDEcWLdDM2kl Content-Type: text/x-patch; charset=UTF-8; name="0002-tests-allow-copying-through-dangling-symlinks-on-mac.patch" Content-Disposition: attachment; filename*0="0002-tests-allow-copying-through-dangling-symlinks-on-mac.pa"; filename*1="tch" Content-Transfer-Encoding: base64 RnJvbSAwYWUyZjhmZTUxNGQxMGQxMmU4ZjY0MGM1M2FiZWIxMTQzZTFkMGMxIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/UD1DMz1BMWRyYWlnPTIwQnJhZHk/ PSA8UEBkcmFpZ0JyYWR5LmNvbT4KRGF0ZTogTW9uLCAxMyBGZWIgMjAyMyAxOTowMjoxMiAr MDAwMApTdWJqZWN0OiBbUEFUQ0ggMi8zXSB0ZXN0czogYWxsb3cgY29weWluZyB0aHJvdWdo IGRhbmdsaW5nIHN5bWxpbmtzIG9uIG1hY09TCgoqIHRlc3RzL2NwL3RocnUtZGFuZ2xpbmcu c2g6IGZjbG9uZWZpbGVhdCgpIGNhbiBjb3B5CnRocm91Z2ggZGFuZ2xpbmcgc3ltbGlua3Mg cm9idXN0bHksIHNvIGRvbid0IGVuZm9yY2UKYSBmYWlsdXJlIGluIHRoaXMgY2FzZSBvbiBt YWNPUy4KLS0tCiB0ZXN0cy9jcC90aHJ1LWRhbmdsaW5nLnNoIHwgMjAgKysrKysrKysrKysr Ky0tLS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCAxMyBpbnNlcnRpb25zKCspLCA3IGRlbGV0aW9u cygtKQoKZGlmZiAtLWdpdCBhL3Rlc3RzL2NwL3RocnUtZGFuZ2xpbmcuc2ggYi90ZXN0cy9j cC90aHJ1LWRhbmdsaW5nLnNoCmluZGV4IDFjYmIxMmQxMS4uNjBkNDE3ODg2IDEwMDc1NQot LS0gYS90ZXN0cy9jcC90aHJ1LWRhbmdsaW5nLnNoCisrKyBiL3Rlc3RzL2NwL3RocnUtZGFu Z2xpbmcuc2gKQEAgLTI2LDEzICsyNiwxOSBAQCBlY2hvICJjcDogbm90IHdyaXRpbmcgdGhy b3VnaCBkYW5nbGluZyBzeW1saW5rICdkYW5nbGUnIiBcCiAgICAgPiBleHAtZXJyIHx8IGZy YW1ld29ya19mYWlsdXJlXwogCiAKLSMgU3RhcnRpbmcgd2l0aCA2LjkuOTAsIHRoaXMgdXNh Z2UgZmFpbHMsIGJ5IGRlZmF1bHQ6Ci1mb3Igb3B0IGluICcnICctZic7IGRvCi0gIHJldHVy bnNfIDEgY3AgJG9wdCBmIGRhbmdsZSA+IGVyciAyPiYxIHx8IGZhaWw9MQotICBjb21wYXJl IGV4cC1lcnIgZXJyIHx8IGZhaWw9MQotICB0ZXN0IC1mIG5vLXN1Y2ggJiYgZmFpbD0xCi1k b25lCi0KKyMgbWFjT1MgbWF5IHVzZSBmY2xvbmVmaWxlYXQoKSB3aGljaCBkb2VzIGNvcHkg dGhyb3VnaCB0aGUgZGFuZ2xpbmcgc3ltbGluaworY2FzZSAkaG9zdF90cmlwbGV0IGluCisg ICpkYXJ3aW4qKSBza2lwX2RhbmdsaW5nPTEgOzsKK2VzYWMKKworaWYgdGVzdCAteiAiJHNr aXBfZGFuZ2xpbmciOyB0aGVuCisgICMgU3RhcnRpbmcgd2l0aCA2LjkuOTAsIHRoaXMgdXNh Z2UgZmFpbHMsIGJ5IGRlZmF1bHQ6CisgIGZvciBvcHQgaW4gJycgJy1mJzsgZG8KKyAgICBy ZXR1cm5zXyAxIGNwICRvcHQgZiBkYW5nbGUgPiBlcnIgMj4mMSB8fCBmYWlsPTEKKyAgICBj b21wYXJlIGV4cC1lcnIgZXJyIHx8IGZhaWw9MQorICAgIHRlc3QgLWYgbm8tc3VjaCAmJiBm YWlsPTEKKyAgZG9uZQorZmkKIAogIyBCdXQgeW91IGNhbiBzZXQgUE9TSVhMWV9DT1JSRUNU IHRvIGdldCB0aGUgaGlzdG9yaWNhbCBiZWhhdmlvci4KIGVudiBQT1NJWExZX0NPUlJFQ1Q9 MSBjcCBmIGRhbmdsZSA+IG91dCAyPiYxIHx8IGZhaWw9MQotLSAKMi4yNi4yCgo= --------------NbBDFh0nhDWzIDEcWLdDM2kl Content-Type: text/x-patch; charset=UTF-8; name="0003-copy-disable-sparse-copy-on-macOS.patch" Content-Disposition: attachment; filename="0003-copy-disable-sparse-copy-on-macOS.patch" Content-Transfer-Encoding: base64 RnJvbSBhMzgyNmQ3NjE5OTA3MWMyODcwODYxMDdhYmViYTEyZjkzMzVmZGEwIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBHZW9yZ2UgVmFsa292IDxndmFsa292QGdtYWlsLmNv bT4KRGF0ZTogTW9uLCAxMyBGZWIgMjAyMyAxODo1MDo0OSArMDAwMApTdWJqZWN0OiBbUEFU Q0ggMy8zXSBjb3B5OiBkaXNhYmxlIHNwYXJzZSBjb3B5IG9uIG1hY09TCgpEdWUgdG8gYSBi dWcgaW4gbWFjT1MsIHNwYXJzZSBjb3BpZXMgYXJlIGNvcnJ1cHRlZCBvbiB2aXJ0dWFsIGRp c2tzCmZvcm1hdHRlZCB3aXRoIEFQRlMuIEhGUyBpcyBub3QgYWZmZWN0ZWQuIEFmZmVjdGVk IGFyZSBjb3JldXRpbHMKaW5zdGFsbCwgYW5kIGdjcCB3aGVuIGNvbXBpbGVkIHdpdGggU0VF S19IT0xFLCBhcyB3ZWxsIGFzIG1hY09TIEZpbmRlci4KCldoaWxlIHJlYWRpbmcgdGhlIGVu dGlyZSBmaWxlIHJldHVybnMgdmFsaWQgZGF0YSwgc2Nhbm5pbmcgZm9yCmFsbG9jYXRlZCBz ZWdtZW50cyBtYXkgcmV0dXJuIGhvbGVzIHdoZXJlIHZhbGlkIGRhdGEgaXMgcHJlc2VudC4K SW4gdGhpcyBjYXNlIGEgc3BhcnNlIGNvcHkgZG9lcyBub3QgY29udGFpbiB0aGVzZSBzZWdt ZW50cyBhbmQgcmV0dXJucwp6ZXJvZXMgaW5zdGVhZC4gT25jZSB0aGUgdmlydHVhbCBkaXNr IGlzIGRpc21vdW50ZWQgYW5kIHRoZW4KbW91bnRlZCBhZ2FpbiwgYSBzcGFyc2UgY29weSBw cm9kdWNlcyBjb3JyZWN0IHJlc3VsdHMuCgpUaGlzIGJyZWFrcyBPcGVuV1JUIGJ1aWxkIG9u IG1hY09TLiBEZXRhaWxzOgpodHRwczovL2dpdGh1Yi5jb20vb3BlbndydC9vcGVud3J0L3B1 bGwvMTE5NjAKaHR0cHM6Ly9naXRodWIuY29tL29wZW53cnQvb3BlbndydC9wdWxsLzExOTYw I2lzc3VlY29tbWVudC0xNDIzMTg1NTc5CgoqIE5FV1M6IE1lbnRpb24gdGhlIGJ1ZyBmaXgu Ciogc3JjL2NvcHkuYyAoaW5mZXJfc2NhbnR5cGUpOiBBdm9pZCBTRUVLX0RBVEEgb24gX19B UFBMRV9fLgoqIHRlc3RzL3NlZWstZGF0YS1jYXBhYmxlOiBMaWtld2lzZS4KCkFkZHJlc3Nl cyBodHRwczovL2J1Z3MuZ251Lm9yZy82MTM4NgotLS0KIE5FV1MgICAgICAgICAgICAgICAg ICAgIHwgNCArKysrCiBzcmMvY29weS5jICAgICAgICAgICAgICB8IDMgKystCiB0ZXN0cy9z ZWVrLWRhdGEtY2FwYWJsZSB8IDMgKysrCiAzIGZpbGVzIGNoYW5nZWQsIDkgaW5zZXJ0aW9u cygrKSwgMSBkZWxldGlvbigtKQoKZGlmZiAtLWdpdCBhL05FV1MgYi9ORVdTCmluZGV4IGNi YzYxNGMyZS4uOGY1MGMwYjhlIDEwMDY0NAotLS0gYS9ORVdTCisrKyBiL05FV1MKQEAgLTI5 LDYgKzI5LDEwIEBAIEdOVSBjb3JldXRpbHMgTkVXUyAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIC0qLSBvdXRsaW5lIC0qLQogICBmYWxsaW5nIGJhY2sgZnJvbSBjb3B5 X2ZpbGVfcmFuZ2UgdG8gYSBiZXR0ZXIgc3VwcG9ydGVkIHN0YW5kYXJkIGNvcHkuCiAgIFtp c3N1ZSBpbnRyb2R1Y2VkIGluIGNvcmV1dGlscy05LjBdCiAKKyAgY3AsIG12LCBhbmQgaW5z dGFsbCBub3cgYXZvaWQgdGhlIHVzZSBvZiBsc2VlaytTRUVLX0hPTEUgb24gbWFjT1MsCisg IGFzIGRhdGEgY29ycnVwdGlvbiB3YXMgc2VlbiBpbiBzb21lIGNhc2VzIG9uIEFQRlMgZmls ZSBzeXN0ZW1zLgorICBbaXNzdWUgaW50cm9kdWNlZCBpbiBjb3JldXRpbHMtOS4wXQorCiAg ICdtdiAtLWJhY2t1cD1zaW1wbGUgZiBkLycgbm8gbG9uZ2VyIG1pc3Rha2VubHkgYmFja3Mg dXAgZC9mIHRvIGZ+LgogICBbYnVnIGludHJvZHVjZWQgaW4gY29yZXV0aWxzLTkuMV0KIApk aWZmIC0tZ2l0IGEvc3JjL2NvcHkuYyBiL3NyYy9jb3B5LmMKaW5kZXggODM3MGY1NWJkLi5l OThhMzZiMTMgMTAwNjQ0Ci0tLSBhL3NyYy9jb3B5LmMKKysrIGIvc3JjL2NvcHkuYwpAQCAt MTA2Miw3ICsxMDYyLDggQEAgaW5mZXJfc2NhbnR5cGUgKGludCBmZCwgc3RydWN0IHN0YXQg Y29uc3QgKnNiLAogICAgICAgICAgJiYgU1RfTkJMT0NLUyAoKnNiKSA8IHNiLT5zdF9zaXpl IC8gU1RfTkJMT0NLU0laRSkpCiAgICAgcmV0dXJuIFBMQUlOX1NDQU5UWVBFOwogCi0jaWZk ZWYgU0VFS19IT0xFCisjaWYgZGVmaW5lZCBTRUVLX0hPTEUgJiYgIWRlZmluZWQgX19BUFBM RV9fCisgIC8qIEF2b2lkIG9uIG1hY09TIChBUEZTKSBkdWUgdG8gYnVnICM2MTM4Ni4gICov CiAgIG9mZl90IGV4dF9zdGFydCA9IGxzZWVrIChmZCwgMCwgU0VFS19EQVRBKTsKICAgaWYg KDAgPD0gZXh0X3N0YXJ0IHx8IGVycm5vID09IEVOWElPKQogICAgIHsKZGlmZiAtLWdpdCBh L3Rlc3RzL3NlZWstZGF0YS1jYXBhYmxlIGIvdGVzdHMvc2Vlay1kYXRhLWNhcGFibGUKaW5k ZXggY2M2MzcyMjE0Li4yNzVhNDY0NjQgMTAwNjQ0Ci0tLSBhL3Rlc3RzL3NlZWstZGF0YS1j YXBhYmxlCisrKyBiL3Rlc3RzL3NlZWstZGF0YS1jYXBhYmxlCkBAIC00LDYgKzQsOSBAQCBp bXBvcnQgc3lzLCBvcywgZXJybm8sIHBsYXRmb3JtCiBpZiBsZW4oc3lzLmFyZ3YpICE9IDI6 CiAgICAgc3lzLmV4aXQoMSkKIAoraWYgcGxhdGZvcm0uc3lzdGVtKCkgPT0gIkRhcndpbiI6 CisgICAgc3lzLmV4aXQoMSkgIyBhcyBwZXIgYnVnIDYxMzg2CisKIGlmIG5vdCBoYXNhdHRy KG9zLCAnU0VFS19EQVRBJyk6CiAgICAgIyBOb3QgYXZhaWxhYmxlIG9uIHB5dGhvbiAyLCBv ciBvbiBkYXJ3aW4gcHl0aG9uIDMKICAgICAjIEFsc28gRGFyd2luIHN3YXBzIFNFRUtfREFU QS9TRUVLX0hPTEUgZGVmaW5pdGlvbnMKLS0gCjIuMjYuMgoK --------------NbBDFh0nhDWzIDEcWLdDM2kl--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 14 Feb 2023 07:04:13 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 14 02:04:13 2023 Received: from localhost ([127.0.0.1]:52345 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pRpML-0004Jo-7a for submit <at> debbugs.gnu.org; Tue, 14 Feb 2023 02:04:13 -0500 Received: from mail-wm1-f47.google.com ([209.85.128.47]:37530) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pRpMJ-0004JZ-LM for 61386 <at> debbugs.gnu.org; Tue, 14 Feb 2023 02:04:12 -0500 Received: by mail-wm1-f47.google.com with SMTP id m20-20020a05600c3b1400b003e1e754657aso4507590wms.2 for <61386 <at> debbugs.gnu.org>; Mon, 13 Feb 2023 23:04:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1676358245; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=eSFBxraKydeYTuBCrW9MYLVLOtuZ8wV0QD2+lUrEMq0=; b=E0NVFNzF9lqOlMeLyOiMP/Su4Byn0ACo0FUhGXaCRZGTPcHNZp89HPaW4GJijGsfDM hkwb0UBJxQBjl+9xPG0UpPV4eyLKURNNViswr1xg2Lb1k0GtpF6wr2mvJAsmmIgGxdqL /j+YFCGs0+9SP7DJfNK5Bu+8pXFzF7xR3/F8gUZVyNQ42Y6p0/C2eW3JMWoX7ScG6n2j LqZeSjthZNRb9m3cM3a0E2nq+WuUB3P2WQPYy7nd19CryqlZY7Pa+Y/ORExSfo4AJ3SQ 7jgFZTPpDTqHeADdq9K8mGrBK/P9r1l2TOz0OXtGdzD4eaOgXtyRRhgKmX9R5QHDWchY MuIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1676358245; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eSFBxraKydeYTuBCrW9MYLVLOtuZ8wV0QD2+lUrEMq0=; b=Ys5I0IPLMMZOx6S5ESf+ePSihk0Rahqcwj0XBFQr78xEq2mEoXW8fFk/oEgs/zXTc5 fVnhqccwRZ4OBX5GKPHDCBROJTKEBtDMM/DVMR0+nyV1AH0ExTxCcxs4b1RPSdpSh22G Jh2NG4AnBeKF0vnaV821wrCxdBR1FTKe28OoM7LsunnKVosdfZ8ELvLhaL9G7jvdsQpp kP6KUP4Lb2v8jgF6PW8L9IUt8W4OhZOGD+fFVGakB2Gx6nF48HCgns7Dbb/q0/tQK1ce 4JoXpekMFdVFO92JcAA/xEgh/jqhkxkeP2aACt8t/b0dMW+Mi0E6TbtETDQn8g9IKM9f Msmw== X-Gm-Message-State: AO0yUKVQU1rDr7t5Rr+WTOgo0HKCpuX9DLzMNiqGiKgiFPP8CEcuDaXa cbka6C41r/6Ohwuh764X0jI= X-Google-Smtp-Source: AK7set9pgug1g6uEKQr3B754Uk+57ae1kPGUCMKPi9H9zA62ykSjUANivGTcg8Q6lazb+5QsB/1/bg== X-Received: by 2002:a05:600c:3c9e:b0:3dc:3b1a:5d2d with SMTP id bg30-20020a05600c3c9e00b003dc3b1a5d2dmr1395844wmb.0.1676358245532; Mon, 13 Feb 2023 23:04:05 -0800 (PST) Received: from smtpclient.apple (212-39-89-241.ip.btc-net.bg. [212.39.89.241]) by smtp.gmail.com with ESMTPSA id p24-20020a05600c1d9800b003dd1bd0b915sm19008159wms.22.2023.02.13.23.04.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Feb 2023 23:04:04 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> Date: Tue, 14 Feb 2023 09:04:02 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <6EF12062-DADE-42AE-8D75-0DB15D460650@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> To: Paul Eggert <eggert@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On 2023-02-14, at 8:12 AM, Paul Eggert <eggert@HIDDEN> wrote: >=20 > On 2023-02-13 09:16, George Valkov wrote: >> 1. We apply my original patch to disable sparse-copy on macOS. = Otherwise since fclonefileat is not used whenever we overwrite a file, = corruption will still occur. >=20 > I'm not entirely sold on this patch, because I still don't understand = what's happening. The original bug report in = <https://github.com/openwrt/openwrt/pull/11960> basically says only "it = doesn't work", and I'd like to know more. >=20 > Part of the reason I'm hesitating is that there are multiple ways of = interpreting what SEEK_HOLE and SEEK_DATA mean, and perhaps it's just = that Apple has come up with yet another way to interpret it, which we = need to know about. >=20 > Another reason to hesitate is that GNU coreutils is not the only core = program that uses SEEK_HOLE and SEEK_DATA. If this really is a generic = Apple problem we need to know which Apple releases have it, so that we = can program around it at the Gnulib level instead of mucking about with = each individual program. >=20 > Yet another reason is that perhaps the bug was introduced by this pair = of changes introduced to Gnulib in November 2021 to try to address = <https://bugs.gnu.org/51857>: >=20 > = https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=3D4db8db34112b86dd= f8bac48f16b5acff732b5fa9 > = https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=3D1a268176fbb184e3= 93c98575e61fe692264c7d91 >=20 > These Gnulib patches are attached. If you revert these, does the = problem go away? >=20 > And yet another reason is that perhaps the bug was introduced by this = pair of Coreutils changes made in October 2021 to try to address = <https://bugs.gnu.org/51433>: >=20 > = https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=3Da9e31457bf6a6= 3072b54a9324cdf59a09441a21f > = https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=3D1753012b8d8cd= d0817155f4756f5bf9bfe347816 >=20 > These Coreutils patches are also attached. Does reverting them help? >=20 > All things considered, I'd like a copy of the dtruss output for = unmodified coreutils 9.1 cp on the failing case. This is really helpful = for debugging this sort of thing. It's what helped us debug = Bug#51857.<0001-lseek-port-around-macOS-SEEK_DATA-glitch.patch><0002-lseek= -port-around-macOS-SEEK_HOLE-glitch.patch><0001-cp-defend-better-against-F= reeBSD-9.1-zfs-bug.patch><0002-cp-revert-unnecessary-FreeBSD-workaround.pa= tch> Hi Paul, I can check later today. One thing I notices is that the = sparse-copy corruption has been present on 2021-10-06. (October 6th). As = I recall it was not present on coreutils-8, it was introduced as soon as = coreutils-9 came, precisely ever since sparse-copy was enabled. I will = have to check if I can find any old logs or investigate again. But I = don=E2=80=99t think reverting these patches will make any difference, = since the issue was present before them. Can we bring someone from the gcc team here? They know best how the = build system for gcc works, and might be able to create a simple PoC to = craft a sparse file that gets corrupted on copy. Currently I need to = rebuild gcc, which takes 2-3 minutes and we have no clue how it is = crafted. Any idea how to make Apple aware? Last time I contacted the = product-security@HIDDEN team on 2021-10-06, the replayed two days = later that they are investigating, but did not bother fixing the = underlaying issue. Perhaps if you contact them as a member of = GNU/coreutils, they might be more open to work? If you do, refer them to = Follow-up: 782620861. I=E2=80=99ll come back to test your requests, at a late point today. Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 14 Feb 2023 06:12:58 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 14 01:12:58 2023 Received: from localhost ([127.0.0.1]:52318 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pRoYg-0002ow-SX for submit <at> debbugs.gnu.org; Tue, 14 Feb 2023 01:12:58 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:60968) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pRoYe-0002od-D8 for 61386 <at> debbugs.gnu.org; Tue, 14 Feb 2023 01:12:53 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 4EB32160054; Mon, 13 Feb 2023 22:12:46 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 0CSfFHotyM03; Mon, 13 Feb 2023 22:12:44 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 57D0316005E; Mon, 13 Feb 2023 22:12:44 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 57D0316005E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676355164; bh=nwXo/feuocDqXHe1A8kvYw3c0kUV2ZdX59nTEl2hQMA=; h=Content-Type:Message-ID:Date:MIME-Version:To:From:Subject; b=SIlP60LoVM9qPNdP00e8i8BUOKIkB+lej3WV2IYuFedDd1fOzuV97TUp/yRz6KeJu dQ6VKv3yyD0q6fV9jJqpF55M0FY9t0cMlW5TjQXrJ4WFL8KDKf+p1j+2dX3vNeaY1p 6pjIjwHOK7Y5r4WvsgOQvEY5q7GKVozVMY8frxAE= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id qhQvgnrpReeT; Mon, 13 Feb 2023 22:12:44 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 1DA13160054; Mon, 13 Feb 2023 22:12:44 -0800 (PST) Content-Type: multipart/mixed; boundary="------------U2BcLqFAE7z9KMDFBoqzDx3m" Message-ID: <f2aa59ba-0ef9-df0b-6cdc-ae3413f5502f@HIDDEN> Date: Mon, 13 Feb 2023 22:12:43 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS In-Reply-To: <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) This is a multi-part message in MIME format. --------------U2BcLqFAE7z9KMDFBoqzDx3m Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2023-02-13 09:16, George Valkov wrote: > 1. We apply my original patch to disable sparse-copy on macOS. Otherwise since fclonefileat is not used whenever we overwrite a file, corruption will still occur. I'm not entirely sold on this patch, because I still don't understand what's happening. The original bug report in <https://github.com/openwrt/openwrt/pull/11960> basically says only "it doesn't work", and I'd like to know more. Part of the reason I'm hesitating is that there are multiple ways of interpreting what SEEK_HOLE and SEEK_DATA mean, and perhaps it's just that Apple has come up with yet another way to interpret it, which we need to know about. Another reason to hesitate is that GNU coreutils is not the only core program that uses SEEK_HOLE and SEEK_DATA. If this really is a generic Apple problem we need to know which Apple releases have it, so that we can program around it at the Gnulib level instead of mucking about with each individual program. Yet another reason is that perhaps the bug was introduced by this pair of changes introduced to Gnulib in November 2021 to try to address <https://bugs.gnu.org/51857>: https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=4db8db34112b86ddf8bac48f16b5acff732b5fa9 https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=1a268176fbb184e393c98575e61fe692264c7d91 These Gnulib patches are attached. If you revert these, does the problem go away? And yet another reason is that perhaps the bug was introduced by this pair of Coreutils changes made in October 2021 to try to address <https://bugs.gnu.org/51433>: https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=a9e31457bf6a63072b54a9324cdf59a09441a21f https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=1753012b8d8cdd0817155f4756f5bf9bfe347816 These Coreutils patches are also attached. Does reverting them help? All things considered, I'd like a copy of the dtruss output for unmodified coreutils 9.1 cp on the failing case. This is really helpful for debugging this sort of thing. It's what helped us debug Bug#51857. --------------U2BcLqFAE7z9KMDFBoqzDx3m Content-Type: text/x-patch; charset=UTF-8; name="0001-lseek-port-around-macOS-SEEK_DATA-glitch.patch" Content-Disposition: attachment; filename="0001-lseek-port-around-macOS-SEEK_DATA-glitch.patch" Content-Transfer-Encoding: base64 RnJvbSA0ZGI4ZGIzNDExMmI4NmRkZjhiYWM0OGYxNmI1YWNmZjczMmI1ZmE5IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBNb24sIDE1IE5vdiAyMDIxIDE1OjA4OjI1IC0wODAwClN1YmplY3Q6IFtQQVRD SCAxLzJdIGxzZWVrOiBwb3J0IGFyb3VuZCBtYWNPUyBTRUVLX0RBVEEgZ2xpdGNoCgpQcm9i bGVtIHJlcG9ydGVkIGJ5IFN1ZGhpcCBOYXNoaSAoQnVnIzUxODU3KS4KKiBkb2MvcG9zaXgt ZnVuY3Rpb25zL2xzZWVrLnRleGkgKGxzZWVrKTogTWVudGlvbiBtYWNPUyBTRUVLX0RBVEEK aXNzdWUuCiogbGliL2xzZWVrLmMgKHJwbF9sc2Vlayk6IFdvcmsgYXJvdW5kIG1hY09TIHBv cnRhYmlsaXR5IGdsaXRjaC4KKiBtNC9sc2Vlay5tNCAoZ2xfRlVOQ19MU0VFSyk6IFJlcGxh Y2UgbHNlZWsgb24gRGFyd2luLgoqIG1vZHVsZXMvbHNlZWsgKERlcGVuZHMtb24pOiBEZXBl bmQgb24gbXN2Yy1ub3Rocm93CmFuZCBmc3RhdCBvbmx5IGlmIG5lZWRlZC4KLS0tCiBDaGFu Z2VMb2cgICAgICAgICAgICAgICAgICAgICAgfCAxMSArKysrKysrKysrKwogZG9jL3Bvc2l4 LWZ1bmN0aW9ucy9sc2Vlay50ZXhpIHwgIDQgKysrKwogbGliL2xzZWVrLmMgICAgICAgICAg ICAgICAgICAgIHwgMTYgKysrKysrKysrKysrKysrKwogbTQvbHNlZWsubTQgICAgICAgICAg ICAgICAgICAgIHwgMTAgKysrKysrKystLQogbW9kdWxlcy9sc2VlayAgICAgICAgICAgICAg ICAgIHwgIDQgKystLQogNSBmaWxlcyBjaGFuZ2VkLCA0MSBpbnNlcnRpb25zKCspLCA0IGRl bGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL0NoYW5nZUxvZyBiL0NoYW5nZUxvZwppbmRleCBm NDcwNzFhNzJkLi43MWEyMjY1NzA2IDEwMDY0NAotLS0gYS9DaGFuZ2VMb2cKKysrIGIvQ2hh bmdlTG9nCkBAIC0xLDMgKzEsMTQgQEAKKzIwMjEtMTEtMTUgIFBhdWwgRWdnZXJ0ICA8ZWdn ZXJ0QGNzLnVjbGEuZWR1PgorCisJbHNlZWs6IHBvcnQgYXJvdW5kIG1hY09TIFNFRUtfREFU QSBnbGl0Y2gKKwlQcm9ibGVtIHJlcG9ydGVkIGJ5IFN1ZGhpcCBOYXNoaSAoQnVnIzUxODU3 KS4KKwkqIGRvYy9wb3NpeC1mdW5jdGlvbnMvbHNlZWsudGV4aSAobHNlZWspOiBNZW50aW9u IG1hY09TIFNFRUtfREFUQQorCWlzc3VlLgorCSogbGliL2xzZWVrLmMgKHJwbF9sc2Vlayk6 IFdvcmsgYXJvdW5kIG1hY09TIHBvcnRhYmlsaXR5IGdsaXRjaC4KKwkqIG00L2xzZWVrLm00 IChnbF9GVU5DX0xTRUVLKTogUmVwbGFjZSBsc2VlayBvbiBEYXJ3aW4uCisJKiBtb2R1bGVz L2xzZWVrIChEZXBlbmRzLW9uKTogRGVwZW5kIG9uIG1zdmMtbm90aHJvdworCWFuZCBmc3Rh dCBvbmx5IGlmIG5lZWRlZC4KKwogMjAyMS0xMS0xMSAgRmFicmljZSBGb250YWluZSAgPGZv bnRhaW5lLmZhYnJpY2VAZ21haWwuY29tPiAgKHRpbnkgY2hhbmdlKQogCiAJc2lnc2Vndjog Zml4IGJ1aWxkcyBvbiBtaWNyb2JsYXplZWwsIG9yMWsKZGlmZiAtLWdpdCBhL2RvYy9wb3Np eC1mdW5jdGlvbnMvbHNlZWsudGV4aSBiL2RvYy9wb3NpeC1mdW5jdGlvbnMvbHNlZWsudGV4 aQppbmRleCA0YTlkNTVkY2Y3Li4yZjhlMmI1ODc3IDEwMDY0NAotLS0gYS9kb2MvcG9zaXgt ZnVuY3Rpb25zL2xzZWVrLnRleGkKKysrIGIvZG9jL3Bvc2l4LWZ1bmN0aW9ucy9sc2Vlay50 ZXhpCkBAIC05LDYgKzksMTAgQEAgR251bGliIG1vZHVsZTogbHNlZWsKIFBvcnRhYmlsaXR5 IHByb2JsZW1zIGZpeGVkIGJ5IEdudWxpYjoKIEBpdGVtaXplCiBAaXRlbQorT24gc29tZSBw bGF0Zm9ybXMsIEBjb2Rle2xzZWVrIChmZCwgb2Zmc2V0LCBTRUVLX0RBVEEpfSByZXR1cm5z IGEgdmFsdWUKK2dyZWF0ZXIgdGhhbiBAY29kZXtvZmZzZXR9IGV2ZW4gd2hlbiBAY29kZXtv ZmZzZXR9IGFkZHJlc3NlcyBkYXRhOgorbWFjT1MgMTIKK0BpdGVtCiBUaGlzIGZ1bmN0aW9u IGlzIGRlY2xhcmVkIGluIGEgZGlmZmVyZW50IGhlYWRlciBmaWxlIChuYW1lbHksIEBjb2Rl ezxpby5oPn0pCiBvbiBzb21lIHBsYXRmb3JtczoKIE1TVkMgMTQuCmRpZmYgLS1naXQgYS9s aWIvbHNlZWsuYyBiL2xpYi9sc2Vlay5jCmluZGV4IDAwNDI1NDZhOGUuLjdkY2Q2YzlkYTcg MTAwNjQ0Ci0tLSBhL2xpYi9sc2Vlay5jCisrKyBiL2xpYi9sc2Vlay5jCkBAIC01Miw2ICs1 MiwyMiBAQCBycGxfbHNlZWsgKGludCBmZCwgb2ZmX3Qgb2Zmc2V0LCBpbnQgd2hlbmNlKQog ICAgICAgZXJybm8gPSBFU1BJUEU7CiAgICAgICByZXR1cm4gLTE7CiAgICAgfQorI2VsaWYg ZGVmaW5lZCBfX0FQUExFX18gJiYgZGVmaW5lZCBfX01BQ0hfXyAmJiBkZWZpbmVkIFNFRUtf REFUQQorICBpZiAod2hlbmNlID09IFNFRUtfREFUQSkKKyAgICB7CisgICAgICAvKiBJZiBP RkZTRVQgcG9pbnRzIHRvIGRhdGEsIG1hY09TIGxzZWVrK1NFRUtfREFUQSByZXR1cm5zIHRo ZQorICAgICAgICAgc3RhcnQgUyBvZiB0aGUgZmlyc3QgZGF0YSByZWdpb24gdGhhdCBiZWdp bnMgKmFmdGVyKiBPRkZTRVQsCisgICAgICAgICB3aGVyZSB0aGUgcmVnaW9uIGZyb20gT0ZG U0VUIHRvIFMgY29uc2lzdHMgb2YgcG9zc2libHktZW1wdHkKKyAgICAgICAgIGRhdGEgZm9s bG93ZWQgYnkgYSBwb3NzaWJseS1lbXB0eSBob2xlLiAgVG8gd29yayBhcm91bmQgdGhpcwor ICAgICAgICAgcG9ydGFiaWxpdHkgZ2xpdGNoLCBjaGVjayB3aGV0aGVyIE9GRlNFVCBpcyB3 aXRoaW4gZGF0YSBieQorICAgICAgICAgdXNpbmcgbHNlZWsrU0VFS19IT0xFLCBhbmQgaWYg c28gcmV0dXJuIHRvIE9GRlNFVCBieSB1c2luZworICAgICAgICAgbHNlZWsrU0VFS19TRVQu ICAqLworICAgICAgb2ZmX3QgbmV4dF9ob2xlID0gbHNlZWsgKGZkLCBvZmZzZXQsIFNFRUtf SE9MRSk7CisgICAgICBpZiAobmV4dF9ob2xlIDwgMCkKKyAgICAgICAgcmV0dXJuIG5leHRf aG9sZTsKKyAgICAgIGlmIChuZXh0X2hvbGUgIT0gb2Zmc2V0KQorICAgICAgICB3aGVuY2Ug PSBTRUVLX1NFVDsKKyAgICB9CiAjZWxzZQogICAvKiBCZU9TIGxzZWVrIG1pc3Rha2VubHkg c3VjY2VlZHMgb24gcGlwZXMuLi4gICovCiAgIHN0cnVjdCBzdGF0IHN0YXRidWY7CmRpZmYg LS1naXQgYS9tNC9sc2Vlay5tNCBiL200L2xzZWVrLm00CmluZGV4IDBhZjYzNzgwYWIuLmZh YWIwOWI3MzQgMTAwNjQ0Ci0tLSBhL200L2xzZWVrLm00CisrKyBiL200L2xzZWVrLm00CkBA IC0xLDQgKzEsNCBAQAotIyBsc2Vlay5tNCBzZXJpYWwgMTEKKyMgbHNlZWsubTQgc2VyaWFs IDEyCiBkbmwgQ29weXJpZ2h0IChDKSAyMDA3LCAyMDA5LTIwMjEgRnJlZSBTb2Z0d2FyZSBG b3VuZGF0aW9uLCBJbmMuCiBkbmwgVGhpcyBmaWxlIGlzIGZyZWUgc29mdHdhcmU7IHRoZSBG cmVlIFNvZnR3YXJlIEZvdW5kYXRpb24KIGRubCBnaXZlcyB1bmxpbWl0ZWQgcGVybWlzc2lv biB0byBjb3B5IGFuZC9vciBkaXN0cmlidXRlIGl0LApAQCAtNTksNyArNTksNyBAQCBBQ19E RUZVTihbZ2xfRlVOQ19MU0VFS10sCiAgICAgICAgICA7OwogICAgICBlc2FjCiAgICAgXSkK LSAgaWYgdGVzdCAkZ2xfY3ZfZnVuY19sc2Vla19waXBlID0gbm87IHRoZW4KKyAgaWYgdGVz dCAiJGdsX2N2X2Z1bmNfbHNlZWtfcGlwZSIgPSBubzsgdGhlbgogICAgIFJFUExBQ0VfTFNF RUs9MQogICAgIEFDX0RFRklORShbTFNFRUtfUElQRV9CUk9LRU5dLCBbMV0sCiAgICAgICBb RGVmaW5lIHRvIDEgaWYgbHNlZWsgZG9lcyBub3QgZGV0ZWN0IHBpcGVzLl0pCkBAIC02OSw0 ICs2OSwxMCBAQCBBQ19ERUZVTihbZ2xfRlVOQ19MU0VFS10sCiAgIGlmIHRlc3QgJFdJTkRP V1NfNjRfQklUX09GRl9UID0gMTsgdGhlbgogICAgIFJFUExBQ0VfTFNFRUs9MQogICBmaQor CisgIGRubCBtYWNPUyBTRUVLX0RBVEEgaXMgaW5jb21wYXRpYmxlIHdpdGggb3RoZXIgcGxh dGZvcm1zLgorICBjYXNlICRob3N0X29zIGluCisgICAgZGFyd2luKikKKyAgICAgIFJFUExB Q0VfTFNFRUs9MTs7CisgIGVzYWMKIF0pCmRpZmYgLS1naXQgYS9tb2R1bGVzL2xzZWVrIGIv bW9kdWxlcy9sc2VlawppbmRleCBjZWQ0NDMxMjM1Li5mNjA4MDkzMTlmIDEwMDY0NAotLS0g YS9tb2R1bGVzL2xzZWVrCisrKyBiL21vZHVsZXMvbHNlZWsKQEAgLTksOCArOSw4IEBAIERl cGVuZHMtb246CiB1bmlzdGQKIHN5c190eXBlcwogbGFyZ2VmaWxlCi1tc3ZjLW5vdGhyb3cg ICAgW3Rlc3QgJFJFUExBQ0VfTFNFRUsgPSAxXQotZnN0YXQgICAgICAgICAgIFt0ZXN0ICRS RVBMQUNFX0xTRUVLID0gMV0KK21zdmMtbm90aHJvdyAgICBbdGVzdCAkV0lORE9XU182NF9C SVRfT0ZGX1QgPSAxXQorZnN0YXQgICAgICAgICAgIFt0ZXN0ICIkZ2xfY3ZfZnVuY19sc2Vl a19waXBlIiA9IG5vXQogCiBjb25maWd1cmUuYWM6CiBnbF9GVU5DX0xTRUVLCi0tIAoyLjM3 LjIKCg== --------------U2BcLqFAE7z9KMDFBoqzDx3m Content-Type: text/x-patch; charset=UTF-8; name="0002-lseek-port-around-macOS-SEEK_HOLE-glitch.patch" Content-Disposition: attachment; filename="0002-lseek-port-around-macOS-SEEK_HOLE-glitch.patch" Content-Transfer-Encoding: base64 RnJvbSAxYTI2ODE3NmZiYjE4NGUzOTNjOTg1NzVlNjFmZTY5MjI2NGM3ZDkxIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBNb24sIDE1IE5vdiAyMDIxIDIyOjE3OjQ0IC0wODAwClN1YmplY3Q6IFtQQVRD SCAyLzJdIGxzZWVrOiBwb3J0IGFyb3VuZCBtYWNPUyBTRUVLX0hPTEUgZ2xpdGNoCgpQcm9i bGVtIHJlcG9ydGVkIGJ5IFN1ZGhpcCBOYXNoaSAoQnVnIzUxODU3IzQ3KS4KKiBsaWIvbHNl ZWsuYyAocnBsX2xzZWVrKTogV29yayBhcm91bmQgbWFjT1MgbHNlZWsrU0VFS19IT0xFCnJl dHVybmluZyAtMSB3aXRoIEVOWElPIGlmIHRoZXJlIGFyZSBubyBob2xlcyBiZWZvcmUgRU9G LApjb250cmFyeSB0byB0aGUgbWFjT1MgZG9jdW1lbnRhdGlvbi4KLS0tCiBDaGFuZ2VMb2cg ICB8IDYgKysrKysrCiBsaWIvbHNlZWsuYyB8IDYgKysrKy0tCiAyIGZpbGVzIGNoYW5nZWQs IDEwIGluc2VydGlvbnMoKyksIDIgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvQ2hhbmdl TG9nIGIvQ2hhbmdlTG9nCmluZGV4IDcxYTIyNjU3MDYuLmVmYzNhMzg4N2UgMTAwNjQ0Ci0t LSBhL0NoYW5nZUxvZworKysgYi9DaGFuZ2VMb2cKQEAgLTEsNSArMSwxMSBAQAogMjAyMS0x MS0xNSAgUGF1bCBFZ2dlcnQgIDxlZ2dlcnRAY3MudWNsYS5lZHU+CiAKKwlsc2VlazogcG9y dCBhcm91bmQgbWFjT1MgU0VFS19IT0xFIGdsaXRjaAorCVByb2JsZW0gcmVwb3J0ZWQgYnkg U3VkaGlwIE5hc2hpIChCdWcjNTE4NTcjNDcpLgorCSogbGliL2xzZWVrLmMgKHJwbF9sc2Vl ayk6IFdvcmsgYXJvdW5kIG1hY09TIGxzZWVrK1NFRUtfSE9MRQorCXJldHVybmluZyAtMSB3 aXRoIEVOWElPIGlmIHRoZXJlIGFyZSBubyBob2xlcyBiZWZvcmUgRU9GLAorCWNvbnRyYXJ5 IHRvIHRoZSBtYWNPUyBkb2N1bWVudGF0aW9uLgorCiAJbHNlZWs6IHBvcnQgYXJvdW5kIG1h Y09TIFNFRUtfREFUQSBnbGl0Y2gKIAlQcm9ibGVtIHJlcG9ydGVkIGJ5IFN1ZGhpcCBOYXNo aSAoQnVnIzUxODU3KS4KIAkqIGRvYy9wb3NpeC1mdW5jdGlvbnMvbHNlZWsudGV4aSAobHNl ZWspOiBNZW50aW9uIG1hY09TIFNFRUtfREFUQQpkaWZmIC0tZ2l0IGEvbGliL2xzZWVrLmMg Yi9saWIvbHNlZWsuYwppbmRleCA3ZGNkNmM5ZGE3Li5lOWE5NmFkMjBhIDEwMDY0NAotLS0g YS9saWIvbHNlZWsuYworKysgYi9saWIvbHNlZWsuYwpAQCAtNjEsMTAgKzYxLDEyIEBAIHJw bF9sc2VlayAoaW50IGZkLCBvZmZfdCBvZmZzZXQsIGludCB3aGVuY2UpCiAgICAgICAgICBk YXRhIGZvbGxvd2VkIGJ5IGEgcG9zc2libHktZW1wdHkgaG9sZS4gIFRvIHdvcmsgYXJvdW5k IHRoaXMKICAgICAgICAgIHBvcnRhYmlsaXR5IGdsaXRjaCwgY2hlY2sgd2hldGhlciBPRkZT RVQgaXMgd2l0aGluIGRhdGEgYnkKICAgICAgICAgIHVzaW5nIGxzZWVrK1NFRUtfSE9MRSwg YW5kIGlmIHNvIHJldHVybiB0byBPRkZTRVQgYnkgdXNpbmcKLSAgICAgICAgIGxzZWVrK1NF RUtfU0VULiAgKi8KKyAgICAgICAgIGxzZWVrK1NFRUtfU0VULiAgQWxzbywgY29udHJhcnkg dG8gdGhlIG1hY09TIGRvY3VtZW50YXRpb24sCisgICAgICAgICBsc2VlaytTRUVLX0hPTEUg Y2FuIGZhaWwgd2l0aCBFTlhJTyBpZiB0aGVyZSBhcmUgbm8gaG9sZXMgb24KKyAgICAgICAg IG9yIGFmdGVyIE9GRlNFVC4gIFdoYXQgYSBtZXNzISAgKi8KICAgICAgIG9mZl90IG5leHRf aG9sZSA9IGxzZWVrIChmZCwgb2Zmc2V0LCBTRUVLX0hPTEUpOwogICAgICAgaWYgKG5leHRf aG9sZSA8IDApCi0gICAgICAgIHJldHVybiBuZXh0X2hvbGU7CisgICAgICAgIHJldHVybiBl cnJubyA9PSBFTlhJTyA/IG9mZnNldCA6IG5leHRfaG9sZTsKICAgICAgIGlmIChuZXh0X2hv bGUgIT0gb2Zmc2V0KQogICAgICAgICB3aGVuY2UgPSBTRUVLX1NFVDsKICAgICB9Ci0tIAoy LjM3LjIKCg== --------------U2BcLqFAE7z9KMDFBoqzDx3m Content-Type: text/x-patch; charset=UTF-8; name="0001-cp-defend-better-against-FreeBSD-9.1-zfs-bug.patch" Content-Disposition: attachment; filename="0001-cp-defend-better-against-FreeBSD-9.1-zfs-bug.patch" Content-Transfer-Encoding: base64 RnJvbSBhOWUzMTQ1N2JmNmE2MzA3MmI1NGE5MzI0Y2RmNTlhMDk0NDFhMjFmIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBGcmksIDI5IE9jdCAyMDIxIDE4OjAxOjM0IC0wNzAwClN1YmplY3Q6IFtQQVRD SCAxLzJdIGNwOiBkZWZlbmQgYmV0dGVyIGFnYWluc3QgRnJlZUJTRCA5LjEgemZzIGJ1ZwpN SU1FLVZlcnNpb246IDEuMApDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9VVRG LTgKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogOGJpdAoKUHJvYmxlbSByZXBvcnRlZCBi eSBQw6FkcmFpZyBCcmFkeSAoQnVnIzUxNDMzIzE0KS4KKiBzcmMvY29weS5jIChsc2Vla19j b3B5LCBpbmZlcl9zY2FudHlwZSk6IFJlcG9ydCBhbiBlcnJvciBpZgpsc2VlayB3aXRoIFNF RUtfREFUQSBvciBTRUVLX0hPTEUgcmV0dXJucyBsZXNzIHRoYW4gLTEsCmFzIHRoaXMgaXMg YW4gbHNlZWsgYnVnLgotLS0KIHNyYy9jb3B5LmMgfCAxNSArKysrKystLS0tLS0tLS0KIDEg ZmlsZSBjaGFuZ2VkLCA2IGluc2VydGlvbnMoKyksIDkgZGVsZXRpb25zKC0pCgpkaWZmIC0t Z2l0IGEvc3JjL2NvcHkuYyBiL3NyYy9jb3B5LmMKaW5kZXggY2I5MDE4ZjkzLi4xY2JjOTQ4 MGMgMTAwNjQ0Ci0tLSBhL3NyYy9jb3B5LmMKKysrIGIvc3JjL2NvcHkuYwpAQCAtNTMwLDcg KzUzMCw3IEBAIGxzZWVrX2NvcHkgKGludCBzcmNfZmQsIGludCBkZXN0X2ZkLCBjaGFyICpi dWYsIHNpemVfdCBidWZfc2l6ZSwKICAgICAgIG9mZl90IGV4dF9lbmQgPSBsc2VlayAoc3Jj X2ZkLCBleHRfc3RhcnQsIFNFRUtfSE9MRSk7CiAgICAgICBpZiAoZXh0X2VuZCA8IDApCiAg ICAgICAgIHsKLSAgICAgICAgICBpZiAoZXJybm8gIT0gRU5YSU8pCisgICAgICAgICAgaWYg KCEgKGV4dF9lbmQgPT0gLTEgJiYgZXJybm8gPT0gRU5YSU8pKQogICAgICAgICAgICAgZ290 byBjYW5ub3RfbHNlZWs7CiAgICAgICAgICAgZXh0X2VuZCA9IHNyY190b3RhbF9zaXplOwog ICAgICAgICAgIGlmIChleHRfZW5kIDw9IGV4dF9zdGFydCkKQEAgLTYwNywxMiArNjA3LDgg QEAgbHNlZWtfY29weSAoaW50IHNyY19mZCwgaW50IGRlc3RfZmQsIGNoYXIgKmJ1Ziwgc2l6 ZV90IGJ1Zl9zaXplLAogICAgICAgICB9CiAKICAgICAgIGV4dF9zdGFydCA9IGxzZWVrIChz cmNfZmQsIGRlc3RfcG9zLCBTRUVLX0RBVEEpOwotICAgICAgaWYgKGV4dF9zdGFydCA8IDAp Ci0gICAgICAgIHsKLSAgICAgICAgICBpZiAoZXJybm8gIT0gRU5YSU8pCi0gICAgICAgICAg ICBnb3RvIGNhbm5vdF9sc2VlazsKLSAgICAgICAgICBicmVhazsKLSAgICAgICAgfQorICAg ICAgaWYgKGV4dF9zdGFydCA8IDAgJiYgISAoZXh0X3N0YXJ0ID09IC0xICYmIGVycm5vID09 IEVOWElPKSkKKyAgICAgICAgZ290byBjYW5ub3RfbHNlZWs7CiAgICAgfQogCiAgIC8qIFdo ZW4gdGhlIHNvdXJjZSBmaWxlIGVuZHMgd2l0aCBhIGhvbGUsIHdlIGhhdmUgdG8gZG8gYSBs aXR0bGUgbW9yZSB3b3JrLApAQCAtMTA5NywxMCArMTA5MywxMSBAQCBpbmZlcl9zY2FudHlw ZSAoaW50IGZkLCBzdHJ1Y3Qgc3RhdCBjb25zdCAqc2IsCiAKICNpZmRlZiBTRUVLX0hPTEUK ICAgc2Nhbl9pbmZlcmVuY2UtPmV4dF9zdGFydCA9IGxzZWVrIChmZCwgMCwgU0VFS19EQVRB KTsKLSAgaWYgKDAgPD0gc2Nhbl9pbmZlcmVuY2UtPmV4dF9zdGFydCkKKyAgaWYgKDAgPD0g c2Nhbl9pbmZlcmVuY2UtPmV4dF9zdGFydAorICAgICAgfHwgKHNjYW5faW5mZXJlbmNlLT5l eHRfc3RhcnQgPT0gLTEgJiYgZXJybm8gPT0gRU5YSU8pKQogICAgIHJldHVybiBMU0VFS19T Q0FOVFlQRTsKICAgZWxzZSBpZiAoZXJybm8gIT0gRUlOVkFMICYmICFpc19FTk9UU1VQIChl cnJubykpCi0gICAgcmV0dXJuIGVycm5vID09IEVOWElPID8gTFNFRUtfU0NBTlRZUEUgOiBF UlJPUl9TQ0FOVFlQRTsKKyAgICByZXR1cm4gRVJST1JfU0NBTlRZUEU7CiAjZW5kaWYKIAog ICByZXR1cm4gWkVST19TQ0FOVFlQRTsKLS0gCjIuMzcuMgoK --------------U2BcLqFAE7z9KMDFBoqzDx3m Content-Type: text/x-patch; charset=UTF-8; name="0002-cp-revert-unnecessary-FreeBSD-workaround.patch" Content-Disposition: attachment; filename="0002-cp-revert-unnecessary-FreeBSD-workaround.patch" Content-Transfer-Encoding: base64 RnJvbSAxNzUzMDEyYjhkOGNkZDA4MTcxNTVmNDc1NmY1YmY5YmZlMzQ3ODE2IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBTYXQsIDMwIE9jdCAyMDIxIDEwOjAwOjEwIC0wNzAwClN1YmplY3Q6IFtQQVRD SCAyLzJdIGNwOiByZXZlcnQgdW5uZWNlc3NhcnkgRnJlZUJTRCB3b3JrYXJvdW5kCk1JTUUt VmVyc2lvbjogMS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD1VVEYtOApD b250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0CgpUaGF0IHdhcyBhIGZhbHNlIGFsYXJt IGR1ZSB0byBhIGJ1ZyBpbiBGcmVlQlNEIDkuMSB0cnVzczsKc2VlIFDDoWRyYWlnIEJyYWR5 4oCZcyByZXBvcnQgKEJ1ZyM1MTQzMyMyOSkuCiogc3JjL2NvcHkuYyAobHNlZWtfY29weSwg aW5mZXJfc2NhbnR5cGUpOiBEb27igJl0IGJvdGhlciBjaGVja2luZwp3aGV0aGVyIGxzZWVr IHJldHVybmVkIC0xLiAgVGhpcyBkb2VzbuKAmXQgZW50aXJlbHkgcmV2ZXJ0IHRoZQpwcmV2 aW91cyBjaGFuZ2UsIGFzIGl0IGtlZXBzIHRoZSBjb2RlIHNpbXBsaWZpY2F0aW9uIG9mIHRo ZQpwcmV2aW91cyBjaGFuZ2Ugd2hpbGUgcmV2ZXJ0aW5nIHRoZSBjaGVjayBmb3IgLTEuCi0t LQogc3JjL2NvcHkuYyB8IDcgKysrLS0tLQogMSBmaWxlIGNoYW5nZWQsIDMgaW5zZXJ0aW9u cygrKSwgNCBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9zcmMvY29weS5jIGIvc3JjL2Nv cHkuYwppbmRleCAxY2JjOTQ4MGMuLmE2NTIzZWQ5NyAxMDA2NDQKLS0tIGEvc3JjL2NvcHku YworKysgYi9zcmMvY29weS5jCkBAIC01MzAsNyArNTMwLDcgQEAgbHNlZWtfY29weSAoaW50 IHNyY19mZCwgaW50IGRlc3RfZmQsIGNoYXIgKmJ1Ziwgc2l6ZV90IGJ1Zl9zaXplLAogICAg ICAgb2ZmX3QgZXh0X2VuZCA9IGxzZWVrIChzcmNfZmQsIGV4dF9zdGFydCwgU0VFS19IT0xF KTsKICAgICAgIGlmIChleHRfZW5kIDwgMCkKICAgICAgICAgewotICAgICAgICAgIGlmICgh IChleHRfZW5kID09IC0xICYmIGVycm5vID09IEVOWElPKSkKKyAgICAgICAgICBpZiAoZXJy bm8gIT0gRU5YSU8pCiAgICAgICAgICAgICBnb3RvIGNhbm5vdF9sc2VlazsKICAgICAgICAg ICBleHRfZW5kID0gc3JjX3RvdGFsX3NpemU7CiAgICAgICAgICAgaWYgKGV4dF9lbmQgPD0g ZXh0X3N0YXJ0KQpAQCAtNjA3LDcgKzYwNyw3IEBAIGxzZWVrX2NvcHkgKGludCBzcmNfZmQs IGludCBkZXN0X2ZkLCBjaGFyICpidWYsIHNpemVfdCBidWZfc2l6ZSwKICAgICAgICAgfQog CiAgICAgICBleHRfc3RhcnQgPSBsc2VlayAoc3JjX2ZkLCBkZXN0X3BvcywgU0VFS19EQVRB KTsKLSAgICAgIGlmIChleHRfc3RhcnQgPCAwICYmICEgKGV4dF9zdGFydCA9PSAtMSAmJiBl cnJubyA9PSBFTlhJTykpCisgICAgICBpZiAoZXh0X3N0YXJ0IDwgMCAmJiBlcnJubyAhPSBF TlhJTykKICAgICAgICAgZ290byBjYW5ub3RfbHNlZWs7CiAgICAgfQogCkBAIC0xMDkzLDgg KzEwOTMsNyBAQCBpbmZlcl9zY2FudHlwZSAoaW50IGZkLCBzdHJ1Y3Qgc3RhdCBjb25zdCAq c2IsCiAKICNpZmRlZiBTRUVLX0hPTEUKICAgc2Nhbl9pbmZlcmVuY2UtPmV4dF9zdGFydCA9 IGxzZWVrIChmZCwgMCwgU0VFS19EQVRBKTsKLSAgaWYgKDAgPD0gc2Nhbl9pbmZlcmVuY2Ut PmV4dF9zdGFydAotICAgICAgfHwgKHNjYW5faW5mZXJlbmNlLT5leHRfc3RhcnQgPT0gLTEg JiYgZXJybm8gPT0gRU5YSU8pKQorICBpZiAoMCA8PSBzY2FuX2luZmVyZW5jZS0+ZXh0X3N0 YXJ0IHx8IGVycm5vID09IEVOWElPKQogICAgIHJldHVybiBMU0VFS19TQ0FOVFlQRTsKICAg ZWxzZSBpZiAoZXJybm8gIT0gRUlOVkFMICYmICFpc19FTk9UU1VQIChlcnJubykpCiAgICAg cmV0dXJuIEVSUk9SX1NDQU5UWVBFOwotLSAKMi4zNy4yCgo= --------------U2BcLqFAE7z9KMDFBoqzDx3m--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 13 Feb 2023 18:04:28 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 13 13:04:27 2023 Received: from localhost ([127.0.0.1]:51482 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pRdBj-0002R1-I3 for submit <at> debbugs.gnu.org; Mon, 13 Feb 2023 13:04:27 -0500 Received: from mail-wm1-f45.google.com ([209.85.128.45]:53814) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pRdBi-0002Qg-1b for 61386 <at> debbugs.gnu.org; Mon, 13 Feb 2023 13:04:26 -0500 Received: by mail-wm1-f45.google.com with SMTP id he5so1463507wmb.3 for <61386 <at> debbugs.gnu.org>; Mon, 13 Feb 2023 10:04:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=6N/MTzT3189eNDG+ImmbsydmxoqEOQ5ot7MWfpY1r+I=; b=DY9/EnEzFpHK3m6ALxlpzLlnr7W1pa+cjqZBqo+3+pi3ubeFfDmHvCxfPyTpp/rwzQ 8B2oqDZwSmCwVA/w3q3Cm94zZAPNwuXhLLt/DCc6z1kqV8TlG6BFP2TZGp3oGMhDvK7y NcbNxovrVRk1DgOF/Q3DeUbGwivTZtHnHd3F4Pj3lBmWW8k3BZGf1ryUdHJ7mdN6YzP0 htSDIIi9XDZaD3sz+wc/uaAnikfpR5C5a2nqngeaCNYRfKNxiiJ6W5v2qlEQ9t5Tr10l WcgbreNs3tPPjDOjH4CWPIHyTG61Smef4lQLqoUE/Ufac0Me76NJE49/7hhv4WGQUl4J Vcqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6N/MTzT3189eNDG+ImmbsydmxoqEOQ5ot7MWfpY1r+I=; b=H1JysCipcumYHmHP/W07MtXp+5qRSXnoyIPoMLIY8k9einFDvzyeFtsoe/fXaVFq+4 2mqwWFmcu84UEvOElA1NUY0EbJoljdtYXhsZSuGIAPW1YRRv2SrqxYXJTrgz2kirwn5x ELuLAWRNNifeCdIcC5b0sMa5uLuDE0L8M4AqVrKFtd1VEB+umc3tbcFA9DnI0Uqv/V4i NBINvKpieM9LFyf6Mi7HYseDiKx6eAydNV2MRVb1nJVP3NxCP0S9phz9fq/9eLWg8j0Q T+pDrZfhAyi+Zo5ayMRIa0TMGworU/FA45CE2Ri8jWMzIB6sISPJnuxB9DBo1B4T2p7F iQuQ== X-Gm-Message-State: AO0yUKXgtwNX9bggXOrc6xboQKLHlFtZ9+KfBcck/s5pdBqzb9CK1YB/ 9NZGSqShUG1L/vouFs/KBRQ= X-Google-Smtp-Source: AK7set8enoMkFlWfyxzMuugwcILt7nKAb0E13O2WHgWlSKa3OYz+CQXxvP9jjAZ+0LKAC3tAnVrchA== X-Received: by 2002:a05:600c:992:b0:3df:ed95:d757 with SMTP id w18-20020a05600c099200b003dfed95d757mr19377400wmp.34.1676311459287; Mon, 13 Feb 2023 10:04:19 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id p24-20020a05600c1d9800b003dd1bd0b915sm17493256wms.22.2023.02.13.10.04.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 13 Feb 2023 10:04:18 -0800 (PST) Message-ID: <27f273c8-3c3f-5cf3-8b5b-978a29dce9db@HIDDEN> Date: Mon, 13 Feb 2023 18:04:18 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, Paul Eggert <eggert@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 13/02/2023 17:16, George Valkov wrote: > >> On 2023-02-13, at 4:05 PM, Pádraig Brady <P@HIDDEN> wrote: >> This amendment seems to make the treatment of umask and setuid consistent >> between using fclonefileat() and not. >> >> diff --git a/src/copy.c b/src/copy.c >> index 782fab98d..8370f55bd 100644 >> --- a/src/copy.c >> +++ b/src/copy.c >> @@ -1280,10 +1280,11 @@ copy_reg (char const *src_name, char const *dst_name, >> return_val = false; >> } >> } >> - mode_already_preserved = (fc_flags & CLONE_ACL) != 0; >> + mode_already_preserved = (fc_flags & CLONE_ACL) != 0 >> + && cloned_mode == src_mode; >> dest_desc = -1; >> omitted_permissions = dst_mode & ~cloned_mode; >> - extra_permissions = 0; >> + extra_permissions = dst_mode & ~ cached_umask (); >> goto set_dest_mode; >> } >> else if (! handle_clone_fail (dst_dirfd, dst_relname, src_name, >> >> With this, all tests pass (apart from thru-dangling.sh as mentioned above >> which I'll skip in another patch). >> Paul I'm happy to merge this amendment, mention the improvement in NEWS, >> and push in your name if you're OK with it. > > To summarise: > 1. We apply my original patch to disable sparse-copy on macOS. Otherwise since fclonefileat is not used whenever we overwrite a file, corruption will still occur. An updated copy of the patch is attached below: > By doing this we fail the following test so you have to disable it on macOS: > FAIL: tests/cp/sparse-perf.sh Right. I'll adjust your patch to also get tests/seek-data-capable to return false on Darwin, which will then skip that test. > 2. Next we add support for fclonefileat using cp: improve use of fclonefileat > A few more tests fail. > > 3. We fix those tests using the patch from your last mail. Then we have to disable the following test on macOS: > FAIL: tests/cp/thru-dangling.sh > > Final results: copying sparse-files on macOS no longer produces corrupted copies. Performance is greatly optimised by cloning instead of copying, where possible. The following two tests now fail on macOS and will be disabled: > FAIL: tests/cp/sparse-perf.sh handled above > FAIL: tests/cp/thru-dangling.sh I'll fix this > Looks good to me. Yes, please add me to the commit history. Also feel free to add my web site if appropriate. Finally please let me know when the patches are pushed, so that I can update OpenWRT to use the latest version of coreutils. > > Pádraig and Paul, thank you very much for your kind help! It was nice working with you and learning new experience. Cheers, mates! Summary sounds good. Thanks for all the testing. Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 13 Feb 2023 17:21:19 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 13 12:21:19 2023 Received: from localhost ([127.0.0.1]:51373 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pRcVx-0004l8-9R for submit <at> debbugs.gnu.org; Mon, 13 Feb 2023 12:21:19 -0500 Received: from mail-wm1-f45.google.com ([209.85.128.45]:37515) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pRcVs-0004kp-BC for 61386 <at> debbugs.gnu.org; Mon, 13 Feb 2023 12:21:16 -0500 Received: by mail-wm1-f45.google.com with SMTP id m20-20020a05600c3b1400b003e1e754657aso3359007wms.2 for <61386 <at> debbugs.gnu.org>; Mon, 13 Feb 2023 09:21:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=yHhFuMuSyjQDid04gg76gA3OdXU6J9nrWrVmxrC2TMU=; b=FUXe3WZcAzmZr19LSGpd7aGQGdYJDZ6f/eA2pHeKQ+boBz9iHfdJPh397lvzvgadoY df2OrM0n7yWempHxPGxORz2qrOcwSQNo9sKweRQ9Cc9YFwiNfjhsfg7SgDHvCmmBpFU/ PBDwI2UJ++jIktKxNRxv/h41uEa6p71CubE0PfCf4Rcw0+tl2zc8Gk9jhaZ8Fds8xPj7 VFpGa+is0vTT9vlT8PqdlYwJ85ENpm3FIfP4sXghAGiDUKlTVRgxYAmK62xypjggnzIK GkHuXM3zQMk+1acXG3CsT6QrQCcXHkaCxPSj3aWx7fAgR01vbflZUMKSF3DONczHVzZg wiWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yHhFuMuSyjQDid04gg76gA3OdXU6J9nrWrVmxrC2TMU=; b=8EicwAa2yy/NAhLsSaZDC4Ns4F/Dcq9kskI6ZZM0Wp3OSr3PXnH2IwEyLrlH3al+58 Y9raXFlkBXK8oITtdQeZGPL5n5A10R4Z8/olq6BvRBXPATwks/A6gg00qgRrMiiZLlBd q3giJjarMC9Am3KrZw7WU0uvv/r/TrT2rHgpGeVNx0VcOUmeeir1tCvV6enKsTLMI05q s9Hu3lClbwlsnqP8pfusvZmFZ2iT0sdR5AwL1caXnvZno/D/kbKlSOJuz7as+J3Gtwjl fNxIjsQAYJCKdhX7xDQScvkNvoSn4Cd/E20L3pm7p2aPxpGx3f0E6U6q3V59BdY/yN4D bmyg== X-Gm-Message-State: AO0yUKWl2WwqLNPrDmvYAW+MWAozdzIYL10N345zBzhCNjlG/ShQKktn TFGH4xhFbD+QMrTgfFAf6Cg= X-Google-Smtp-Source: AK7set/GNFIgPp6h7wfN0fEH+3i/ZAn88UEALn1lM37m6x2CsLuZUhM8SJpv+60jnC/jruE7KI37ag== X-Received: by 2002:a05:600c:1c1b:b0:3dc:4234:fe89 with SMTP id j27-20020a05600c1c1b00b003dc4234fe89mr25854617wms.2.1676308866440; Mon, 13 Feb 2023 09:21:06 -0800 (PST) Received: from smtpclient.apple (212-39-89-241.ip.btc-net.bg. [212.39.89.241]) by smtp.gmail.com with ESMTPSA id i6-20020adfe486000000b002c5560a228bsm4061107wrm.95.2023.02.13.09.21.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Feb 2023 09:21:05 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> Date: Mon, 13 Feb 2023 19:21:04 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <A2257095-5C15-4B58-B55F-DA492468828A@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN> <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN> To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>, Paul Eggert <eggert@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Test results on macOS 12.6.3 = https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/test-02= -d374d32ccf12f8cf33c8f823d573498b7c8b27a4-clone.txt Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 13 Feb 2023 17:16:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 13 12:16:24 2023
Received: from localhost ([127.0.0.1]:51368 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pRcRE-0004ci-AD
for submit <at> debbugs.gnu.org; Mon, 13 Feb 2023 12:16:24 -0500
Received: from mail-wm1-f42.google.com ([209.85.128.42]:55186)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pRcRA-0004cS-J2
for 61386 <at> debbugs.gnu.org; Mon, 13 Feb 2023 12:16:22 -0500
Received: by mail-wm1-f42.google.com with SMTP id n13so9185783wmr.4
for <61386 <at> debbugs.gnu.org>; Mon, 13 Feb 2023 09:16:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:from:to:cc:subject:date:message-id:reply-to;
bh=GLuTrMlAH6gwaQenFyxo1TpEWNUU8h9fmJJSANPa+Ts=;
b=CPpupX+MuhBGd3Pfef9qrVZsafTpnG5eyP5wJrOO5DyxE70+bMrDlKFoZTPqiPteWt
pDJpv51MFwSwbKEcK8SUUELrYJpgzEhg4fRpp5I5ABu8ZokWLw9vt0XDVF2YMCvoLrst
B9Q/TjM5vQapskOemoTHnJwJtJqjePD/p7zB6IgeFYcoB6Bq7yOVd/zwbP7LY7PcH9zc
t83yNUGEAiIfvedmuTUkwfvP6y4Z9lun3Zrn6y7llYwxglUsNeRkgr8W1B+xfYX+brb4
LB3tX3hiNvNEqlzsrT8Zg4JXliUkZDYZdE+03/mysto6mVZpnaVQDre1ylu47XcATqcT
pm8Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=GLuTrMlAH6gwaQenFyxo1TpEWNUU8h9fmJJSANPa+Ts=;
b=klhzBaVcih/seFouy0X9PeF/irPNczVrL3/uukszXEKfWr2hGUjZnxVKhrfGMdKuDV
uqk/DW/50owTKj+jF3pk5FXS4rpxS8wxAP54afKa8no9t/30deVEn3S8oiktXbkCOIJy
lfhS7YTr5SyawQgCd/Xj+ujHUoP+ja7UcjTHW6rmgGmj0RnegMsFOaUhCobxGJAy6ILY
eu6A0NrvAkX8mMZOewAJvYqQSH0r5Rys0wpc/8CoXTjUKx7L+GM0Z7JZSvFIyrj0yt3E
dW31H/gUEjG4kuHxqRqivjQ9CrUP/T7tDbu4Bqc+xxIXQZTPu0Q5hbk3Pdc+4J6Sw2tv
Y1LQ==
X-Gm-Message-State: AO0yUKW6UFmuOfYJY0Sw26t84VprJmi1tUSoyQ3IN6r4mG/YRj7gdKMe
FurGy8ABz488txe+XXpX6Uo=
X-Google-Smtp-Source: AK7set8jqOZolX/a3Gz6pA88evDmUtrXi5AXmoQO3WUu/fuEujvHhYUAmcJVvpLzMbcfPW8l8Rv5qA==
X-Received: by 2002:a05:600c:3ca3:b0:3dc:5ae4:c13d with SMTP id
bg35-20020a05600c3ca300b003dc5ae4c13dmr25833089wmb.4.1676308574445;
Mon, 13 Feb 2023 09:16:14 -0800 (PST)
Received: from smtpclient.apple (212-39-89-241.ip.btc-net.bg. [212.39.89.241])
by smtp.gmail.com with ESMTPSA id
ja13-20020a05600c556d00b003dc4b4dea31sm13790079wmb.27.2023.02.13.09.16.13
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Mon, 13 Feb 2023 09:16:13 -0800 (PST)
From: George Valkov <gvalkov@HIDDEN>
Message-Id: <63F8AEAE-94AE-4E7B-B148-57E71397D840@HIDDEN>
Content-Type: multipart/mixed;
boundary="Apple-Mail=_96C5770C-10B3-4A7E-BD69-594954E64CD9"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Date: Mon, 13 Feb 2023 19:16:11 +0200
In-Reply-To: <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>,
Paul Eggert <eggert@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
<f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--Apple-Mail=_96C5770C-10B3-4A7E-BD69-594954E64CD9
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
> On 2023-02-13, at 4:05 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 12/02/2023 20:36, P=C3=A1draig Brady wrote:
>> On 12/02/2023 14:15, P=C3=A1draig Brady wrote:
>>> On 11/02/2023 21:53, Paul Eggert wrote:
>>>> On 2023-02-10 17:21, George Valkov wrote:
>>>>=20
>>>>> remember to trace all code paths errors.
>>>>=20
>>>> Yes, I've been doing that. However, I don't use macOS and the two =
of us
>>>> are debugging remotely where I write the code and you debug it but
>>>> neither of us know how macOS works. Of course it would be much more
>>>> efficient if we weren't operating with these obstacles, but here we =
are.
>>>>=20
>>>> Because the macOS behavior is poorly documented and we lack the =
source,
>>>> if we can't figure this out fairly quickly coreutils should simply =
stop
>>>> using fclonefileat because it's unreliable for users and a time =
sink for
>>>> us. I'm hoping, though, we can get something working with another =
round
>>>> or two.
>>>>=20
>>>>=20
>>>>> With CLONE_ACL, I get 22 Invalid argument.
>>>>=20
>>>> OK, this means that although Apple has documented CLONE_ACL and =
it's in
>>>> your include files, it doesn't work in your version of macOS, =
because
>>>> either it's not supported by the kernel, or by the filesystem code, =
or
>>>> by the particular filesystem instance, or for some other reason. =
EINVAL
>>>> hints that it's the kernel but that's by no means certain.
>>>>=20
>>>> One possible way to defend against this macOS misbehavior is for cp =
to
>>>> try to use CLONE_ACL, and if that fails with errno =3D=3D EINVAL =
try again
>>>> without CLONE_ACL. I attempted to implement this in the attached =
patch;
>>>> please give it a try.
>>>=20
>>> I tested this on macOS 13.2 and the first fclonefileat(..., =
CLONE_ACL) succeeds.
>>> I tested with various umask, --no-preserve=3Dmode, -p, ... =
combinations
>>> and didn't find any inconsistencies in permissions with =
--reflink=3Dnever copies.
>>> Actually scratch that. There was an older xcode installed so =
CLONE_ACL was 0.
>>> I've not got access at the moment, so will need to retest later with =
newer xcode.
>> Good news is that with newer xcode supporting macOS 13.1
>> CLONE_ACL is set and that all works as expected, with the
>> first fclonefileat() succeeding.
>> I checked with actual ACLs (set with `chmod +a "user:$USER allow =
readattr" file`),
>> and they were copied or dropped as expected. Note ls -l on macOS will =
list
>> ACLs with a "+" rather than a "@", as was mentioned elsewhere in this =
thread.
>>> One divergence with fclonefileat() is that it writes through a =
dangling symlink,
>>> where a standard --reflink=3Dnever copy will fail.
>>> I.e. `make TESTS=3Dtests/cp/thru-dangling.sh SUBDIRS=3D. check` =
fails.
>>> Now there is nothing wrong with writing through the dangling =
symlink,
>>> and in fact standard cp will also with POSIXLY_CORRECT set in the =
environment.
>>> So I may just need need to adjust the test to work / skip in this =
case.
>> Another divergence I noticed is wrt umask and setuid handling.
>> The following transcript shows that with fclonefileat() we do _not_
>> preserve setuid when it is preserved without.
>> Also with fclonefileat() the umask is not honored,
>> while it is honored without.
>> # id -u
>> 0
>> # echo create setuid file
>> # rm src/cp.s; src/cp src/cp src/cp.s; chmod u+s src/cp.s
>> # umask; for reflink in always never; do
>> > for preserve in mode timestamps; do
>> > rm -f src/cp.s.$reflink.$preserve
>> > src/cp --reflink=3D$reflink --preserve=3D$preserve src/cp.s =
src/cp.s.$reflink.$preserve
>> > done
>> > done
>> > ls -le src/cp.s*
>> 0022
>> -rwsrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s
>> -rwxrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.always.mode
>> -rwxrwxr-x 1 root staff 234032 12 Feb 20:21 =
src/cp.s.always.timestamps
>> -rwsrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.never.mode
>> -rwxr-xr-x 1 root staff 234032 12 Feb 20:21 =
src/cp.s.never.timestamps
>=20
> This amendment seems to make the treatment of umask and setuid =
consistent
> between using fclonefileat() and not.
>=20
> diff --git a/src/copy.c b/src/copy.c
> index 782fab98d..8370f55bd 100644
> --- a/src/copy.c
> +++ b/src/copy.c
> @@ -1280,10 +1280,11 @@ copy_reg (char const *src_name, char const =
*dst_name,
> return_val =3D false;
> }
> }
> - mode_already_preserved =3D (fc_flags & CLONE_ACL) !=3D =
0;
> + mode_already_preserved =3D (fc_flags & CLONE_ACL) !=3D =
0
> + && cloned_mode =3D=3D =
src_mode;
> dest_desc =3D -1;
> omitted_permissions =3D dst_mode & ~cloned_mode;
> - extra_permissions =3D 0;
> + extra_permissions =3D dst_mode & ~ cached_umask ();
> goto set_dest_mode;
> }
> else if (! handle_clone_fail (dst_dirfd, dst_relname, =
src_name,
>=20
> With this, all tests pass (apart from thru-dangling.sh as mentioned =
above
> which I'll skip in another patch).
> Paul I'm happy to merge this amendment, mention the improvement in =
NEWS,
> and push in your name if you're OK with it.
To summarise:
1. We apply my original patch to disable sparse-copy on macOS. Otherwise =
since fclonefileat is not used whenever we overwrite a file, corruption =
will still occur. An updated copy of the patch is attached below:
By doing this we fail the following test so you have to disable it on =
macOS:
FAIL: tests/cp/sparse-perf.sh
--Apple-Mail=_96C5770C-10B3-4A7E-BD69-594954E64CD9
Content-Disposition: attachment;
filename=0001-cp-mv-install-Disable-sparse-copy-on-macOS.patch
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="0001-cp-mv-install-Disable-sparse-copy-on-macOS.patch"
Content-Transfer-Encoding: quoted-printable
=46rom=20f0967282fb44e8739d48b2775ea78c613920491c=20Mon=20Sep=2017=20=
00:00:00=202001=0AFrom:=20Georgi=20Valkov=20<gvalkov@HIDDEN>=0ADate:=20=
Mon,=2013=20Feb=202023=2018:34:24=20+0200=0ASubject:=20[PATCH]=20=
cp,mv,install:=20Disable=20sparse=20copy=20on=20macOS=0A=0ADue=20to=20a=20=
bug=20in=20macOS,=20sparse=20copies=20are=20corrupted=20on=20virtual=20=
disks=0Aformatted=20with=20APFS.=20HFS=20is=20not=20affected.=20Affected=20=
are=20coreutils=0Ainstall,=20and=20gcp=20when=20compiled=20with=20=
SEEK_HOLE,=20as=20well=20as=20macOS=20Finder.=0A=0AWhile=20reading=20the=20=
entire=20file=20returns=20valid=20data,=20scanning=20for=0Aallocated=20=
segments=20may=20return=20holes=20where=20valid=20data=20is=20present.=0A=
In=20this=20case=20a=20sparse=20copy=20does=20not=20contain=20these=20=
segments=20and=20return=0Azeroes=20instead.=20Once=20the=20virtual=20=
disk=20is=20dismounted=20and=20then=0Amounted=20again,=20a=20sparse=20=
copy=20produces=20correct=20results.=0A=0AThis=20breaks=20OpenWRT=20=
build=20on=20macOS.=20Details:=0A=
https://github.com/openwrt/openwrt/pull/11960=0A=0ASigned-off-by:=20=
Georgi=20Valkov=20<gvalkov@HIDDEN>=0A---=0A=20src/copy.c=20|=202=20+-=0A=
=201=20file=20changed,=201=20insertion(+),=201=20deletion(-)=0A=0Adiff=20=
--git=20a/src/copy.c=20b/src/copy.c=0Aindex=208370f55bd..4dad681f9=20=
100644=0A---=20a/src/copy.c=0A+++=20b/src/copy.c=0A@@=20-1062,7=20=
+1062,7=20@@=20infer_scantype=20(int=20fd,=20struct=20stat=20const=20=
*sb,=0A=20=20=20=20=20=20=20=20=20=20&&=20ST_NBLOCKS=20(*sb)=20<=20=
sb->st_size=20/=20ST_NBLOCKSIZE))=0A=20=20=20=20=20return=20=
PLAIN_SCANTYPE;=0A=20=0A-#ifdef=20SEEK_HOLE=0A+#if=20defined(SEEK_HOLE)=20=
&&=20!defined(__APPLE__)=0A=20=20=20off_t=20ext_start=20=3D=20lseek=20=
(fd,=200,=20SEEK_DATA);=0A=20=20=20if=20(0=20<=3D=20ext_start=20||=20=
errno=20=3D=3D=20ENXIO)=0A=20=20=20=20=20{=0A--=20=0A2.39.1=0A=0A=
--Apple-Mail=_96C5770C-10B3-4A7E-BD69-594954E64CD9
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
2. Next we add support for fclonefileat using cp: improve use of =
fclonefileat
A few more tests fail.
3. We fix those tests using the patch from your last mail. Then we have =
to disable the following test on macOS:
FAIL: tests/cp/thru-dangling.sh
Final results: copying sparse-files on macOS no longer produces =
corrupted copies. Performance is greatly optimised by cloning instead of =
copying, where possible. The following two tests now fail on macOS and =
will be disabled:
FAIL: tests/cp/sparse-perf.sh
FAIL: tests/cp/thru-dangling.sh
Looks good to me. Yes, please add me to the commit history. Also feel =
free to add my web site if appropriate. Finally please let me know when =
the patches are pushed, so that I can update OpenWRT to use the latest =
version of coreutils.
P=C3=A1draig and Paul, thank you very much for your kind help! It was =
nice working with you and learning new experience. Cheers, mates!
Georgi Valkov
httpstorm.com
nano RTOS
--Apple-Mail=_96C5770C-10B3-4A7E-BD69-594954E64CD9--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 13 Feb 2023 14:15:32 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 13 09:15:32 2023 Received: from localhost ([127.0.0.1]:48049 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pRZcC-00055E-Dq for submit <at> debbugs.gnu.org; Mon, 13 Feb 2023 09:15:32 -0500 Received: from mail-wm1-f48.google.com ([209.85.128.48]:35678) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pRZc9-00054z-JV for 61386 <at> debbugs.gnu.org; Mon, 13 Feb 2023 09:15:31 -0500 Received: by mail-wm1-f48.google.com with SMTP id k8-20020a05600c1c8800b003dc57ea0dfeso11410223wms.0 for <61386 <at> debbugs.gnu.org>; Mon, 13 Feb 2023 06:15:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=QkntrrsHy50YACG+plyfdBavfbcRPGzhqAWxhY2vAWA=; b=NqM0dHlroPFgyYMA0WPnQDRxUCiZfuU0iC7Cu+xn64zAm1OssemVdJo9lStS0CyjZ4 MEiK7WuzcqbPOOR+sgEZXrRH+ZRqEBFUribsG/GvjdOooJXAcH4llzu+AlfwKel6aSij 0Sw/VCqacwWzXiSmEnhGOlAjWeqGS1LBtSU1EnKp4kXbRMEtwj2i1d+JrY/fipdKJmw5 KbMCFY4nonrdTIARfElaOAStTdOaF7iXyr5h10u4d6A/2WZySZ1UEY67zUW3WvIXvJsS 5gJeMB5CyJvDsvYLllzHuEQCH81GteAit7wwczgnCwOq8hD990PZ70Q8VrV4FOyx0YDa 85MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QkntrrsHy50YACG+plyfdBavfbcRPGzhqAWxhY2vAWA=; b=PWlKL4Q5SXe1FuhlFEsgHh1nOQavVbUSejGwLXMarPOtvshiWuQErXPzYldj36ix/0 v1zukFSqHRhuMh50RLGwWq5xhJA914LWJOo5F/RllaKZQUL78G6uvRf7N6ik/pgl0mc4 upHdQl4MfijI4R8khMa5IytxSrnw4q4Q/VNede/KEthOwFGCF+26BJMIuorOYQd+sFTu Sa/O97r8XGVURV3uNy4f1W5ZGeYk1Hz8A48/VChhovK4GsFmxxUtzegCf8Ss9gdHzXib 0dOrrc3EoTd8ktR6dR0g9g4S+SH/zm0kPV/QhqlrgOsekmyUeNNPxECE8mps16hshZuT QhNw== X-Gm-Message-State: AO0yUKVZ0l8AJz9+4un755ljt/6Uuwa7Jx6QJROGwHQUVt8dXRkbeLvW e3r9JtCQCvGoEQBqt9Khx4E= X-Google-Smtp-Source: AK7set9eZDjwpw4yEddVHhfbvt5yY8zVZGZ8LNE7nHnTIsWerCSFCF15sTzzGb3KyiXsP5N4RIFCZA== X-Received: by 2002:a05:600c:3085:b0:3e0:99:b59d with SMTP id g5-20020a05600c308500b003e00099b59dmr18993821wmn.21.1676297723619; Mon, 13 Feb 2023 06:15:23 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id h12-20020adff4cc000000b002be505ab59asm10762880wrp.97.2023.02.13.06.15.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 13 Feb 2023 06:15:23 -0800 (PST) Message-ID: <5b59ac55-c7a7-89a4-45ab-09ff1f38ca6c@HIDDEN> Date: Mon, 13 Feb 2023 14:15:22 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> <5ED86735-B8C7-46EA-B5ED-3B47181AE870@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <5ED86735-B8C7-46EA-B5ED-3B47181AE870@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: Paul Eggert <eggert@HIDDEN>, 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 12/02/2023 23:37, George Valkov wrote: > > I can confirm that. CLONE_ACL is 4 when building on macOS 12.6.3. I rand the compiled sample on the recovery environment, which is equivalent to macOS 13.3. The flag is supported, though I did not observe any difference with or without it. The sample was ran as root, while the source and destination were set to g:staff 600 @. I guess the CLONE_ACL value is more dependent on xcode rather than macOS version. I confirmed again on macOS 13.2 that CLONE_ACL is honored and behaves as expected. I guess on 12.6.3 it's failing with EINVAL and falling back to the CLONE_ACL==0 case. cheers, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 13 Feb 2023 14:05:50 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 13 09:05:50 2023
Received: from localhost ([127.0.0.1]:48031 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pRZSo-0004pp-DL
for submit <at> debbugs.gnu.org; Mon, 13 Feb 2023 09:05:50 -0500
Received: from mail-wr1-f43.google.com ([209.85.221.43]:37718)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <pixelbeat@HIDDEN>) id 1pRZSm-0004pa-A0
for 61386 <at> debbugs.gnu.org; Mon, 13 Feb 2023 09:05:49 -0500
Received: by mail-wr1-f43.google.com with SMTP id m10so3438084wrn.4
for <61386 <at> debbugs.gnu.org>; Mon, 13 Feb 2023 06:05:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=content-transfer-encoding:in-reply-to:references:cc:to:from
:content-language:subject:user-agent:mime-version:date:message-id
:sender:from:to:cc:subject:date:message-id:reply-to;
bh=y28xEmfCgI7zTNURyLc4P7o9g0VicWt9YuVGRgFEWN8=;
b=mkYUCN/FKykpHOwMUXjJrHFAbYvYikMla6mUlcHNuq4sQXt+y6km05DyMIF3K5k6gO
2FrK7PF8onog/HB36bodxUFZC/cJkQlFISQTiXtBVGd95/gD+Rj2V8L5wFyA7xmHmzHe
v9zxJcBwn+27j3GLlCwoP5PCYp30zZH6HU4cIziV5KfYvfFWntJnRH4PSCGKvCpif85s
7fhdmZ2ymx0RKHw/zN92CTdkmH198TCFSk3K2EfbkQu4beRWp90N/B93QiRRctQ+hW4G
pJzrLZl6KeVsDPdPNM4uW0PZuSUmBjE9C7poqQzrlTkTw6fdozSD2kulmKoWemSylEFr
TO4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=content-transfer-encoding:in-reply-to:references:cc:to:from
:content-language:subject:user-agent:mime-version:date:message-id
:sender:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=y28xEmfCgI7zTNURyLc4P7o9g0VicWt9YuVGRgFEWN8=;
b=JOPMX8p7zAPuLObDeT+AHX0T41uXANThldpRLSnIuDldQ2JSsMdPEtECRo6Ttuup4S
Q8Bsphw/KNsGtAupF5/6lP3QUNLiBZmSboAcx4ScVswZX7/oGfr1w6nr+OzPottdkSVo
p2zbYDYBCR198uzfhsLUfEPbwOzdgIS2eSRzVa7ZrY0I879zVElyZRKwHxRkvaPmRUlZ
PelTdN9NMhCXHkmtX1L/UqR4nPe6rm7HISbN7aiASQxt3UxQeblXftjGDlBoBe5ujeEE
YM2D8s6Dn8m7x2Z5PYLTj5hYgPqAHOVdK7YPchJeh+5Yl/XuMa3epsotT0ptnPEd/RDF
SiHQ==
X-Gm-Message-State: AO0yUKWTs6CNVBNdYmIXTi8dAx2fVlOaZqLfAjjFGSlOgo0ckIVf91a7
gtWOhKp2WOdnex3so1R+TgI=
X-Google-Smtp-Source: AK7set9RqUt/Ko5kY1ZQJMjO1tJABGRiqVfAT1VdR96BWJcYQpsA8TljFBKkYLnzZWNbgW3mQyS9gQ==
X-Received: by 2002:adf:f446:0:b0:2c5:563a:85c8 with SMTP id
f6-20020adff446000000b002c5563a85c8mr3684715wrp.11.1676297142185;
Mon, 13 Feb 2023 06:05:42 -0800 (PST)
Received: from [192.168.1.9]
(95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175])
by smtp.googlemail.com with ESMTPSA id
d7-20020adfe887000000b002c551f7d452sm5339454wrm.98.2023.02.13.06.05.41
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Mon, 13 Feb 2023 06:05:41 -0800 (PST)
Message-ID: <f7f2a9d8-9fac-13a9-1b06-57eca40f83c3@HIDDEN>
Date: Mon, 13 Feb 2023 14:05:40 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101
Thunderbird/109.0
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Content-Language: en-US
From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
In-Reply-To: <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.9 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.9 (-)
On 12/02/2023 20:36, Pádraig Brady wrote:
> On 12/02/2023 14:15, Pádraig Brady wrote:
>> On 11/02/2023 21:53, Paul Eggert wrote:
>>> On 2023-02-10 17:21, George Valkov wrote:
>>>
>>>> remember to trace all code paths errors.
>>>
>>> Yes, I've been doing that. However, I don't use macOS and the two of us
>>> are debugging remotely where I write the code and you debug it but
>>> neither of us know how macOS works. Of course it would be much more
>>> efficient if we weren't operating with these obstacles, but here we are.
>>>
>>> Because the macOS behavior is poorly documented and we lack the source,
>>> if we can't figure this out fairly quickly coreutils should simply stop
>>> using fclonefileat because it's unreliable for users and a time sink for
>>> us. I'm hoping, though, we can get something working with another round
>>> or two.
>>>
>>>
>>>> With CLONE_ACL, I get 22 Invalid argument.
>>>
>>> OK, this means that although Apple has documented CLONE_ACL and it's in
>>> your include files, it doesn't work in your version of macOS, because
>>> either it's not supported by the kernel, or by the filesystem code, or
>>> by the particular filesystem instance, or for some other reason. EINVAL
>>> hints that it's the kernel but that's by no means certain.
>>>
>>> One possible way to defend against this macOS misbehavior is for cp to
>>> try to use CLONE_ACL, and if that fails with errno == EINVAL try again
>>> without CLONE_ACL. I attempted to implement this in the attached patch;
>>> please give it a try.
>>
>> I tested this on macOS 13.2 and the first fclonefileat(..., CLONE_ACL) succeeds.
>> I tested with various umask, --no-preserve=mode, -p, ... combinations
>> and didn't find any inconsistencies in permissions with --reflink=never copies.
>> Actually scratch that. There was an older xcode installed so CLONE_ACL was 0.
>> I've not got access at the moment, so will need to retest later with newer xcode.
>
> Good news is that with newer xcode supporting macOS 13.1
> CLONE_ACL is set and that all works as expected, with the
> first fclonefileat() succeeding.
> I checked with actual ACLs (set with `chmod +a "user:$USER allow readattr" file`),
> and they were copied or dropped as expected. Note ls -l on macOS will list
> ACLs with a "+" rather than a "@", as was mentioned elsewhere in this thread.
>
>> One divergence with fclonefileat() is that it writes through a dangling symlink,
>> where a standard --reflink=never copy will fail.
>> I.e. `make TESTS=tests/cp/thru-dangling.sh SUBDIRS=. check` fails.
>> Now there is nothing wrong with writing through the dangling symlink,
>> and in fact standard cp will also with POSIXLY_CORRECT set in the environment.
>> So I may just need need to adjust the test to work / skip in this case.
>
> Another divergence I noticed is wrt umask and setuid handling.
> The following transcript shows that with fclonefileat() we do _not_
> preserve setuid when it is preserved without.
> Also with fclonefileat() the umask is not honored,
> while it is honored without.
>
> # id -u
> 0
>
> # echo create setuid file
> # rm src/cp.s; src/cp src/cp src/cp.s; chmod u+s src/cp.s
>
> # umask; for reflink in always never; do
> > for preserve in mode timestamps; do
> > rm -f src/cp.s.$reflink.$preserve
> > src/cp --reflink=$reflink --preserve=$preserve src/cp.s src/cp.s.$reflink.$preserve
> > done
> > done
> > ls -le src/cp.s*
> 0022
> -rwsrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s
> -rwxrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.always.mode
> -rwxrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.always.timestamps
> -rwsrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.never.mode
> -rwxr-xr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.never.timestamps
This amendment seems to make the treatment of umask and setuid consistent
between using fclonefileat() and not.
diff --git a/src/copy.c b/src/copy.c
index 782fab98d..8370f55bd 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -1280,10 +1280,11 @@ copy_reg (char const *src_name, char const *dst_name,
return_val = false;
}
}
- mode_already_preserved = (fc_flags & CLONE_ACL) != 0;
+ mode_already_preserved = (fc_flags & CLONE_ACL) != 0
+ && cloned_mode == src_mode;
dest_desc = -1;
omitted_permissions = dst_mode & ~cloned_mode;
- extra_permissions = 0;
+ extra_permissions = dst_mode & ~ cached_umask ();
goto set_dest_mode;
}
else if (! handle_clone_fail (dst_dirfd, dst_relname, src_name,
With this, all tests pass (apart from thru-dangling.sh as mentioned above
which I'll skip in another patch).
Paul I'm happy to merge this amendment, mention the improvement in NEWS,
and push in your name if you're OK with it.
cheers,
Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 12 Feb 2023 23:38:01 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Feb 12 18:38:01 2023
Received: from localhost ([127.0.0.1]:47137 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pRLuy-0000ge-Tq
for submit <at> debbugs.gnu.org; Sun, 12 Feb 2023 18:38:01 -0500
Received: from mail-wm1-f44.google.com ([209.85.128.44]:43921)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pRLux-0000gS-44
for 61386 <at> debbugs.gnu.org; Sun, 12 Feb 2023 18:37:59 -0500
Received: by mail-wm1-f44.google.com with SMTP id
az4-20020a05600c600400b003dff767a1f1so7807405wmb.2
for <61386 <at> debbugs.gnu.org>; Sun, 12 Feb 2023 15:37:59 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=j2/7gyhOTTXMjVDRvmE/gc8tYx8c+NnGxLivPQ3lJRQ=;
b=GpUV7SUT4YjyCQQsUmta/09TqS9Tm9X7ZZk36NOLW42/SeaGqQnEEStzCzQ8U4afH7
Iix/b+j5k9bl4KxZFxukaBSqmugDI7wJhEDFnS4gcz6d0a9Tytc45yagwijYv35uTjcC
FgQJamaanjhKK1giVkPlSsEc959aKImkmqFvcOQpzLoj58ibsj+prXxk88xlDzuLRmMI
uhfbKrPmNbWx6LAO1IKDylQz8DJ+q/imR3BXEEcb1vKVHMPmzmmqBJaaFTQTXJ6v2bng
++Ksu0b9UEYagpzHv9ecadiE9NP7wc8frg1rrpqzGxwfLC2rhzlPjILxCGtaYoVrh3Wa
tZpQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=j2/7gyhOTTXMjVDRvmE/gc8tYx8c+NnGxLivPQ3lJRQ=;
b=exxVUKl6GM7xgHbNdTRW2qhxSB0PrCire+EL/lVwWTSyOEtimBui+8v09+f/EeNCvG
3JapfugB0htdrLfc1xwMEVYKtrxP75TmLaFtHVS/dsvCJ6Xd8O2TzsRqkfgG5rEcJ6rk
9vS50KXFja3eNl0DAV1Wj7W7pjwoX7tNXkdmy5wLgjhK+V0l4YjMJnu30YAvqYkxUgAL
gXja/jFSfThPYt12CXaPrePrLQev0z1Lox/eDjwYwOJfLIbJbvS26Le2qZuarOeF8paC
NgDaDgLIMSYt6B8ndlF5kt/DfbOT8xXWBZvERkvgrGstvfH6V6UiXqQnDP3W6kFCXJiH
Wlqw==
X-Gm-Message-State: AO0yUKWC5ZcgAurlV6yhdO4JtpQ+MZCramce6kKPeXBXTYVSDmOGbdbS
qFRnfi9Sq17miQZEbiDcxoY=
X-Google-Smtp-Source: AK7set8+CPNfL19NZs/6r/tx5essHcUXfq100iZgheDGF9Wucf7n78o7J17J9/mWiijd5jsjw7q1LQ==
X-Received: by 2002:a05:600c:4f8e:b0:3dc:5032:2895 with SMTP id
n14-20020a05600c4f8e00b003dc50322895mr21715249wmq.0.1676245073121;
Sun, 12 Feb 2023 15:37:53 -0800 (PST)
Received: from smtpclient.apple (212-5-158-145.ip.btc-net.bg. [212.5.158.145])
by smtp.gmail.com with ESMTPSA id
v17-20020a5d43d1000000b002c54bf0e9f7sm6382753wrr.117.2023.02.12.15.37.50
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Sun, 12 Feb 2023 15:37:51 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
Date: Mon, 13 Feb 2023 01:37:49 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <5ED86735-B8C7-46EA-B5ED-3B47181AE870@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN>
<895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 1.5 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: > On 2023-02-12, at 10:36 PM, Pádraig Brady wrote: > > On
12/02/2023 14:15, Pádraig Brady wrote: >> On 11/02/2023 21:53, Paul Eggert
wrote: >>> On 2023-02-10 17:21, George Valkov wrote: >>> >>>> r [...]
Content analysis details: (1.5 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (gvalkov[at]gmail.com)
1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
[212.5.158.145 listed in dnsbl.sorbs.net]
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust
[209.85.128.44 listed in list.dnswl.org]
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[209.85.128.44 listed in wl.mailspike.net]
X-Debbugs-Envelope-To: 61386
Cc: Paul Eggert <eggert@HIDDEN>, 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)
> On 2023-02-12, at 10:36 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 12/02/2023 14:15, P=C3=A1draig Brady wrote:
>> On 11/02/2023 21:53, Paul Eggert wrote:
>>> On 2023-02-10 17:21, George Valkov wrote:
>>>=20
>>>> remember to trace all code paths errors.
>>>=20
>>> Yes, I've been doing that. However, I don't use macOS and the two of =
us
>>> are debugging remotely where I write the code and you debug it but
>>> neither of us know how macOS works. Of course it would be much more
>>> efficient if we weren't operating with these obstacles, but here we =
are.
>>>=20
>>> Because the macOS behavior is poorly documented and we lack the =
source,
>>> if we can't figure this out fairly quickly coreutils should simply =
stop
>>> using fclonefileat because it's unreliable for users and a time sink =
for
>>> us. I'm hoping, though, we can get something working with another =
round
>>> or two.
>>>=20
>>>=20
>>>> With CLONE_ACL, I get 22 Invalid argument.
>>>=20
>>> OK, this means that although Apple has documented CLONE_ACL and it's =
in
>>> your include files, it doesn't work in your version of macOS, =
because
>>> either it's not supported by the kernel, or by the filesystem code, =
or
>>> by the particular filesystem instance, or for some other reason. =
EINVAL
>>> hints that it's the kernel but that's by no means certain.
>>>=20
>>> One possible way to defend against this macOS misbehavior is for cp =
to
>>> try to use CLONE_ACL, and if that fails with errno =3D=3D EINVAL try =
again
>>> without CLONE_ACL. I attempted to implement this in the attached =
patch;
>>> please give it a try.
>> I tested this on macOS 13.2 and the first fclonefileat(..., =
CLONE_ACL) succeeds.
>> I tested with various umask, --no-preserve=3Dmode, -p, ... =
combinations
>> and didn't find any inconsistencies in permissions with =
--reflink=3Dnever copies.
>> Actually scratch that. There was an older xcode installed so =
CLONE_ACL was 0.
>> I've not got access at the moment, so will need to retest later with =
newer xcode.
>=20
> Good news is that with newer xcode supporting macOS 13.1
> CLONE_ACL is set and that all works as expected, with the
> first fclonefileat() succeeding.
I can confirm that. CLONE_ACL is 4 when building on macOS 12.6.3. I rand =
the compiled sample on the recovery environment, which is equivalent to =
macOS 13.3. The flag is supported, though I did not observe any =
difference with or without it. The sample was ran as root, while the =
source and destination were set to g:staff 600 @.
> I checked with actual ACLs (set with `chmod +a "user:$USER allow =
readattr" file`),
> and they were copied or dropped as expected. Note ls -l on macOS will =
list
> ACLs with a "+" rather than a "@", as was mentioned elsewhere in this =
thread.
@ is shown on files that have extended attributes. I can remove them =
with
xattr -cr . # recursive
xattr -cr file # single item
+ means ACL
ll -e -@
drwx------@ 3 g staff 96 May 16 2021 Applications/
com.apple.quarantine 21
drwx------+ 8 g staff 256 May 16 2021 Music/
0: group:everyone deny delete
>> One divergence with fclonefileat() is that it writes through a =
dangling symlink,
>> where a standard --reflink=3Dnever copy will fail.
>> I.e. `make TESTS=3Dtests/cp/thru-dangling.sh SUBDIRS=3D. check` =
fails.
>> Now there is nothing wrong with writing through the dangling symlink,
>> and in fact standard cp will also with POSIXLY_CORRECT set in the =
environment.
>> So I may just need need to adjust the test to work / skip in this =
case.
The same test fails on macOS 12.6.3. See test-01*.txt.
https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/
I ran your test, followed by make check-TESTS for both original (8 =
failed) and patched (12 filed).
> Another divergence I noticed is wrt umask and setuid handling.
> The following transcript shows that with fclonefileat() we do _not_
> preserve setuid when it is preserved without.
> Also with fclonefileat() the umask is not honored,
> while it is honored without.
>=20
> # id -u
> 0
>=20
> # echo create setuid file
> # rm src/cp.s; src/cp src/cp src/cp.s; chmod u+s src/cp.s
>=20
> # umask; for reflink in always never; do
> > for preserve in mode timestamps; do
> > rm -f src/cp.s.$reflink.$preserve
> > src/cp --reflink=3D$reflink --preserve=3D$preserve src/cp.s =
src/cp.s.$reflink.$preserve
> > done
> > done
> > ls -le src/cp.s*
> 0022
> -rwsrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s
> -rwxrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.always.mode
> -rwxrwxr-x 1 root staff 234032 12 Feb 20:21 =
src/cp.s.always.timestamps
> -rwsrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.never.mode
> -rwxr-xr-x 1 root staff 234032 12 Feb 20:21 =
src/cp.s.never.timestamps
The manpage is clonefile.2.txt, use the link above.
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 12 Feb 2023 20:36:36 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Feb 12 15:36:36 2023 Received: from localhost ([127.0.0.1]:46924 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pRJ5P-0004EE-M6 for submit <at> debbugs.gnu.org; Sun, 12 Feb 2023 15:36:36 -0500 Received: from mail-wm1-f49.google.com ([209.85.128.49]:53065) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pRJ5N-0004Dv-CD for 61386 <at> debbugs.gnu.org; Sun, 12 Feb 2023 15:36:34 -0500 Received: by mail-wm1-f49.google.com with SMTP id z13so7417025wmp.2 for <61386 <at> debbugs.gnu.org>; Sun, 12 Feb 2023 12:36:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=1cByYBDMDl+P8QtmT88yeVpH/+TnaLn01NeKdr6KZ2k=; b=nGL3jq1mb3qQSMatK6wAPUvuo7lcgwleD5XApOa316Vg8XOv/97Gk2euasEBS8gPeG NWA7COtkkzRML/xQL72rSdeMevjDHCW/uwCMGb1f2SvYqIOc4/+wFPrfkrsREq7lj4YJ emG78w0ReSXbAKmz+oM8ylwA5/nIQZwqSwX3V1NmsBiyfu4mRhB1ER6Ffjx1v/wVrtaI DpnDSP2rLHED+EewtGkLUXTPVVbYRUVEADQ/B6llhbFbE74XILlZDU7Hz7xhgLPx8Bci bc8LS25FbMf/juOhlY07M+C9ADrZ0C/uHHVf+65B4ZhzL2qNwfKrrsa3kXvJL4PTcmMr XtcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1cByYBDMDl+P8QtmT88yeVpH/+TnaLn01NeKdr6KZ2k=; b=t4s0Vlb2r78Geyb2ZMe4ir0m3TCyUQoFu7MbkNjOIRj0HH1BqeyrG3t5iHHafvEykv M7vO3lSh6jbBlvpToKqRE04pbrdBENmQj3viwK6hYak7R8J14OG7adYh8+Ah44t6tu8u SlDFVZ9QBhb0X94oNVEg4iFUrPsY6AbnBJnqe8mmmymvgzcfk8EE4ZxGJ96SW0HXaV+6 VsveGjoai5YbAfZc7tz02y0YzI/MlkurkOWUA6jR2VGoaiEpZ8SxSCYvGzg/LKVHJTcT swswgTSa1j/K5/4Gx+ymtZbFCwzR+rHEaU8AKx5GwG18fRB3O69lFBzZ8DMog3VPTuDa VAlg== X-Gm-Message-State: AO0yUKXXolbKS/05k1npfNIfzohWajRQmv/25wlRtLCrFEIkIviNrukd jioia/+ot40CNehS5gHxx1g= X-Google-Smtp-Source: AK7set8D8tBQeDtYQQtjOwvLkIMbbv+GZpCcaRpSeeeMlsB1dVzq1bROTF8/5g/FNSs2PAgGRUfjTQ== X-Received: by 2002:a05:600c:747:b0:3dc:438a:c381 with SMTP id j7-20020a05600c074700b003dc438ac381mr16810304wmn.28.1676234186334; Sun, 12 Feb 2023 12:36:26 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id n7-20020a5d4207000000b002c3ce97ec38sm8995987wrq.115.2023.02.12.12.36.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 12 Feb 2023 12:36:25 -0800 (PST) Message-ID: <895887d1-1196-c45d-b522-0f3545a4e9f4@HIDDEN> Date: Sun, 12 Feb 2023 20:36:24 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> In-Reply-To: <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 12/02/2023 14:15, Pádraig Brady wrote: > On 11/02/2023 21:53, Paul Eggert wrote: >> On 2023-02-10 17:21, George Valkov wrote: >> >>> remember to trace all code paths errors. >> >> Yes, I've been doing that. However, I don't use macOS and the two of us >> are debugging remotely where I write the code and you debug it but >> neither of us know how macOS works. Of course it would be much more >> efficient if we weren't operating with these obstacles, but here we are. >> >> Because the macOS behavior is poorly documented and we lack the source, >> if we can't figure this out fairly quickly coreutils should simply stop >> using fclonefileat because it's unreliable for users and a time sink for >> us. I'm hoping, though, we can get something working with another round >> or two. >> >> >>> With CLONE_ACL, I get 22 Invalid argument. >> >> OK, this means that although Apple has documented CLONE_ACL and it's in >> your include files, it doesn't work in your version of macOS, because >> either it's not supported by the kernel, or by the filesystem code, or >> by the particular filesystem instance, or for some other reason. EINVAL >> hints that it's the kernel but that's by no means certain. >> >> One possible way to defend against this macOS misbehavior is for cp to >> try to use CLONE_ACL, and if that fails with errno == EINVAL try again >> without CLONE_ACL. I attempted to implement this in the attached patch; >> please give it a try. > > I tested this on macOS 13.2 and the first fclonefileat(..., CLONE_ACL) succeeds. > I tested with various umask, --no-preserve=mode, -p, ... combinations > and didn't find any inconsistencies in permissions with --reflink=never copies. > Actually scratch that. There was an older xcode installed so CLONE_ACL was 0. > I've not got access at the moment, so will need to retest later with newer xcode. Good news is that with newer xcode supporting macOS 13.1 CLONE_ACL is set and that all works as expected, with the first fclonefileat() succeeding. I checked with actual ACLs (set with `chmod +a "user:$USER allow readattr" file`), and they were copied or dropped as expected. Note ls -l on macOS will list ACLs with a "+" rather than a "@", as was mentioned elsewhere in this thread. > One divergence with fclonefileat() is that it writes through a dangling symlink, > where a standard --reflink=never copy will fail. > I.e. `make TESTS=tests/cp/thru-dangling.sh SUBDIRS=. check` fails. > Now there is nothing wrong with writing through the dangling symlink, > and in fact standard cp will also with POSIXLY_CORRECT set in the environment. > So I may just need need to adjust the test to work / skip in this case. Another divergence I noticed is wrt umask and setuid handling. The following transcript shows that with fclonefileat() we do _not_ preserve setuid when it is preserved without. Also with fclonefileat() the umask is not honored, while it is honored without. # id -u 0 # echo create setuid file # rm src/cp.s; src/cp src/cp src/cp.s; chmod u+s src/cp.s # umask; for reflink in always never; do > for preserve in mode timestamps; do > rm -f src/cp.s.$reflink.$preserve > src/cp --reflink=$reflink --preserve=$preserve src/cp.s src/cp.s.$reflink.$preserve > done > done > ls -le src/cp.s* 0022 -rwsrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s -rwxrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.always.mode -rwxrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.always.timestamps -rwsrwxr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.never.mode -rwxr-xr-x 1 root staff 234032 12 Feb 20:21 src/cp.s.never.timestamps cheers, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 12 Feb 2023 14:15:55 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Feb 12 09:15:55 2023 Received: from localhost ([127.0.0.1]:44413 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pRD90-0005UA-R5 for submit <at> debbugs.gnu.org; Sun, 12 Feb 2023 09:15:55 -0500 Received: from mail-wr1-f53.google.com ([209.85.221.53]:37534) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pRD8z-0005Tt-9M for 61386 <at> debbugs.gnu.org; Sun, 12 Feb 2023 09:15:54 -0500 Received: by mail-wr1-f53.google.com with SMTP id m10so871732wrn.4 for <61386 <at> debbugs.gnu.org>; Sun, 12 Feb 2023 06:15:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=6+bRXY6l//Tn8HOAQDwdm++cFzbw79483CXGjG+8AQs=; b=IXz7TGWjB+J016D9iuwT3pPlnxeEglizttPodQI8Zz48KYs97CAtSrY18SqZs+zTfx MyNH2enx0gRtOX5WwCsWbkyO+NLZU4NCvfSu/yAZc4slFX+oBEnFCW94i30mhawO3Htl 1sx3FeWVZPIXrFlHxeyDYBFTkO4lm9YQF6jspymUFCXCZbpM+qNxq+atrXzQdf1h6Eov PQrJ7pxUjE2yAuEL23JifFAJU94bEC07r971XYzWL5Ir9vIsxiX9M2XNw7t0WGhd33Yc XNYFuxZCc3rKmxcjDpPCSm7TpDG+1y0y4Vn4//eG8sMrSApk7mTsDpPyUFMU08pQhNdA L/RQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6+bRXY6l//Tn8HOAQDwdm++cFzbw79483CXGjG+8AQs=; b=3nO9FkefPlC6rpANr2yXqKgiRsaOFrEahTnPQEmXCr3i1vnNj0PRPZA8NGfR2Jp6Xb FAjdd6CckdZbvxONefNrkJVn+XGxMnL4+Mp6V3l3pJCiZs37ct4d0ELKpKqx+cLtoLAh vu7PFrcrEHP1E4Lb5ZwhCFqcFq5smY9aSkiNuNAa9mueSp4/Kl315ENZZablgfq5vvmc 6Y2Limfc6Pjo2W37uZHDXj/qZCXeT+vHDrDPOaBxUwQqilCIuQdqKx/5u5YjM3BtImvX TwevbSDvGDC46KOqeXwAQAPU4/5obPCwGhZAn1jVxGjZkA4e2FYfSKXQyMxNP5mAYZ4I pOVQ== X-Gm-Message-State: AO0yUKWoCljhXznHlBqn1u5qitnejmv5X1xv3SKg2Xoy+2qiwvfsJDs3 wvI5zBwMtBec0+L70WNc6G0= X-Google-Smtp-Source: AK7set+Y5lr4U6ZniaXXT1oDyNiPt9dUX2/uAvtKlnMB6r4tGtJfJx/gcWkmGoOmWfs3/19mSsy/+A== X-Received: by 2002:a05:6000:1c7:b0:2c5:585d:74c6 with SMTP id t7-20020a05600001c700b002c5585d74c6mr928723wrx.12.1676211346997; Sun, 12 Feb 2023 06:15:46 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id g17-20020a056000119100b002c54e26bca5sm3889779wrx.49.2023.02.12.06.15.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 12 Feb 2023 06:15:46 -0800 (PST) Message-ID: <375fb1b4-3b71-d6e4-4c44-aa2d7e588171@HIDDEN> Date: Sun, 12 Feb 2023 14:15:45 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 11/02/2023 21:53, Paul Eggert wrote: > On 2023-02-10 17:21, George Valkov wrote: > >> remember to trace all code paths errors. > > Yes, I've been doing that. However, I don't use macOS and the two of us > are debugging remotely where I write the code and you debug it but > neither of us know how macOS works. Of course it would be much more > efficient if we weren't operating with these obstacles, but here we are. > > Because the macOS behavior is poorly documented and we lack the source, > if we can't figure this out fairly quickly coreutils should simply stop > using fclonefileat because it's unreliable for users and a time sink for > us. I'm hoping, though, we can get something working with another round > or two. > > >> With CLONE_ACL, I get 22 Invalid argument. > > OK, this means that although Apple has documented CLONE_ACL and it's in > your include files, it doesn't work in your version of macOS, because > either it's not supported by the kernel, or by the filesystem code, or > by the particular filesystem instance, or for some other reason. EINVAL > hints that it's the kernel but that's by no means certain. > > One possible way to defend against this macOS misbehavior is for cp to > try to use CLONE_ACL, and if that fails with errno == EINVAL try again > without CLONE_ACL. I attempted to implement this in the attached patch; > please give it a try. I tested this on macOS 13.2 and the first fclonefileat(..., CLONE_ACL) succeeds. I tested with various umask, --no-preserve=mode, -p, ... combinations and didn't find any inconsistencies in permissions with --reflink=never copies. Actually scratch that. There was an older xcode installed so CLONE_ACL was 0. I've not got access at the moment, so will need to retest later with newer xcode. One divergence with fclonefileat() is that it writes through a dangling symlink, where a standard --reflink=never copy will fail. I.e. `make TESTS=tests/cp/thru-dangling.sh SUBDIRS=. check` fails. Now there is nothing wrong with writing through the dangling symlink, and in fact standard cp will also with POSIXLY_CORRECT set in the environment. So I may just need need to adjust the test to work / skip in this case. cheers, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 12 Feb 2023 02:18:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 11 21:18:28 2023
Received: from localhost ([127.0.0.1]:43643 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pR1wi-0003Xv-2C
for submit <at> debbugs.gnu.org; Sat, 11 Feb 2023 21:18:28 -0500
Received: from mail-wm1-f52.google.com ([209.85.128.52]:36523)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pR1wg-0003Xg-0Z
for 61386 <at> debbugs.gnu.org; Sat, 11 Feb 2023 21:18:27 -0500
Received: by mail-wm1-f52.google.com with SMTP id
c4-20020a1c3504000000b003d9e2f72093so8928841wma.1
for <61386 <at> debbugs.gnu.org>; Sat, 11 Feb 2023 18:18:25 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=BZ9WygIa8xlvxnRwidsbiDCDcEHE7z22ujBWmprWo2A=;
b=N8+LtW0TiUyt79RWHExIBkEg01IAueTIhIfle0XMjLrJHD2yH3VZMTj1cPwIDjJdd2
pKyXengi7cXgs1hsKyBWYZS0z6qj2a+NxEmBQw437HM3st9XB4DXViVwhjTPytkEGCuZ
sguTzU278fE5DywYr/+CzAeQ1NfA0nL3+9tefe7St3hPpvZgD4Xf2jqHZX9c/QzRVOOi
LUeKClcccPX/35Zt5o5fCm3q7ch5a+x++9LXSJrhG+toSgzo9CHX+QMjDUULwkmijrKm
mbU+IKzuh8yTre6TSZHVLwdnqZMhblNkj9U7ieRnvKJfDSklPreOPr3+rofChpuDZIQU
BQoQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=BZ9WygIa8xlvxnRwidsbiDCDcEHE7z22ujBWmprWo2A=;
b=hhtPWLIXqcyUdCFpCNQ+u/VPeh4WVLJ4lqxFOCM+GLu8bo5Ox2pgfkoGUkTGNEP9qG
gb+mQjo74lsDEm0d9kp5Vy/U+kG3Mn1P7NAkMIjDVX5H6hq02r4uoHP2UMZMQZ+wZJ9j
02DKJTB1NuMCQ0WgbzeL00YolddykcbKKBnTShlp8IMsjjSWwLGlZoM8Em/9VEBwdjoQ
TEeS4MSCIUjCYwLlteOHFAk/XZ0Hyie273p9DAFcFABTWBIfKthGmWIM9WLW1caXZw7Z
x/V7j+X5sddK112+Cgnsjpr+Rnxqmd7oD/YxM9RvkfTeRpH2b2Uv8srm/IZI2Cq8zraj
6O0g==
X-Gm-Message-State: AO0yUKVoD04t51rudUrKMuDcnJ6w5I0q9mPQ88znxRGo2QzTy3RFpP5U
4QQgpff6Cub/w6nfiT5OFqo=
X-Google-Smtp-Source: AK7set+q0qnnlHv2M3Bm0dGxLWMgOweCEVFMOOMvvSikQd2CaB4HW6AJ1GcT2VAnNCRPPBuehF7KHg==
X-Received: by 2002:a05:600c:3ca3:b0:3dc:5ae4:c13d with SMTP id
bg35-20020a05600c3ca300b003dc5ae4c13dmr21226281wmb.4.1676168300116;
Sat, 11 Feb 2023 18:18:20 -0800 (PST)
Received: from smtpclient.apple (212-39-89-217.ip.btc-net.bg. [212.39.89.217])
by smtp.gmail.com with ESMTPSA id
f8-20020a05600c4e8800b003da28dfdedcsm10750492wmq.5.2023.02.11.18.18.18
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Sat, 11 Feb 2023 18:18:19 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <64563c0c-7654-3eee-abd0-1f30f4e1d119@HIDDEN>
Date: Sun, 12 Feb 2023 04:18:17 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <8316F2E5-341D-4392-A29D-7B188C050026@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
<0F5204BA-5080-4F8C-BFDD-44E72282C444@HIDDEN>
<64563c0c-7654-3eee-abd0-1f30f4e1d119@HIDDEN>
To: Paul Eggert <eggert@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
> On 2023-02-12, at 2:47 AM, Paul Eggert <eggert@HIDDEN> wrote:
>=20
> On 2023-02-11 16:38, George Valkov wrote:
>> This might help:
>> https://github.com/apple/darwin-xnu/blob/main/bsd/sys/clonefile.h
>=20
> It doesn't help, because it doesn't mention CLONE_ACL.
Here is what I found: The version of vfs_syscalls.c on that repository =
is 3 years old and does not support CLONE_ACL. Still it should provide a =
good idea about the implementation before this flags was introduced.
=
https://github.com/apple/darwin-xnu/blob/2ff845c2e033bd0ff64b5b6aa6063a1f8=
f65aa32/bsd/vfs/vfs_syscalls.c#L8201
if (uap->flags & ~(CLONE_NOFOLLOW | CLONE_NOOWNERCOPY)) {
return EINVAL;
}
I was able to run my sample on macOS 13 recovery environment. The =
CLONE_ACL flag is supported there. I don=E2=80=99t see any difference in =
the final result with or without the flag. Both clones have UNIX =
permissions, extended attributes and time stamp from the source.
Darwin gMac.lan 22.3.0 Darwin Kernel Version 22.3.0: Thu Jan 5 20:53:49 =
PST 2023; root:xnu-8792.81.2~2/RELEASE_X86_64 x86_64
fd 3 dir 4
fclonefileat 0 0 Undefined error: 0
fclonefileat 0 0 Undefined error: 0 CLONE_ACL
-rw-------@ 1 501 staff 553 12 Feb 00:50 A
-rw-------@ 1 501 staff 553 12 Feb 00:50 B
-rw-------@ 1 501 staff 553 12 Feb 00:50 CLONE_ACL
I tried running cp with your patch there, but it depends on a dynamic =
library and fails to run. My attempt to use chroot failed, probably due =
to file signatures: Killed 9.
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 12 Feb 2023 00:47:36 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 11 19:47:36 2023 Received: from localhost ([127.0.0.1]:43581 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pR0Wl-0001CV-Tu for submit <at> debbugs.gnu.org; Sat, 11 Feb 2023 19:47:36 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:33926) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pR0Wi-0001BO-AI for 61386 <at> debbugs.gnu.org; Sat, 11 Feb 2023 19:47:33 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 685A3160092; Sat, 11 Feb 2023 16:47:26 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id PsRz15ztnipi; Sat, 11 Feb 2023 16:47:25 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 7D52716009F; Sat, 11 Feb 2023 16:47:25 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 7D52716009F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676162845; bh=RRr8CcdC9KkhcXWM917eGMrpYrwi1+BsXuNyt8d11+o=; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type: Content-Transfer-Encoding; b=beiTzRJNowGF2dRyOT7fWfyFIi5bJ05oXo6bB56bAAhoPXDaUCnNgJ4jxSAAe0wgp OkBNpRiPMB8/nhaAwJrUr3/uy+PMd6iY6lWu0WTpOjj+aVbI2KzNFwcn7pZcwD3GAz jPEBFv3KUt757gOSgVEQxg0TioIQeNxbCFZ9hKKE= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id mzqagcPAV-Uc; Sat, 11 Feb 2023 16:47:25 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 42D5B160092; Sat, 11 Feb 2023 16:47:25 -0800 (PST) Message-ID: <64563c0c-7654-3eee-abd0-1f30f4e1d119@HIDDEN> Date: Sat, 11 Feb 2023 16:47:24 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> <0F5204BA-5080-4F8C-BFDD-44E72282C444@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <0F5204BA-5080-4F8C-BFDD-44E72282C444@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) On 2023-02-11 16:38, George Valkov wrote: > This might help: > https://github.com/apple/darwin-xnu/blob/main/bsd/sys/clonefile.h It doesn't help, because it doesn't mention CLONE_ACL. > The kernel should be open source. No, the macOS kernel is not free software. Nor is iOS's kernel.
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 12 Feb 2023 00:38:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 11 19:38:49 2023
Received: from localhost ([127.0.0.1]:43564 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pR0OH-0000xw-AL
for submit <at> debbugs.gnu.org; Sat, 11 Feb 2023 19:38:49 -0500
Received: from mail-wr1-f45.google.com ([209.85.221.45]:39437)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pR0OF-0000xi-PS
for 61386 <at> debbugs.gnu.org; Sat, 11 Feb 2023 19:38:48 -0500
Received: by mail-wr1-f45.google.com with SMTP id a2so8916869wrd.6
for <61386 <at> debbugs.gnu.org>; Sat, 11 Feb 2023 16:38:47 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=XEwtQLLXmrKHT2WrCZQNlT8YSXxZr6dieIxbX70tQcI=;
b=mSv0CaNvsWY9OCIjsPuMlb8nire9T9zNxspicECvVq4BFHw9wboPtS/aZpQ5ujPPKz
PMAmDTP35aUje8/bUUBk2KD8wsbIXo/g6IkAYk0FMLVW+sxpcMCnwQgU4UzAuGMLpXt4
jv0Cqaisz+PAK/ppKNRhFwfXu7n5VWTGiEWwLdzJzoXKWuVIEN4Ag47zm6ds2Il1zrUa
kPmWP41QMh2mxrLVrwDcLeHs7Ded2G4xzvRbv8nz8FDJaHnw41hREQtWlGGbNgzqi/KN
Cli3jLM1G4ihItolVmkDEyNsb73OjfniAJq/uIjfv61l2I01tU2dEps9wi4mK8j0Llyh
ijZg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=XEwtQLLXmrKHT2WrCZQNlT8YSXxZr6dieIxbX70tQcI=;
b=u44kIhN3Mpf3PJa16Hl+aNIWNhAsgxIsHSCAm/BVAMv7h47Bt1x2BiXDetIDlfmsyx
d3FQs0PDKim5elOD1P1L4Db6RHMtngV+vlKccnOio7X95h7QEJNfN2H24OT+/yDuh0U6
iHHJHfW+7c77nd8ceOfXDmV+DSAOIKAj66PzROFdqPmvXSjJiDXqJ1K0QfPdKt9d2v7q
7MKxfHQDigCcvySc2EcgZrVpr+wlA/9Md5VF4Itqup1MbMuOZfq06moihYb++x6PmiAh
dzMSK5tILqOYIJtvl2fz/jPX2fqR9FNyIpHHbYjViB4m2JnEWO3xCRpcQuVWsfJrvQzj
cVUQ==
X-Gm-Message-State: AO0yUKXr0dsVijlAlm1ApvwAldLASUbjdNHh3hFMayp/KuAm4X7scHmt
mFfjZbJo/9jVyVNAAoIJGVE=
X-Google-Smtp-Source: AK7set8b6kfJcDIh61ouWHm7kw0AtJokI7MwM92lza9FXU6J30qVZcCa8dz/zY4egpQgRZk/yQekoA==
X-Received: by 2002:a5d:5589:0:b0:2c5:557f:93b8 with SMTP id
i9-20020a5d5589000000b002c5557f93b8mr132466wrv.7.1676162321532;
Sat, 11 Feb 2023 16:38:41 -0800 (PST)
Received: from smtpclient.apple (212-39-89-217.ip.btc-net.bg. [212.39.89.217])
by smtp.gmail.com with ESMTPSA id
m9-20020adfe0c9000000b002c3d29d83d2sm7199420wri.63.2023.02.11.16.38.39
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Sat, 11 Feb 2023 16:38:40 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
Date: Sun, 12 Feb 2023 02:38:39 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <0F5204BA-5080-4F8C-BFDD-44E72282C444@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
<82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN>
To: Paul Eggert <eggert@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
> On 2023-02-11, at 11:53 PM, Paul Eggert <eggert@HIDDEN> wrote:
>=20
> On 2023-02-10 17:21, George Valkov wrote:
>=20
>> remember to trace all code paths errors.
>=20
> Yes, I've been doing that. However, I don't use macOS and the two of =
us are debugging remotely where I write the code and you debug it but =
neither of us know how macOS works. Of course it would be much more =
efficient if we weren't operating with these obstacles, but here we are.
Yes, it=E2=80=99s true. It takes a lot of energy from both of us. =
Especially the time to document observations in hope to be helpful. =
There is a saying that two people working together is more than each on =
their own.
Let=E2=80=99s work together! I have a good experience with mutiplatform =
development: macOS, Linux, BSD, Windows. For the most part it=E2=80=99s =
the same as Linux with minor differences, so pretty easy to support =
according to my experience. Build environment and debugging tools are =
all set. That will keep you comfortable and productive. Just drop me a =
private mail and we can set a conference call. Then test for standard =
compliance according to manpage.
> Because the macOS behavior is poorly documented and we lack the =
source, if we can't figure this out fairly quickly coreutils should =
simply stop using fclonefileat because it's unreliable for users and a =
time sink for us. I'm hoping, though, we can get something working with =
another round or two.
Sad but true. This might help:
https://github.com/apple/darwin-xnu/blob/main/bsd/sys/clonefile.h
The kernel should be open source. We might be able to find the =
implementation there.
I hope we can keep fclonefileat.
>> With CLONE_ACL, I get 22 Invalid argument.
>=20
> OK, this means that although Apple has documented CLONE_ACL and it's =
in your include files, it doesn't work in your version of macOS, because =
either it's not supported by the kernel, or by the filesystem code, or =
by the particular filesystem instance, or for some other reason. EINVAL =
hints that it's the kernel but that's by no means certain.
>=20
> One possible way to defend against this macOS misbehavior is for cp to =
try to use CLONE_ACL, and if that fails with errno =3D=3D EINVAL try =
again without CLONE_ACL. I attempted to implement this in the attached =
patch; please give it a try.
Good approach. Tour patch confirms it works. copy.c says it has been =
introduced in 12.6, I=E2=80=99m using 12.6.3. That suggests it is =
available in headers, but not available in the kernel yet. I will check =
if it is implemented in the recovery environment for macOS 13.
I added some code for tracing. The check works as intended.
int s =3D fclonefileat (source_desc, dst_dirfd, dst_relname, fc_flags);
int a =3D s ? errno : 0;
if (s !=3D 0 && (fc_flags & CLONE_ACL) !=3D 0 && errno =3D=3D EINVAL)
{
printf("s %2i errno %2i %s fc_flags %x !!\n", s, a, strerror(a), =
fc_flags);
fc_flags &=3D ~CLONE_ACL;
s =3D fclonefileat (source_desc, dst_dirfd, dst_relname, fc_flags);
a =3D s ? errno : 0;
}
printf("s %2i errno %2i %s fc_flags %x\n", s, a, strerror(a), =
fc_flags);
./coreutils-clone/src/cp A B
s 0 errno 0 Success fc_flags 2
./coreutils-clone/src/cp -p A C
s -1 errno 22 Invalid argument fc_flags 4 !!
s 0 errno 0 Success fc_flags 0
Extended attributes are preserved in both copies: note the @ flag.
-rw-------@ 1 g staff 415 Feb 11 02:10 A
-rw-------@ 1 g staff 415 Feb 12 01:30 B
-rw-------@ 1 g staff 415 Feb 11 02:10 C
Good luck, Paul!
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 11 Feb 2023 21:53:56 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 11 16:53:56 2023 Received: from localhost ([127.0.0.1]:43452 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQxoi-000599-1F for submit <at> debbugs.gnu.org; Sat, 11 Feb 2023 16:53:56 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:50538) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pQxoe-00058q-AU for 61386 <at> debbugs.gnu.org; Sat, 11 Feb 2023 16:53:54 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id D531F16009F; Sat, 11 Feb 2023 13:53:45 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id R1kVQAvGReAa; Sat, 11 Feb 2023 13:53:44 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 9682B160092; Sat, 11 Feb 2023 13:53:44 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 9682B160092 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676152424; bh=xTbrXbpnIU2Eqderv0+Xujqvbr3yyMtFzI3jSVGnBLU=; h=Content-Type:Message-ID:Date:MIME-Version:To:From:Subject; b=p99RaDKV+FyKj/9XYHbdzfJhFtoxp3APkdTkh1RIR1dMmEqaSfq3DX3BikQwmXfOu ajdWVTUn5YduhJ0vgy41UJIrlvsQ0xAQON2ClkQcomyw8P0cYI+EX1jzuz8nKH4YpG fyLXVQvBQCB2ZuuGIimFSC7yKG0iI+O+znLiAcL8= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Qqi01ycsxJDZ; Sat, 11 Feb 2023 13:53:44 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 64C721600A5; Sat, 11 Feb 2023 13:53:44 -0800 (PST) Content-Type: multipart/mixed; boundary="------------YNti50GSltfWBNOip0mPvYNd" Message-ID: <82620d01-3dc1-b038-43a2-4a97bca5c482@HIDDEN> Date: Sat, 11 Feb 2023 13:53:43 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Content-Language: en-US To: George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS In-Reply-To: <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN> X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) This is a multi-part message in MIME format. --------------YNti50GSltfWBNOip0mPvYNd Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2023-02-10 17:21, George Valkov wrote: > remember to trace all code paths errors. Yes, I've been doing that. However, I don't use macOS and the two of us are debugging remotely where I write the code and you debug it but neither of us know how macOS works. Of course it would be much more efficient if we weren't operating with these obstacles, but here we are. Because the macOS behavior is poorly documented and we lack the source, if we can't figure this out fairly quickly coreutils should simply stop using fclonefileat because it's unreliable for users and a time sink for us. I'm hoping, though, we can get something working with another round or two. > With CLONE_ACL, I get 22 Invalid argument. OK, this means that although Apple has documented CLONE_ACL and it's in your include files, it doesn't work in your version of macOS, because either it's not supported by the kernel, or by the filesystem code, or by the particular filesystem instance, or for some other reason. EINVAL hints that it's the kernel but that's by no means certain. One possible way to defend against this macOS misbehavior is for cp to try to use CLONE_ACL, and if that fails with errno == EINVAL try again without CLONE_ACL. I attempted to implement this in the attached patch; please give it a try. > do we want to go the quick route of unlinking the destination and creating a fresh clone? No, POSIX is clear that plain cp should not unlink and re-create the destination. Of course we can add a non-POSIX flag to do that, and we already have: cp --remove-destination. --------------YNti50GSltfWBNOip0mPvYNd Content-Type: text/x-patch; charset=UTF-8; name="0001-cp-improve-use-of-fclonefileat.patch" Content-Disposition: attachment; filename="0001-cp-improve-use-of-fclonefileat.patch" Content-Transfer-Encoding: base64 RnJvbSBlYTg5MDQ2MDZlOGU3NDllMmY2ZDIyYzFhYmIwMzEwZmFjMzcyMmY1IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBGcmksIDEwIEZlYiAyMDIzIDEzOjM0OjU0IC0wODAwClN1YmplY3Q6IFtQQVRD SF0gY3A6IGltcHJvdmUgdXNlIG9mIGZjbG9uZWZpbGVhdAoKKiBzcmMvY29weS5jIChjb3B5 X3JlZyk6IFVzZSBDTE9ORV9BQ0wgaWYgYXZhaWxhYmxlIGFuZCB3b3JraW5nLgpJZiB0aGUg b25seSBwcm9ibGVtIHdpdGggZmNsb25lZmlsZWF0IGlzIHRoYXQgaXQgd291bGQgY3JlYXRl IHRoZQpmaWxlIHdpdGggdGhlIHdyb25nIHRpbWVzdGFtcCwgb3Igd2l0aCB0b28gZmV3IHBl cm1pc3Npb25zLApkbyB0aGF0IGJ1dCBmaXggdGhlIHRpbWVzdGFtcCBhbmQgcGVybWlzc2lv bnMgYWZ0ZXJ3YXJkcywKcmF0aGVyIHRoYW4gZmFsbGluZyBiYWNrIG9uIGEgdHJhZGl0aW9u YWwgY29weS4KKENMT05FX0FDTCkgW0hBVkVfRkNMT05FRklMRUFUICYmICFVU0VfWEFUVFJd OiBEZWZhdWx0IHRvIDAuCi0tLQogc3JjL2NvcHkuYyB8IDUyICsrKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCA0 NCBpbnNlcnRpb25zKCspLCA4IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3NyYy9jb3B5 LmMgYi9zcmMvY29weS5jCmluZGV4IGRmYmI1NTdkZS4uNzgyZmFiOThkIDEwMDY0NAotLS0g YS9zcmMvY29weS5jCisrKyBiL3NyYy9jb3B5LmMKQEAgLTExNDYsNiArMTE0Niw3IEBAIGNv cHlfcmVnIChjaGFyIGNvbnN0ICpzcmNfbmFtZSwgY2hhciBjb25zdCAqZHN0X25hbWUsCiAg IHVuaW9uIHNjYW5faW5mZXJlbmNlIHNjYW5faW5mZXJlbmNlOwogICBib29sIHJldHVybl92 YWwgPSB0cnVlOwogICBib29sIGRhdGFfY29weV9yZXF1aXJlZCA9IHgtPmRhdGFfY29weV9y ZXF1aXJlZDsKKyAgYm9vbCBtb2RlX2FscmVhZHlfcHJlc2VydmVkID0gZmFsc2U7CiAgIGJv b2wgcHJlc2VydmVfeGF0dHIgPSBVU0VfWEFUVFIgJiB4LT5wcmVzZXJ2ZV94YXR0cjsKIAog ICBzb3VyY2VfZGVzYyA9IG9wZW4gKHNyY19uYW1lLApAQCAtMTI0NCwxNyArMTI0NSw0NyBA QCBjb3B5X3JlZyAoY2hhciBjb25zdCAqc3JjX25hbWUsIGNoYXIgY29uc3QgKmRzdF9uYW1l LAogICBpZiAoKm5ld19kc3QpCiAgICAgewogI2lmIEhBVkVfRkNMT05FRklMRUFUICYmICFV U0VfWEFUVFIKLS8qIENMT05FX05PT1dORVJDT1BZIG9ubHkgYXZhaWxhYmxlIG9uIG1hY29z ID49IDEwLjEzLiAgKi8KKyMgaWZuZGVmIENMT05FX0FDTAorIyAgZGVmaW5lIENMT05FX0FD TCAwIC8qIEFkZGVkIGluIG1hY09TIDEyLjYuICAqLworIyBlbmRpZgogIyBpZm5kZWYgQ0xP TkVfTk9PV05FUkNPUFkKLSMgIGRlZmluZSBDTE9ORV9OT09XTkVSQ09QWSAwCisjICBkZWZp bmUgQ0xPTkVfTk9PV05FUkNPUFkgMCAvKiBBZGRlZCBpbiBtYWNPUyAxMC4xMy4gICovCiAj IGVuZGlmCi0gICAgICBpbnQgZmNfZmxhZ3MgPSB4LT5wcmVzZXJ2ZV9vd25lcnNoaXAgPyAw IDogQ0xPTkVfTk9PV05FUkNPUFk7CisgICAgICBtb2RlX3QgY2xvbmVkX21vZGVfYml0cyA9 IFNfSVNWVFggfCBTX0lSV1hVR087CisgICAgICBtb2RlX3QgY2xvbmVkX21vZGUgPSBzcmNf bW9kZSAmIGNsb25lZF9tb2RlX2JpdHM7CisgICAgICBpbnQgZmNfZmxhZ3MgPSAoKHgtPnBy ZXNlcnZlX21vZGUgPyBDTE9ORV9BQ0wgOiAwKQorICAgICAgICAgICAgICAgICAgICAgIHwg KHgtPnByZXNlcnZlX293bmVyc2hpcCA/IDAgOiBDTE9ORV9OT09XTkVSQ09QWSkpOwogICAg ICAgaWYgKGRhdGFfY29weV9yZXF1aXJlZCAmJiB4LT5yZWZsaW5rX21vZGUKLSAgICAgICAg ICAmJiB4LT5wcmVzZXJ2ZV9tb2RlICYmIHgtPnByZXNlcnZlX3RpbWVzdGFtcHMKKyAgICAg ICAgICAmJiAoeC0+cHJlc2VydmVfbW9kZSB8fCAhIChjbG9uZWRfbW9kZSAmIH5kc3RfbW9k ZSkpCiAgICAgICAgICAgJiYgKHgtPnByZXNlcnZlX293bmVyc2hpcCB8fCBDTE9ORV9OT09X TkVSQ09QWSkpCiAgICAgICAgIHsKLSAgICAgICAgICBpZiAoZmNsb25lZmlsZWF0IChzb3Vy Y2VfZGVzYywgZHN0X2RpcmZkLCBkc3RfcmVsbmFtZSwgZmNfZmxhZ3MpID09IDApCi0gICAg ICAgICAgICBnb3RvIGNsb3NlX3NyY19kZXNjOworICAgICAgICAgIGludCBzID0gZmNsb25l ZmlsZWF0IChzb3VyY2VfZGVzYywgZHN0X2RpcmZkLCBkc3RfcmVsbmFtZSwgZmNfZmxhZ3Mp OworICAgICAgICAgIGlmIChzICE9IDAgJiYgKGZjX2ZsYWdzICYgQ0xPTkVfQUNMKSAhPSAw ICYmIGVycm5vID09IEVJTlZBTCkKKyAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgZmNf ZmxhZ3MgJj0gfkNMT05FX0FDTDsKKyAgICAgICAgICAgICAgcyA9IGZjbG9uZWZpbGVhdCAo c291cmNlX2Rlc2MsIGRzdF9kaXJmZCwgZHN0X3JlbG5hbWUsIGZjX2ZsYWdzKTsKKyAgICAg ICAgICAgIH0KKyAgICAgICAgICBpZiAocyA9PSAwKQorICAgICAgICAgICAgeworICAgICAg ICAgICAgICBpZiAoIXgtPnByZXNlcnZlX3RpbWVzdGFtcHMpCisgICAgICAgICAgICAgICAg eworICAgICAgICAgICAgICAgICAgc3RydWN0IHRpbWVzcGVjIHRpbWVzcGVjWzJdOworICAg ICAgICAgICAgICAgICAgdGltZXNwZWNbMF0udHZfbnNlYyA9IHRpbWVzcGVjWzFdLnR2X25z ZWMgPSBVVElNRV9OT1c7CisgICAgICAgICAgICAgICAgICBpZiAodXRpbWVuc2F0IChkc3Rf ZGlyZmQsIGRzdF9yZWxuYW1lLCB0aW1lc3BlYywKKyAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIEFUX1NZTUxJTktfTk9GT0xMT1cpCisgICAgICAgICAgICAgICAgICAgICAg IT0gMCkKKyAgICAgICAgICAgICAgICAgICAgeworICAgICAgICAgICAgICAgICAgICAgIGVy cm9yICgwLCBlcnJubywgXygidXBkYXRpbmcgdGltZXMgZm9yICVzIiksCisgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIHF1b3RlYWYgKGRzdF9uYW1lKSk7CisgICAgICAgICAgICAg ICAgICAgICAgcmV0dXJuX3ZhbCA9IGZhbHNlOworICAgICAgICAgICAgICAgICAgICB9Cisg ICAgICAgICAgICAgICAgfQorICAgICAgICAgICAgICBtb2RlX2FscmVhZHlfcHJlc2VydmVk ID0gKGZjX2ZsYWdzICYgQ0xPTkVfQUNMKSAhPSAwOworICAgICAgICAgICAgICBkZXN0X2Rl c2MgPSAtMTsKKyAgICAgICAgICAgICAgb21pdHRlZF9wZXJtaXNzaW9ucyA9IGRzdF9tb2Rl ICYgfmNsb25lZF9tb2RlOworICAgICAgICAgICAgICBleHRyYV9wZXJtaXNzaW9ucyA9IDA7 CisgICAgICAgICAgICAgIGdvdG8gc2V0X2Rlc3RfbW9kZTsKKyAgICAgICAgICAgIH0KICAg ICAgICAgICBlbHNlIGlmICghIGhhbmRsZV9jbG9uZV9mYWlsIChkc3RfZGlyZmQsIGRzdF9y ZWxuYW1lLCBzcmNfbmFtZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICBkc3RfbmFtZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAtMSwgZmFsc2UgLyogV2UgZGlkbid0IGNyZWF0ZSBkc3QgICovLApAQCAtMTQ4NSw5ICsx NTE2LDE0IEBAIGNvcHlfcmVnIChjaGFyIGNvbnN0ICpzcmNfbmFtZSwgY2hhciBjb25zdCAq ZHN0X25hbWUsCiAKICAgc2V0X2F1dGhvciAoZHN0X25hbWUsIGRlc3RfZGVzYywgc3JjX3Ni KTsKIAorI2lmIEhBVkVfRkNMT05FRklMRUFUICYmICFVU0VfWEFUVFIKK3NldF9kZXN0X21v ZGU6CisjZW5kaWYKICAgaWYgKHgtPnByZXNlcnZlX21vZGUgfHwgeC0+bW92ZV9tb2RlKQog ICAgIHsKLSAgICAgIGlmIChjb3B5X2FjbCAoc3JjX25hbWUsIHNvdXJjZV9kZXNjLCBkc3Rf bmFtZSwgZGVzdF9kZXNjLCBzcmNfbW9kZSkgIT0gMAorICAgICAgaWYgKCFtb2RlX2FscmVh ZHlfcHJlc2VydmVkCisgICAgICAgICAgJiYgKGNvcHlfYWNsIChzcmNfbmFtZSwgc291cmNl X2Rlc2MsIGRzdF9uYW1lLCBkZXN0X2Rlc2MsIHNyY19tb2RlKQorICAgICAgICAgICAgICAh PSAwKQogICAgICAgICAgICYmIHgtPnJlcXVpcmVfcHJlc2VydmUpCiAgICAgICAgIHJldHVy bl92YWwgPSBmYWxzZTsKICAgICB9CkBAIC0xNTE3LDcgKzE1NTMsNyBAQCBjb3B5X3JlZyAo Y2hhciBjb25zdCAqc3JjX25hbWUsIGNoYXIgY29uc3QgKmRzdF9uYW1lLAogICAgIH0KIAog Y2xvc2Vfc3JjX2FuZF9kc3RfZGVzYzoKLSAgaWYgKGNsb3NlIChkZXN0X2Rlc2MpIDwgMCkK KyAgaWYgKDAgPD0gZGVzdF9kZXNjICYmIGNsb3NlIChkZXN0X2Rlc2MpIDwgMCkKICAgICB7 CiAgICAgICBlcnJvciAoMCwgZXJybm8sIF8oImZhaWxlZCB0byBjbG9zZSAlcyIpLCBxdW90 ZWFmIChkc3RfbmFtZSkpOwogICAgICAgcmV0dXJuX3ZhbCA9IGZhbHNlOwotLSAKMi4zOS4x Cgo= --------------YNti50GSltfWBNOip0mPvYNd--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 11 Feb 2023 20:39:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 11 15:39:47 2023 Received: from localhost ([127.0.0.1]:43284 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQwex-0002bO-Ew for submit <at> debbugs.gnu.org; Sat, 11 Feb 2023 15:39:47 -0500 Received: from mail-wm1-f47.google.com ([209.85.128.47]:55854) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pQweq-0002av-Gd for 61386 <at> debbugs.gnu.org; Sat, 11 Feb 2023 15:39:45 -0500 Received: by mail-wm1-f47.google.com with SMTP id r18so6159062wmq.5 for <61386 <at> debbugs.gnu.org>; Sat, 11 Feb 2023 12:39:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=zqFwp6yRm7QU+LDG41UgLsod3qpTshYuFa5yEV1zS30=; b=UPGD9JET5BcJ+V62dgcfLxcKw96CJ/pWyPcgBR5VBbnfzvZg2lwmRB1NnqBHLkhT3P yaoPNmd0C2y85TA9QzC49zxbI8Y6DdIA11iaq5C2u+MkPtfCbP03hLxlloOy53MXXUg8 hBRJ37BxbpS48xikru5medaZu6LO+DQCwNHyAwL0tyx3SVr6n8fM9xdv1Da+UXP4ld5q f1PRihm62hzuI3zLBuDozpFLYFhLvvG9WNDcGbdKAo2mp//dDZTdn5ibOE5czlFGszkV gPKHjsrgzl9oFQPycievRParpHV2v7CgDhi2DQEYmOzWLziNzBLOYzHk3F2joKhbGmrP vLPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zqFwp6yRm7QU+LDG41UgLsod3qpTshYuFa5yEV1zS30=; b=qSdKp8JJvjnb7NQqoULL/PWcfOUm92p5Z2oTfCtPecZQ7DRRc43f0e1wyzNV1HkJ8E AOewsMg0QrtSFBctheJ33crlSnemEz5MVp88CgWZRuie35Mkt9CXyoUDSqwShqwytNFd xFLcQNZMWsHPeln8TxR8P8XSZOqlhR7OrjzLqQSTGT4Z7/MlUkvdO1hkZz/OiqnRCNlu +WoIfYiJ/rceMxNpO/Hrl4LnF/rvrq7IwZotruvHDUEy/A7LnzmnZAOKhja23+jb6FPT wsHKOOVnfLQO0lfbJs2UhMnVntstUFDOfmZmo4Wi0Tt7aQP/qP4RR8Gauv4atNCwnP0U RkSw== X-Gm-Message-State: AO0yUKV39vUwHuJb5odEQNgbPbXkT1tzOKr9GX07kY2bi3yqpDNKpL11 bOrUZz54bz2bHtqjbVCdGqE= X-Google-Smtp-Source: AK7set823vXlAE/CGNDcC3aM2/TT9wmMTx+Y6Y8DHnaWCose6yWw6kXqXxp3lfPpGi+SkSgcZawRfw== X-Received: by 2002:a05:600c:3b8b:b0:3e1:bfc:f57b with SMTP id n11-20020a05600c3b8b00b003e10bfcf57bmr13674913wms.2.1676147974254; Sat, 11 Feb 2023 12:39:34 -0800 (PST) Received: from smtpclient.apple (212-39-89-217.ip.btc-net.bg. [212.39.89.217]) by smtp.gmail.com with ESMTPSA id q11-20020a7bce8b000000b003dd9232f036sm11731186wmj.23.2023.02.11.12.39.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 11 Feb 2023 12:39:33 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <14c0a047-7643-970a-5935-36090ab0398e@HIDDEN> Date: Sat, 11 Feb 2023 22:39:29 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <6C3433D4-0B0F-4B45-A362-E40669F1FC2C@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <e71684d9-83f1-bde1-d0df-3e0e6e28996e@HIDDEN> <E51E518A-C826-408D-A81F-AD26A962AD7F@HIDDEN> <14c0a047-7643-970a-5935-36090ab0398e@HIDDEN> To: Paul Eggert <eggert@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On 2023-02-11, at 8:19 PM, Paul Eggert <eggert@HIDDEN> wrote: >=20 > On 2023-02-10 17:27, George Valkov wrote: >> I=E2=80=99d prefer a timestamp consistent with source. >=20 > You can get that with 'cp -p' (or with 'cp --preserve=3Dtimestamps' = etc.). So this is not an issue of whether you can get what you prefer. = It's merely an issue about what cp's default behavior is, when the user = doesn't specify options. Thanks! > POSIX is reasonably clear that if you use plain cp to create a file, = the file's last-modified time should be the current time. And it's not = just a question of standards conformance: lots of real-world uses of = 'cp' expect this behavior. For example, people use plain 'cp' in = Makefiles, and 'make' relies on the last-modified time being the current = time. If cp changed this behavior, it'd break a lot of builds. Ah, yes, make relays on timestamps. So we should update timestamp after = clone by default. Let me know when you have a patch without CLONE_ACL and I will test it. Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 11 Feb 2023 18:20:08 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 11 13:20:08 2023 Received: from localhost ([127.0.0.1]:43162 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQuTo-0006tx-F5 for submit <at> debbugs.gnu.org; Sat, 11 Feb 2023 13:20:08 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:36530) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pQuTl-0006tM-Im for 61386 <at> debbugs.gnu.org; Sat, 11 Feb 2023 13:20:07 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 29384160092; Sat, 11 Feb 2023 10:19:59 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Ejg9J9VYLxD7; Sat, 11 Feb 2023 10:19:55 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id C127F1600AF; Sat, 11 Feb 2023 10:19:55 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu C127F1600AF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676139595; bh=nhCVd1UnpUJPPusyK3w9Tnv4PrWooqV4XXZnoPjBlVE=; h=Message-ID:Date:MIME-Version:To:From:Subject:Content-Type: Content-Transfer-Encoding; b=LXyqUNbg01sGFhtO7vB8IURhPtb1VHfK4tkKkj/RpdZIr0u0lzwcqHkC4SDG/IlvO lVhQlw545i+aYG2GBiJ1ElWgmP6842oJF0dXom7k/2bYDVGDSOGfIwK1VGriY52wVL An0XzH44iectvUUSMzT0I393ZBX+vjTr6Zlw7224= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id qISdgyXOjsYM; Sat, 11 Feb 2023 10:19:55 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 8D72C160092; Sat, 11 Feb 2023 10:19:55 -0800 (PST) Message-ID: <14c0a047-7643-970a-5935-36090ab0398e@HIDDEN> Date: Sat, 11 Feb 2023 10:19:55 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> <e71684d9-83f1-bde1-d0df-3e0e6e28996e@HIDDEN> <E51E518A-C826-408D-A81F-AD26A962AD7F@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS In-Reply-To: <E51E518A-C826-408D-A81F-AD26A962AD7F@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) On 2023-02-10 17:27, George Valkov wrote: > I=E2=80=99d prefer a timestamp consistent with source. You can get that with 'cp -p' (or with 'cp --preserve=3Dtimestamps' etc.)= .=20 So this is not an issue of whether you can get what you prefer. It's=20 merely an issue about what cp's default behavior is, when the user=20 doesn't specify options. POSIX is reasonably clear that if you use plain cp to create a file, the=20 file's last-modified time should be the current time. And it's not just=20 a question of standards conformance: lots of real-world uses of 'cp'=20 expect this behavior. For example, people use plain 'cp' in Makefiles,=20 and 'make' relies on the last-modified time being the current time. If=20 cp changed this behavior, it'd break a lot of builds.
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 11 Feb 2023 01:58:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 20:58:41 2023
Received: from localhost ([127.0.0.1]:38400 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQfA1-0007XG-0I
for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 20:58:41 -0500
Received: from mail-lf1-f41.google.com ([209.85.167.41]:43997)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pQf9y-0007X0-H9
for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 20:58:39 -0500
Received: by mail-lf1-f41.google.com with SMTP id f34so11056596lfv.10
for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 17:58:38 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20210112; t=1676080712;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=gAa/brFkuUW05/CmGjvH6lWDxubWMPoG4v0gdaAtozM=;
b=F9KxBNKwcgInUv0HbdEh1y/+/om2/4o16dz7sk78QiYSGk2qkDSAIfithO22fmPRk5
Mm2bPVaw6eA4vNmJ70X+8NscJ1KQLuncDGDU+4bVUAQ/tk4IYBTlF3i7WbvSttCnjZbs
YFPxyIdVkLsagTk5GM+fBFX5ksd8iH+82C5osLM/T1OIFcIlJY36NcefqKMTrDYOdyGc
uF7Bg5/4AZUdyMlt/oZ1DPsVxXV99YyBrL4i9NKqJbxO4TX3NmSkQ93T4zgDZ9Qz2h6k
ntKpkuYZt7VaETjvrQgzwE8kmd/UA3bj54U1sY+72pOZijhes0hPySgfSzvAxJix+VVg
MKTQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112; t=1676080712;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=gAa/brFkuUW05/CmGjvH6lWDxubWMPoG4v0gdaAtozM=;
b=J+kd2AjuD8c1nny3VMvN5dZjagJplbhrl9EHM2NkfF3ky57C1kSDZ6itUwo5KSJTHp
Lir8aSbZtbMRZfXROPY7kUU4asY12stCeUmZJl/6LXy9UQLVlyZFctGUbvPHOek+EPfr
zykzYNil/0QUXjGsnWsy5W56tqh4F88JK0oIQKhcASihQZSuySjk1rkOzLmKh76P55am
o84RIXumB5QSwzriOhx0SGhR3xCpoIPaES+XOKOQvxAfqSSfctkT9La3Qi/Cdp9GWQmj
hM0R/kCFcIMJcW+u1CbRZLoNZzweMxcTPV+NjCtmD9rdZWv8pj4xhOuLHwKS/CsrL9u8
aRUQ==
X-Gm-Message-State: AO0yUKVBtPphmeywo5fji7gjb5Aly+8E0XJULed82TU1H2rQxiafal6L
uH7POCot0vJwvuC5CdmFfbxETga217I+Tg==
X-Google-Smtp-Source: AK7set8tjf25AQs6NqzQmlTPAQhbud+hqhRPgaSMCA6O6gGhAwaYd0aFG4aIeW6VIjJZoae1/hKgUw==
X-Received: by 2002:a17:906:7485:b0:88c:f97c:7c87 with SMTP id
e5-20020a170906748500b0088cf97c7c87mr14210552ejl.2.1676078848306;
Fri, 10 Feb 2023 17:27:28 -0800 (PST)
Received: from smtpclient.apple (212-39-89-169.ip.btc-net.bg. [212.39.89.169])
by smtp.gmail.com with ESMTPSA id
m2-20020a1709061ec200b0087848a5daf5sm3117193ejj.225.2023.02.10.17.27.26
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Fri, 10 Feb 2023 17:27:27 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <e71684d9-83f1-bde1-d0df-3e0e6e28996e@HIDDEN>
Date: Sat, 11 Feb 2023 03:27:25 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <E51E518A-C826-408D-A81F-AD26A962AD7F@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
<e71684d9-83f1-bde1-d0df-3e0e6e28996e@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 1.5 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: > On 2023-02-11, at 12:02 AM, Pádraig Brady wrote: > > On
10/02/2023 21:50, Paul Eggert wrote: >> On 2/10/23 13:35, George Valkov wrote:
>>> Since the source and it’s clone have separate metadata [...]
Content analysis details: (1.5 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
[212.39.89.169 listed in dnsbl.sorbs.net]
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (gvalkov[at]gmail.com)
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[209.85.167.41 listed in wl.mailspike.net]
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust
[209.85.167.41 listed in list.dnswl.org]
X-Debbugs-Envelope-To: 61386
Cc: Paul Eggert <eggert@HIDDEN>, 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)
> On 2023-02-11, at 12:02 AM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 10/02/2023 21:50, Paul Eggert wrote:
>> On 2/10/23 13:35, George Valkov wrote:
>>> Since the source and it=E2=80=99s clone have separate metadata,
>>> it should be possible to change it on the clone, to comply with =
standards.
>> Attached is a hacky patch to do that. It also uses the new CLONE_ACL
>> flag. The old code apparently mishandled ACLs; the new code tries to =
do
>> a better job.
>=20
> Yes this is looking much better.
> I'm not fully convinced that we should also update the times,
> but not against it either.
I=E2=80=99d prefer a timestamp consistent with source. This is the =
default when
we clone a file. So no action needed. If we fall-back to copy, we need =
to set it.
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 11 Feb 2023 01:55:26 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 20:55:26 2023 Received: from localhost ([127.0.0.1]:38394 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQf6s-0007SZ-Ag for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 20:55:26 -0500 Received: from mail-wr1-f43.google.com ([209.85.221.43]:38404) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pQf6q-0007SM-3x for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 20:55:25 -0500 Received: by mail-wr1-f43.google.com with SMTP id ba1so6916771wrb.5 for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 17:55:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=O6kuFoXzcGa1e1Puu1+9KBRC3dBwAPyM72HdxQbj+WE=; b=nUyMfUKNHfldiodml68Kf2RDaWE7FwoRDWNYBdUaM/Xvrw6lZSfy9AiSxBz4vbpDPY LGjJ/SK+5FkgPoJsrIZ+9KKarIG9/2hgA/ZMYse0uGlG/nnPcHXpafiERTbNRU5cblEK a9yOLi6RIdmehUjifqGO5uAMUCBwSYg2AvPw3kBzNrYl58vuQQi2CUChHK4SDkCLuPKn uDaythOqqjfNu9oe1EXJKcYTe8sL21TicXAeKXWmBOfuoGC1G+r+SRRxo5wrl2VuvK/Y ZOAr5RtVP/xKYy4aqXGJbS+J5qRKFi4grzOXcspdwJfk0ZSs8KdJ/GfdBK3A+LuiWGgA VzsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O6kuFoXzcGa1e1Puu1+9KBRC3dBwAPyM72HdxQbj+WE=; b=MEFFv5tIQRZsUUVhBnVnt8h/UxgpTl5Jjp39N1CJPYWbwdLiXMHtXguDb1ooq+Sv0I wnwI5owMqa3XblX+E6eze6Emf3B9Difil+/sfScJVxZy/zSpcrna1Piv0qYq+kx4yPhq w+GefeB7F5Yk6FKND5UmNtR2nbXs1jQlcHwFLYnMfBjQWomS39uPctAIixM1gsHvEFGi SYSF2ca1KKJQMd0NZKq77B6yoiOkKEbnhUcfVrUYZBq4QvA6NgxhE/S1PefEhwcDBK+A MhwBAKNVfdyBBDHxof2yzifvJHo+dzaCUVBBNFSkjF5azGARHuKve7JBV+Em4u3eta/1 FDLg== X-Gm-Message-State: AO0yUKUg2oywQHm5WC+p4d6C7t2FfcG2gyCrAblwe1VmQ1xgVJexKahU YFj4Tu/VzNCOVNMGtDmMzWvmxtSHN/Kz8w== X-Google-Smtp-Source: AK7set9PjXaCFpgY8wMg9DlqyfOR/yp46a//XRz18XMvrdDYqRIMkvvWdnE26zeCiPx6wajQsKd/lQ== X-Received: by 2002:a05:600c:3ca3:b0:3dc:5ae4:c13d with SMTP id bg35-20020a05600c3ca300b003dc5ae4c13dmr18919698wmb.4.1676080083336; Fri, 10 Feb 2023 17:48:03 -0800 (PST) Received: from smtpclient.apple (212-39-89-169.ip.btc-net.bg. [212.39.89.169]) by smtp.gmail.com with ESMTPSA id n2-20020a05600c3b8200b003dfe659f9b1sm16995608wms.3.2023.02.10.17.48.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Feb 2023 17:48:02 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <2277295c-652d-2f46-ae18-f93f202ca82d@HIDDEN> Date: Sat, 11 Feb 2023 03:48:00 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <4D44CBC0-CAB6-4930-A488-8597DC9B6DBC@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <c60ab2bd-fff8-472b-fbe9-d9f0aa72eb37@HIDDEN> <B529D5D5-9C66-42BC-9D8F-9563D84DBD8E@HIDDEN> <2277295c-652d-2f46-ae18-f93f202ca82d@HIDDEN> To: Paul Eggert <eggert@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: > On 2023-02-11, at 2:48 AM, Paul Eggert wrote: > > I want to emphasize the fact that we need to get to the bottom of the SEEK_HOLE / SEEK_DATA situation on macOS. Core programs other than 'cp' use [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server [212.39.89.169 listed in dnsbl.sorbs.net] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (gvalkov[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.221.43 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.221.43 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.5 (/) > On 2023-02-11, at 2:48 AM, Paul Eggert <eggert@HIDDEN> wrote: >=20 > I want to emphasize the fact that we need to get to the bottom of the = SEEK_HOLE / SEEK_DATA situation on macOS. Core programs other than 'cp' = use these options, and working around the bug in 'cp' won't fix the bug = elsewhere. >=20 > One possibility is to modify Gnulib so that SEEK_HOLE and SEEK_DATA = are never used by any Gnulib-using application. This will help somewhat. = But not every program uses Gnulib. Would it help, if I provide access to macOS? > The bug is really at the macOS level and it needs to get fixed there. = Has anyone filed a bug report? (I'm not a macOS user and so can't file = one.) I wrote to product-security@HIDDEN on 2021-10-08. > I have reviewed your report, and we are investigating. No further replay. I also tried phone and development support. They are extremely hard to work with. They always keep us in the dark and never admit to anything. 30 months of repair services, and erasing my data every time. Perhaps we can try in public here? https://github.com/apple It helps me value and appreciate working with OpenWRT and you kind = people. Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 11 Feb 2023 01:53:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 20:53:38 2023
Received: from localhost ([127.0.0.1]:38386 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQf58-0007PJ-6a
for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 20:53:38 -0500
Received: from mail-wr1-f42.google.com ([209.85.221.42]:40561)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pQf55-0007P4-QH
for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 20:53:36 -0500
Received: by mail-wr1-f42.google.com with SMTP id r2so6905930wrv.7
for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 17:53:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=uyiYcifZrvYrTMi3ZcX171R6eUvr+j++msMfALXaHkU=;
b=VxZTApCH2/U703df0CpLLS0E4xHNi1u+1OnVavnhS/ob5n3K5S61go8WArEXkJsM1h
APYOvJiZ/QmEqI2VIgLfKHi1imo80cM24XEIByLXexZPosW452PzKog+CqBNklkYCOQc
p7wdU8diJTv6n5vJffGelMEBacaVVmg0EBLcOjSdtgxEpRq7baB9WWwQ69/tx65vY0NA
SjDBnEufQb+ZVmWVCiJo/zcuZRJ4ktLpZGZuvFKKTE9q4HQsql5qPlXXkQfHDG0XsxPt
D/y+UkmlftA0L+/+WFK2OsFerZnbvDgYK0ng+lqwuLB/0ufow8CV4NFYpCeFQwz6dyxN
7KZQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=uyiYcifZrvYrTMi3ZcX171R6eUvr+j++msMfALXaHkU=;
b=bFE7YHCOZkui+ydDjR7lDbWnjpcsOYNHAJWtCMGlS1Addsp3rcWYqAdoxfAhxJi2Ft
GdOy+9iDe1TyMflVItMwQwHfziFQ021DCjepY7sJn3JzHPHA2DjxefluDplIySbe+7E5
w6x+Bthi09Hpv1m6Ti/9BXSIUx5H0/msdIr64eBHnFFiP2FXFy1RKXzcD3s//yYVB14f
JwXBRMJXoPBcK2ZRzJmDuFH1MjRQwdlupQKz6oSI7ATuIxg0UzNKjthktgRKk6iI/H9b
I6xXFjmyVcAuJpf2Bi5xL7A8hBNwDKwPQb/wyNmOIMW3c3La1ASOyZkklL2c0/xAtqJZ
ynzQ==
X-Gm-Message-State: AO0yUKU+tAd0YjUyWDG+r88kG4cIe/1s2NCWZ3SrSGRRhQOMwgpALXTb
2FFyjQnjG7OUx52MRH6ESyCgsQn6tBvmMw==
X-Google-Smtp-Source: AK7set8QWIt46ajfhH1tl1DbRODDq29OfUUzlFi3Z7eQHHbl1Kgc0eJ51Ik2nTRREEcaFSQwQUEAEg==
X-Received: by 2002:adf:ea4c:0:b0:2c4:986:e271 with SMTP id
j12-20020adfea4c000000b002c40986e271mr6254155wrn.5.1676078506891;
Fri, 10 Feb 2023 17:21:46 -0800 (PST)
Received: from smtpclient.apple (212-39-89-169.ip.btc-net.bg. [212.39.89.169])
by smtp.gmail.com with ESMTPSA id
m18-20020a5d6252000000b002c3dc4131f5sm4889838wrv.18.2023.02.10.17.21.45
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Fri, 10 Feb 2023 17:21:45 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
Date: Sat, 11 Feb 2023 03:21:44 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <DD5D647B-0539-483A-943B-AFDBBCDBE9D7@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN>
<335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN>
To: Paul Eggert <eggert@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 1.5 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: > On 2023-02-10, at 11:50 PM, Paul Eggert wrote: > > On 2/10/23
13:35, George Valkov wrote: > >> Since the source and it’s clone have separate
metadata, >> it should be possible to change it on th [...]
Content analysis details: (1.5 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (gvalkov[at]gmail.com)
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[209.85.221.42 listed in wl.mailspike.net]
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust
[209.85.221.42 listed in list.dnswl.org]
1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
[212.39.89.169 listed in dnsbl.sorbs.net]
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)
> On 2023-02-10, at 11:50 PM, Paul Eggert <eggert@HIDDEN> wrote:
>=20
> On 2/10/23 13:35, George Valkov wrote:
>=20
>> Since the source and it=E2=80=99s clone have separate metadata,
>> it should be possible to change it on the clone, to comply with =
standards.
>=20
> Attached is a hacky patch to do that. It also uses the new CLONE_ACL =
flag. The old code apparently mishandled ACLs; the new code tries to do =
a better job.
Hint: remember to trace all code paths errors.
The program may and does steer in directions other than what you =
expected.
Without CLONE_ACL, UNIX permissions and macOS extended attributes
from the source file are preserved, timestamp is updated:
cp A B
-rw-------@ 1 g staff 332 Feb 11 01:57 A
-rw-------@ 1 g staff 332 Feb 11 02:19 B
With CLONE_ACL, I get 22 Invalid argument. Then falls-back to something =
else,
UNIX permissions and timestamp are preserved, extended attributes are =
lost:
cp -p A B
-rw-------@ 1 g staff 332 Feb 11 01:57 A
-rw------- 1 g staff 332 Feb 11 01:57 B
fclonefileat also requires that the target does not exist, if it does we =
have to remove it,
otherwise it fails: 17 File exists. With your patch, cp does not print =
anything.
Now the question here is:
do we want to go the quick route of unlinking the destination and =
creating a fresh clone?
Or do we consider the possibility of multiple hard-links in which case =
if we want to update
a hard-link, we must overwrite the entire file and not use unlink, =
fclonefileat.
If we are absolutely certain that only one link exists, the first =
approach is safe
and more efficient.
If more than one link exist, we must always overwrite the existing data, =
to keep
hard-links consistent.
> This whole area is a mess, but the patch should improve correctness on =
macOS, while also being efficient in the usual case.
We need to resolve the issues described above first.
>> It feels more natural when the metadata is kept intact for the two =
files.
>=20
> That depends on the application. The longstanding tradition for cp is =
to preserve metadata if you use 'cp -p', and to not preserve with plain =
cp. For interactive use you can use an alias or a little script if you =
prefer -p to be enabled by default.
Thank you for clarifying!
I used the sample I wrote in a previous message to test fclonefileat =
independently
of coreutils: the default behaviour is to preserve everything.
The manpage installed on macOS is from June 3, 2021. While it does =
describe the
behaviour of CLONE_ACL, reality shows it is not supported. CLONE_ACL =
fails with
22 Invalid argument. Which means additional code would be required to
create what cp thinks as default metadata, which might be prone to =
errors
>> Here is a good example: I create a tarball on my OpenWRT router. =
Extract on the Mac.
>=20
> That's fine, because 'tar' historically has preserved the timestamp =
and (if you're root) permissions. Plain cp has not.
Fair enough. For comparison, here is the behaviour of coreutils 9.1 cp =
release:
copy UNIX permissions, without extended attributes or timestamp
cp A B
-rw-------@ 1 g staff 332 Feb 11 01:57 A
-rw------- 1 g staff 332 Feb 11 02:40 B
copy UNIX permissions, extended attributes and timestamp
cp -p A B
-rw-------@ 1 g staff 332 Feb 11 01:57 A
-rw-------@ 1 g staff 332 Feb 11 01:57 B
Good luck!
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 11 Feb 2023 00:49:32 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 19:49:31 2023 Received: from localhost ([127.0.0.1]:38362 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQe55-0005f2-KM for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 19:49:31 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:37608) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pQe50-0005ed-EY for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 19:49:30 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id B629F1600A9; Fri, 10 Feb 2023 16:48:55 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id SwsgdoXSue9f; Fri, 10 Feb 2023 16:48:55 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id F190C1600AA; Fri, 10 Feb 2023 16:48:54 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu F190C1600AA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676076535; bh=Be5S6a1j3IYA1jdT0hQ4Wwz14p+bCR+vpMj/lhrWx20=; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type: Content-Transfer-Encoding; b=dkm2vHSBwUSFHIEfOGOwwsH8Cg6vB25SCrLhII29QVeV6dtPEWXjof7B3QKznXNtn U6Hs40W8au23QKIU5ib0Mf4VF2uihUGjP7ziJgOVeeimtauGOsJlKfKXTkst6aFXYE OqT+gMnNcJHHRvSl0GqWueUCW/25JWbzto1nTCpo= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id LZQCJK305pUG; Fri, 10 Feb 2023 16:48:54 -0800 (PST) Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 648BA1600A9; Fri, 10 Feb 2023 16:48:54 -0800 (PST) Message-ID: <2277295c-652d-2f46-ae18-f93f202ca82d@HIDDEN> Date: Fri, 10 Feb 2023 16:48:54 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <c60ab2bd-fff8-472b-fbe9-d9f0aa72eb37@HIDDEN> <B529D5D5-9C66-42BC-9D8F-9563D84DBD8E@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <B529D5D5-9C66-42BC-9D8F-9563D84DBD8E@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) I want to emphasize the fact that we need to get to the bottom of the SEEK_HOLE / SEEK_DATA situation on macOS. Core programs other than 'cp' use these options, and working around the bug in 'cp' won't fix the bug elsewhere. One possibility is to modify Gnulib so that SEEK_HOLE and SEEK_DATA are never used by any Gnulib-using application. This will help somewhat. But not every program uses Gnulib. The bug is really at the macOS level and it needs to get fixed there. Has anyone filed a bug report? (I'm not a macOS user and so can't file one.)
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 23:48:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 18:48:49 2023
Received: from localhost ([127.0.0.1]:38329 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQd8K-00040R-VS
for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 18:48:49 -0500
Received: from mail-wm1-f48.google.com ([209.85.128.48]:43714)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pQd8I-00040C-Rb
for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 18:48:47 -0500
Received: by mail-wm1-f48.google.com with SMTP id
az4-20020a05600c600400b003dff767a1f1so5270669wmb.2
for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 15:48:46 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=iL+UZkEZlvftrlF182SS6GCgUfbIzjID46qYLNERiWk=;
b=Id4ooGXiio12E7Suzo8RWRLg7JZhc6U6b0xjqteuLd42bcHG5NPdBdx24hYX67abmf
PMxegN6bg040R4xfgvD8OvWmXafUQSYDcVmXGs1PGQlILwyzUl3/gcu/HB4gmQw72Klj
5cgikevDAKe8xzYNopH3VLROQFuiQ06xMZrxeghgar42A4KKDjZMi9NQL/LikUfTekAU
BL649OBwJB8MD3KojswXTy0zg2SXN9Z7zBaI4pC+TE1zQo0AWz/QkaFozrvcT1EPNUWs
T9k/uzDAtvlUlJF3p+XfBzH7vjardZ7in2PLs0xbSVLk0Wer10RYNqcO/zDKNIvRE36h
L9hw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=iL+UZkEZlvftrlF182SS6GCgUfbIzjID46qYLNERiWk=;
b=0F/RW33XlBtaqqvVfBTbalA2xsbNDpzXj4HKNnxLHOpz5nqnMwN2wdeK5l0xf8h6Dc
vgVNKhDAKe95Q5aiEsYeOv7OM/mprkoPez7lcEH4Le0Wl1446oJpAsnFasto78ubEeqz
S5prUjG+I3Xd2jNkBJ7FR8xErTMVJGVd3nvdZ/N1aFG31FlLa+SGqpYzL/qnLxQBsWc0
qEQP/47QNRzoT23fD3ubYwGFwyuc1nsYl4UpqTXWE4GYQGxMc3E5/DwkVN3AHRhFfbqR
rIpe+2XdEimrSkyDWjSV09Iy/FNuL6lAzfwocLnU1wHi38fM3YNe/4l9hCMQAUHJiY8u
NHPQ==
X-Gm-Message-State: AO0yUKUg75LlwPVYHSnDX6abqXTXOVCwAsTTIXhyX22styc62aXZupOi
ICIGEBlm6QpLlB9jFaivml4=
X-Google-Smtp-Source: AK7set8O7YUZvEGWjk4+wATrSHr5Sia/OC6jK4Vijj9H+JQnHFzBhMIgX0treNuoDuVe7aEVADWbWg==
X-Received: by 2002:a05:600c:1c1b:b0:3dc:4234:fe89 with SMTP id
j27-20020a05600c1c1b00b003dc4234fe89mr18775412wms.2.1676072920771;
Fri, 10 Feb 2023 15:48:40 -0800 (PST)
Received: from smtpclient.apple (212-39-89-169.ip.btc-net.bg. [212.39.89.169])
by smtp.gmail.com with ESMTPSA id
c129-20020a1c3587000000b003dc1a525f22sm6723657wma.25.2023.02.10.15.48.39
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Fri, 10 Feb 2023 15:48:39 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <c60ab2bd-fff8-472b-fbe9-d9f0aa72eb37@HIDDEN>
Date: Sat, 11 Feb 2023 01:48:38 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <B529D5D5-9C66-42BC-9D8F-9563D84DBD8E@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
<acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN>
<c60ab2bd-fff8-472b-fbe9-d9f0aa72eb37@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 1.5 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: > On 2023-02-10, at 11:46 PM, Pádraig Brady wrote: > > On
10/02/2023 20:45, Paul Eggert wrote: >> On 2/10/23 10:58, Pádraig Brady
wrote: >>> I was considering "touch"ing the timestamps after also, [...]
Content analysis details: (1.5 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (gvalkov[at]gmail.com)
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust
[209.85.128.48 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[209.85.128.48 listed in wl.mailspike.net]
1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
[212.39.89.169 listed in dnsbl.sorbs.net]
X-Debbugs-Envelope-To: 61386
Cc: Paul Eggert <eggert@HIDDEN>, 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)
> On 2023-02-10, at 11:46 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 10/02/2023 20:45, Paul Eggert wrote:
>> On 2/10/23 10:58, P=C3=A1draig Brady wrote:
>>> I was considering "touch"ing the timestamps after also,
>>> but it's better to just maintain them as we're
>>> pointing to the same data after all.
>> For POSIX conformance we must touch if the user has specified only =
POSIX
>> options (and has not specified -p).
>> And it's not just a POSIX conformance issue. Ordinary users will be
>> surprised if plain 'cp A B' creates a file B with a timestamp from =
last
>> year.
>=20
> Maybe. Though POSIX says cp "shall copy" and we're not making a copy, =
we're making a reflink.
> So technically we're violating POSIX already in that regard.
A hard link is when we have two or more names for the same file.
We can read or write and it affects the same disk content.
That would violate POSIX shall copy, but we are not doing a hard link.
A clone behaves exactly like a copy. Reads and writes affect only
the selected file. It=E2=80=99s rather an optimised copy.
> One might take the view that the fact we write no new data
> means we should not update the data modification time etc. by default,
> and this may be more signal to a user that new data has not in fact =
been written.
We can describe the changes in the documentation. =46rom a user =
perspective,
everything works exactly as before, only faster. Some users might get =
surprised
the first time they copy several gigabytes and we finish instantly. If =
will check:
the copy is there and it works. So they=E2=80=99ll get used to it and be =
happy about the change.
I always wanted a tool that does CoW. I knew APFS supports it.
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 22:42:55 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 17:42:55 2023 Received: from localhost ([127.0.0.1]:38270 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQc6Z-000248-9Y for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 17:42:55 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:52924) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pQc6W-00023o-40 for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 17:42:54 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id B7DC01600A9; Fri, 10 Feb 2023 14:42:45 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 1Ml17DuFNQ8Z; Fri, 10 Feb 2023 14:42:45 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id ED5391600AA; Fri, 10 Feb 2023 14:42:44 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu ED5391600AA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676068965; bh=O1ieqE3M/OJqwgLDEAGzcQh3Ole4kvFoUw/zKbLa8Vw=; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type: Content-Transfer-Encoding; b=eIXkpyps8kGLIIu1IPrvcqkMjEOG0NnCmu+KFphV6NPf7dV8vjgCdrpEBobToZY3W qBDcumWzk4Hi5euD3zEnczgfezdLbP2kgOrxS/mt5yNy1Sh4SAFrf+Z/pbNqqa8MI1 JXmaBFopYB1ZkED9XOqY2nkKtOKSNAUBVm998HVU= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id MMmwn__5OAXx; Fri, 10 Feb 2023 14:42:44 -0800 (PST) Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id CD62A1600A9; Fri, 10 Feb 2023 14:42:44 -0800 (PST) Message-ID: <19813e2f-6ff5-b0c4-6c81-6317cc41c355@HIDDEN> Date: Fri, 10 Feb 2023 14:42:44 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <c60ab2bd-fff8-472b-fbe9-d9f0aa72eb37@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <c60ab2bd-fff8-472b-fbe9-d9f0aa72eb37@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) On 2/10/23 13:46, P=C3=A1draig Brady wrote: > Maybe. Though POSIX says cp "shall copy" and we're not making a copy,=20 > we're making a reflink. If that were an important reason not to clone, then cp should not have=20 made --reflink=3Dauto the default, as clones would not be considered to b= e=20 copies. However, I'm comfortable with the idea that clones satisfy the POSIX=20 definition of copying. The ordinary interpretation of "clone" means a=20 copy, and at the POSIX level users can't tell the difference so the=20 as-if rule applies. And if the file system does deduplication+CoW at the=20 block level, with something like VDO say, there's no good way GNU 'cp'=20 can force a copy rather than cloning regardless.
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 22:03:04 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 17:03:04 2023 Received: from localhost ([127.0.0.1]:38236 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQbU0-00013P-0l for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 17:03:04 -0500 Received: from mail-wm1-f42.google.com ([209.85.128.42]:50760) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pQbTy-00012v-CB for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 17:03:02 -0500 Received: by mail-wm1-f42.google.com with SMTP id bg26so4845336wmb.0 for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 14:03:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=Zm1ju8pUwZOIZ7slz1wuhfD0ENr63grMISTxOLlvjqQ=; b=UZZYN7PT39iTHRU9TaC+Q2Y2tRsTUyKZ6OuFkhHdEAt3Uv1lFDFP5yHxd/weHnXbgZ Uv+0szxArFpECUSSNjbuU653rziQgGriXqxpgcxE9YQgtakd4gjVSBArwbaeBpezHLxk Q/05M5iHbeqbvVgY+7Y18drnMVJOWxmjJmgyz97X8v1YcItKSPYB8+PpBGYPIFVj2Y7V 7YzVjO0srr8dCCKAXmj72BD0dqCYwCD/qZNQ+0XurMzwtnCGS6YquM3tP6BRiZfzlL7j Gl9sEEvPsGSZrUzrA77GQxFTNmsphGw0/TfjlFw6x8SslzrIoUvJMxn4BHbpHft6yxgA 4hCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Zm1ju8pUwZOIZ7slz1wuhfD0ENr63grMISTxOLlvjqQ=; b=nKsj7BbrvOvG/Vprpk3ln52POGiBxSb6BTVZAJKUSzspxLp9BX8EmXztsP2YuxoFLX FWmIyb6DvchKDq96PU+MeFWLbw5rBC0RqOGGdUNlTXy4ZowN9X07QfxGx7S+/hpSDbpY qHiWBCPnKXqPEJxz0kjXMqq4PSdNg3cprHqzki1t+y3yNvzTic6fBsyYowLs4jn58BRN Ng7/ezziOreMoaaQKOpXyynLJKHXlltzmJzsV7yKeLZwECeG53bayLNdVhD8/lvcbwhM M1UMbg1/UbuK0AdKdxL4qroTQ0tqS2wQAy3F3NUmHR38SiYZeANzBuxIbaF2EQeKtzOY KCzg== X-Gm-Message-State: AO0yUKUk9VIuhZZLGHrrrUBQ+cf7kXYzyyknjPJ0+Be8trWbh38Z2llQ HlVvVeZOI3VUtGRA0VA+g5U= X-Google-Smtp-Source: AK7set/mSlFZ+i4ldrOC2Ovw77N3eLG/S6LTlTDnJUP4OMivi7vq6FPA1rds2NN7va7E5JG0PEsPyg== X-Received: by 2002:a05:600c:4a9a:b0:3dc:f24:f2de with SMTP id b26-20020a05600c4a9a00b003dc0f24f2demr13882069wmp.12.1676066576414; Fri, 10 Feb 2023 14:02:56 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id iv12-20020a05600c548c00b003dc521f336esm7064514wmb.14.2023.02.10.14.02.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 10 Feb 2023 14:02:55 -0800 (PST) Message-ID: <e71684d9-83f1-bde1-d0df-3e0e6e28996e@HIDDEN> Date: Fri, 10 Feb 2023 22:02:54 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 10/02/2023 21:50, Paul Eggert wrote: > On 2/10/23 13:35, George Valkov wrote: > >> Since the source and it’s clone have separate metadata, >> it should be possible to change it on the clone, to comply with standards. > > Attached is a hacky patch to do that. It also uses the new CLONE_ACL > flag. The old code apparently mishandled ACLs; the new code tries to do > a better job. Yes this is looking much better. I'm not fully convinced that we should also update the times, but not against it either. > This whole area is a mess, but the patch should improve correctness on > macOS, while also being efficient in the usual case. Yes it's a pity there is no fd (data) only interface for this, rather than dealing with names (metadata) too. thanks, Pádraig.
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 21:50:59 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 16:50:59 2023 Received: from localhost ([127.0.0.1]:38232 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQbII-0000ka-Ea for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 16:50:58 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:47974) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pQbIE-0000kI-Dm for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 16:50:57 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 5FD5D1600A9; Fri, 10 Feb 2023 13:50:48 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 7IA_sUpJene0; Fri, 10 Feb 2023 13:50:47 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 440221600AE; Fri, 10 Feb 2023 13:50:47 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 440221600AE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676065847; bh=GcoBEaQx5IrkoGdq+sB8O3KPStln4eB36hz2Aq9lXj4=; h=Content-Type:Message-ID:Date:MIME-Version:Subject:To:From; b=WwFBnQzbVXyRq0C2MnEj67bVQRhBKpqdPq6McWLaIzfs7bFP1qoi1DTAeABw/ukFR XugwSqtOPKQCKbafCr1hrf8p5SgmiDQlIS5FGAyeVX6/Wf04f/cdz+OUQ86wTIc6XO QvoQD9gptSacDc/qnlhCAfhKFMHDN7lzD2zdJodo= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ct0ebpIhr2Kr; Fri, 10 Feb 2023 13:50:47 -0800 (PST) Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 1F8241600A9; Fri, 10 Feb 2023 13:50:47 -0800 (PST) Content-Type: multipart/mixed; boundary="------------yWm5PBUttzQ4ouxbX70YEyUR" Message-ID: <335a749b-4a89-d04f-8db6-41a60becf98c@HIDDEN> Date: Fri, 10 Feb 2023 13:50:46 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) This is a multi-part message in MIME format. --------------yWm5PBUttzQ4ouxbX70YEyUR Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable On 2/10/23 13:35, George Valkov wrote: > Since the source and it=E2=80=99s clone have separate metadata, > it should be possible to change it on the clone, to comply with standar= ds. Attached is a hacky patch to do that. It also uses the new CLONE_ACL=20 flag. The old code apparently mishandled ACLs; the new code tries to do=20 a better job. This whole area is a mess, but the patch should improve correctness on=20 macOS, while also being efficient in the usual case. > It feels more natural when the metadata is kept intact for the two file= s. That depends on the application. The longstanding tradition for cp is to=20 preserve metadata if you use 'cp -p', and to not preserve with plain cp.=20 For interactive use you can use an alias or a little script if you=20 prefer -p to be enabled by default. > Here is a good example: I create a tarball on my OpenWRT router. Extrac= t on the Mac. That's fine, because 'tar' historically has preserved the timestamp and=20 (if you're root) permissions. Plain cp has not. > I found this link in one of the mailing lists you sent, it explains wha= t was wrong > with the original SEEK_DATA and SEEK_HOLE approach on macOS GNU cp does a pass over the file with SEEK_HOLE and SEEK_DATA, so if I=20 understand things correctly it shouldn't run into the problem mentioned=20 there. In other words, that email doesn't appear to explain our current problem. --------------yWm5PBUttzQ4ouxbX70YEyUR Content-Type: text/x-patch; charset=UTF-8; name="0001-cp-improve-use-of-fclonefileat.patch" Content-Disposition: attachment; filename="0001-cp-improve-use-of-fclonefileat.patch" Content-Transfer-Encoding: base64 RnJvbSA5ZjJjMGFiNjQ5MjVhNzBlMjg3YTkyZTk5MjgxMDBjN2NkNjM2OGMyIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBGcmksIDEwIEZlYiAyMDIzIDEzOjM0OjU0IC0wODAwClN1YmplY3Q6IFtQQVRD SF0gY3A6IGltcHJvdmUgdXNlIG9mIGZjbG9uZWZpbGVhdAoKKiBzcmMvY29weS5jIChjb3B5 X3JlZyk6IFVzZSBDTE9ORV9BQ0wgaWYgYXZhaWxhYmxlLgpJZiB0aGUgb25seSBwcm9ibGVt IHdpdGggZmNsb25lZmlsZWF0IGlzIHRoYXQgaXQgd291bGQgZ2VuZXJhdGUKdGhlIHdyb25n IHRpbWVzdGFtcCwgb3Igd291bGQgY3JlYXRlIHRoZSBmaWxlIHdpdGggdG9vIGZldwpwZXJt aXNzaW9ucywgdXNlIGl0IGFuZCBmaXggdGhlIG1vZGUgYW5kIHRpbWVzdGFtcCBhZnRlcndh cmRzLApyYXRoZXIgdGhhbiBmYWxsaW5nIGJhY2sgb24gYSB0cmFkaXRpb25hbCBjb3B5Lgot LS0KIHNyYy9jb3B5LmMgfCA0OCArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr KysrKysrKy0tLS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCA0MSBpbnNlcnRpb25zKCspLCA3IGRl bGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3NyYy9jb3B5LmMgYi9zcmMvY29weS5jCmluZGV4 IGRmYmI1NTdkZS4uMDRmNzFlYjJmIDEwMDY0NAotLS0gYS9zcmMvY29weS5jCisrKyBiL3Ny Yy9jb3B5LmMKQEAgLTExNDYsNiArMTE0Niw3IEBAIGNvcHlfcmVnIChjaGFyIGNvbnN0ICpz cmNfbmFtZSwgY2hhciBjb25zdCAqZHN0X25hbWUsCiAgIHVuaW9uIHNjYW5faW5mZXJlbmNl IHNjYW5faW5mZXJlbmNlOwogICBib29sIHJldHVybl92YWwgPSB0cnVlOwogICBib29sIGRh dGFfY29weV9yZXF1aXJlZCA9IHgtPmRhdGFfY29weV9yZXF1aXJlZDsKKyAgYm9vbCBtb2Rl X2FscmVhZHlfcHJlc2VydmVkID0gZmFsc2U7CiAgIGJvb2wgcHJlc2VydmVfeGF0dHIgPSBV U0VfWEFUVFIgJiB4LT5wcmVzZXJ2ZV94YXR0cjsKIAogICBzb3VyY2VfZGVzYyA9IG9wZW4g KHNyY19uYW1lLApAQCAtMTI0NCwxNyArMTI0NSw0NSBAQCBjb3B5X3JlZyAoY2hhciBjb25z dCAqc3JjX25hbWUsIGNoYXIgY29uc3QgKmRzdF9uYW1lLAogICBpZiAoKm5ld19kc3QpCiAg ICAgewogI2lmIEhBVkVfRkNMT05FRklMRUFUICYmICFVU0VfWEFUVFIKLS8qIENMT05FX05P T1dORVJDT1BZIG9ubHkgYXZhaWxhYmxlIG9uIG1hY29zID49IDEwLjEzLiAgKi8KKyMgaWZu ZGVmIENMT05FX0FDTAorIyAgZGVmaW5lIENMT05FX0FDTCAwIC8qIEFkZGVkIGluIG1hY09T IDEyLjYuICAqLworIyBlbmRpZgogIyBpZm5kZWYgQ0xPTkVfTk9PV05FUkNPUFkKLSMgIGRl ZmluZSBDTE9ORV9OT09XTkVSQ09QWSAwCisjICBkZWZpbmUgQ0xPTkVfTk9PV05FUkNPUFkg MCAvKiBBZGRlZCBpbiBtYWNPUyAxMC4xMy4gICovCiAjIGVuZGlmCi0gICAgICBpbnQgZmNf ZmxhZ3MgPSB4LT5wcmVzZXJ2ZV9vd25lcnNoaXAgPyAwIDogQ0xPTkVfTk9PV05FUkNPUFk7 CisgICAgICBtb2RlX3QgY2xvbmVkX21vZGVfYml0cyA9IFNfSVNWVFggfCBTX0lSV1hVR087 CisgICAgICBtb2RlX3QgY2xvbmVkX21vZGUgPSBzcmNfbW9kZSAmIGNsb25lZF9tb2RlX2Jp dHM7CisgICAgICBpbnQgZmNfZmxhZ3MgPSAoKHgtPnByZXNlcnZlX21vZGUgPyBDTE9ORV9B Q0wgOiAwKQorICAgICAgICAgICAgICAgICAgICAgIHwgKHgtPnByZXNlcnZlX293bmVyc2hp cCA/IDAgOiBDTE9ORV9OT09XTkVSQ09QWSkpOwogICAgICAgaWYgKGRhdGFfY29weV9yZXF1 aXJlZCAmJiB4LT5yZWZsaW5rX21vZGUKLSAgICAgICAgICAmJiB4LT5wcmVzZXJ2ZV9tb2Rl ICYmIHgtPnByZXNlcnZlX3RpbWVzdGFtcHMKKyAgICAgICAgICAmJiAoeC0+cHJlc2VydmVf bW9kZSB8fCAhIChjbG9uZWRfbW9kZSAmIH5kc3RfbW9kZSkpCiAgICAgICAgICAgJiYgKHgt PnByZXNlcnZlX293bmVyc2hpcCB8fCBDTE9ORV9OT09XTkVSQ09QWSkpCiAgICAgICAgIHsK KyAgICAgICAgICBmcHJpbnRmKHN0ZGVyciwgImNhbGxpbmcgZmNsb25lZmlsZWF0Li4uXG4i KTsKICAgICAgICAgICBpZiAoZmNsb25lZmlsZWF0IChzb3VyY2VfZGVzYywgZHN0X2RpcmZk LCBkc3RfcmVsbmFtZSwgZmNfZmxhZ3MpID09IDApCi0gICAgICAgICAgICBnb3RvIGNsb3Nl X3NyY19kZXNjOworICAgICAgICAgICAgeworICAgICAgICAgICAgICBmcHJpbnRmKHN0ZGVy ciwgImZjbG9uZWZpbGVhdCBzdWNjZWVkZWRcbiIpOworICAgICAgICAgICAgICBpZiAoIXgt PnByZXNlcnZlX3RpbWVzdGFtcHMpCisgICAgICAgICAgICAgICAgeworICAgICAgICAgICAg ICAgICAgc3RydWN0IHRpbWVzcGVjIHRpbWVzcGVjWzJdOworICAgICAgICAgICAgICAgICAg dGltZXNwZWNbMF0udHZfbnNlYyA9IHRpbWVzcGVjWzFdLnR2X25zZWMgPSBVVElNRV9OT1c7 CisgICAgICAgICAgICAgICAgICBmcHJpbnRmKHN0ZGVyciwgImNhbGxpbmcgdXRpbWVuc2F0 Li4uXG4iKTsKKyAgICAgICAgICAgICAgICAgIGlmICh1dGltZW5zYXQgKGRzdF9kaXJmZCwg ZHN0X3JlbG5hbWUsIHRpbWVzcGVjLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgQVRfU1lNTElOS19OT0ZPTExPVykKKyAgICAgICAgICAgICAgICAgICAgICAhPSAwKQor ICAgICAgICAgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgICAgICAgZXJyb3IgKDAs IGVycm5vLCBfKCJ1cGRhdGluZyB0aW1lcyBmb3IgJXMiKSwKKyAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgcXVvdGVhZiAoZHN0X25hbWUpKTsKKyAgICAgICAgICAgICAgICAgICAg ICByZXR1cm5fdmFsID0gZmFsc2U7CisgICAgICAgICAgICAgICAgICAgIH0KKyAgICAgICAg ICAgICAgICAgIGZwcmludGYoc3RkZXJyLCAidXRpbWVuc2F0IHN1Y2NlZWRlZFxuIik7Cisg ICAgICAgICAgICAgICAgfQorICAgICAgICAgICAgICBtb2RlX2FscmVhZHlfcHJlc2VydmVk ID0gKGZjX2ZsYWdzICYgQ0xPTkVfQUNMKSAhPSAwOworICAgICAgICAgICAgICBkZXN0X2Rl c2MgPSAtMTsKKyAgICAgICAgICAgICAgb21pdHRlZF9wZXJtaXNzaW9ucyA9IGRzdF9tb2Rl ICYgfmNsb25lZF9tb2RlOworICAgICAgICAgICAgICBleHRyYV9wZXJtaXNzaW9ucyA9IDA7 CisgICAgICAgICAgICAgIGdvdG8gc2V0X2Rlc3RfbW9kZTsKKyAgICAgICAgICAgIH0KICAg ICAgICAgICBlbHNlIGlmICghIGhhbmRsZV9jbG9uZV9mYWlsIChkc3RfZGlyZmQsIGRzdF9y ZWxuYW1lLCBzcmNfbmFtZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICBkc3RfbmFtZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAtMSwgZmFsc2UgLyogV2UgZGlkbid0IGNyZWF0ZSBkc3QgICovLApAQCAtMTQ4NSw5ICsx NTE0LDE0IEBAIGNvcHlfcmVnIChjaGFyIGNvbnN0ICpzcmNfbmFtZSwgY2hhciBjb25zdCAq ZHN0X25hbWUsCiAKICAgc2V0X2F1dGhvciAoZHN0X25hbWUsIGRlc3RfZGVzYywgc3JjX3Ni KTsKIAorI2lmIEhBVkVfRkNMT05FRklMRUFUICYmICFVU0VfWEFUVFIKK3NldF9kZXN0X21v ZGU6CisjZW5kaWYKICAgaWYgKHgtPnByZXNlcnZlX21vZGUgfHwgeC0+bW92ZV9tb2RlKQog ICAgIHsKLSAgICAgIGlmIChjb3B5X2FjbCAoc3JjX25hbWUsIHNvdXJjZV9kZXNjLCBkc3Rf bmFtZSwgZGVzdF9kZXNjLCBzcmNfbW9kZSkgIT0gMAorICAgICAgaWYgKCFtb2RlX2FscmVh ZHlfcHJlc2VydmVkCisgICAgICAgICAgJiYgKGNvcHlfYWNsIChzcmNfbmFtZSwgc291cmNl X2Rlc2MsIGRzdF9uYW1lLCBkZXN0X2Rlc2MsIHNyY19tb2RlKQorICAgICAgICAgICAgICAh PSAwKQogICAgICAgICAgICYmIHgtPnJlcXVpcmVfcHJlc2VydmUpCiAgICAgICAgIHJldHVy bl92YWwgPSBmYWxzZTsKICAgICB9CkBAIC0xNTE3LDcgKzE1NTEsNyBAQCBjb3B5X3JlZyAo Y2hhciBjb25zdCAqc3JjX25hbWUsIGNoYXIgY29uc3QgKmRzdF9uYW1lLAogICAgIH0KIAog Y2xvc2Vfc3JjX2FuZF9kc3RfZGVzYzoKLSAgaWYgKGNsb3NlIChkZXN0X2Rlc2MpIDwgMCkK KyAgaWYgKDAgPD0gZGVzdF9kZXNjICYmIGNsb3NlIChkZXN0X2Rlc2MpIDwgMCkKICAgICB7 CiAgICAgICBlcnJvciAoMCwgZXJybm8sIF8oImZhaWxlZCB0byBjbG9zZSAlcyIpLCBxdW90 ZWFmIChkc3RfbmFtZSkpOwogICAgICAgcmV0dXJuX3ZhbCA9IGZhbHNlOwotLSAKMi4zOS4x Cgo= --------------yWm5PBUttzQ4ouxbX70YEyUR--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 21:46:46 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 16:46:46 2023 Received: from localhost ([127.0.0.1]:38228 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQbED-0000ae-Pg for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 16:46:46 -0500 Received: from mail-wm1-f52.google.com ([209.85.128.52]:37666) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pQbEB-0000aQ-RJ for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 16:46:44 -0500 Received: by mail-wm1-f52.google.com with SMTP id bg13-20020a05600c3c8d00b003d9712b29d2so7197440wmb.2 for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 13:46:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=TH6sVE0OoLKYuuRSMeoOg+P31qyFQSRsKlcxn/73al4=; b=BRbQFzb0Eziudy+K2u17VN1f1b9ew151NCyh5IVnfz+3QNQvZQMAcj/ijONzY6t/9R BA4br7V6ohl1qOqeNBzRwd4sob1cczUWImgL/Jg5vtRAyiBcdrSgt4FAzyB8z2pagKyS IpV8VrSz3YqGnClneHoXpGBqSNQuPk4qZxHB8WgFawLp1DxM9tD5xCSog+9/S5VYwDzE W0FQqPnz3yiwc+G/4DfrYsT3+zt8Ywm34euoJTWgA6jkZQGNbOqdfuUYqKNbFAKSA02j Q36GSgOsn7h4oQA3hXjyQcvcdLAWGaJPELcBQ4Whdh2Q5W9pzLLtfw6Pt9k8di4kktLf LlAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TH6sVE0OoLKYuuRSMeoOg+P31qyFQSRsKlcxn/73al4=; b=fshM3Lk3rYiUlDPsG6ssARltQ9NCcHIYGSyjsuYPG98Flwi6Hf/1ak9pXLZ3o1FMpL BRtl9KqughYoAbg2s+eGH3uFJgMHRlXXTpWzW6uyEDxR2BH+SKJ7eoY9CELsQWLeVLJt riRMPZVcKEJZtQNYDrGmvtjbXEMSs5pz3ldf8wdzJqt608Ak2eZJ/wZYUAApjUEEiiPh HWntOLdCGt+oaG97QZormTIpa2zWWJjmLRkncJtBEbwFQiZX3K5wHeoGkSvgDF0Mr1r1 MrxZV+qMeftfPfZEPlZb9+fXNh12A5Pi11H8bA3SY3eXgUzuJH7S/qoG9tvH9tCxbBQw +pWA== X-Gm-Message-State: AO0yUKX3upnIUFTdrTBVlsxHTjnuEd15Hv/lSFx9zJ2Z0lD6dQVPSUsi /v/mNywayoz5dXshvOcDrbw= X-Google-Smtp-Source: AK7set8FQdbcEuFGWuOJi92LIsaDG1sTSkwixb4ammyo56TbMwvtlhvCJWkxJoIrmflYHOXvXF07og== X-Received: by 2002:a05:600c:319e:b0:3dc:506e:6559 with SMTP id s30-20020a05600c319e00b003dc506e6559mr13677036wmp.37.1676065596886; Fri, 10 Feb 2023 13:46:36 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id h4-20020a1ccc04000000b003dc4a47605fsm9645371wmb.8.2023.02.10.13.46.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 10 Feb 2023 13:46:36 -0800 (PST) Message-ID: <c60ab2bd-fff8-472b-fbe9-d9f0aa72eb37@HIDDEN> Date: Fri, 10 Feb 2023 21:46:35 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 10/02/2023 20:45, Paul Eggert wrote: > On 2/10/23 10:58, Pádraig Brady wrote: >> I was considering "touch"ing the timestamps after also, >> but it's better to just maintain them as we're >> pointing to the same data after all. > > For POSIX conformance we must touch if the user has specified only POSIX > options (and has not specified -p). > > And it's not just a POSIX conformance issue. Ordinary users will be > surprised if plain 'cp A B' creates a file B with a timestamp from last > year. Maybe. Though POSIX says cp "shall copy" and we're not making a copy, we're making a reflink. So technically we're violating POSIX already in that regard. One might take the view that the fact we write no new data means we should not update the data modification time etc. by default, and this may be more signal to a user that new data has not in fact been written. > Likewise for B's modes. Yes agreed on this. We'll have to honor umask etc. > There's another complication: recent macOS versions have CLONE_ACL, and > we're not using that. Oh right, we should consider that too. thanks, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 21:35:41 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 16:35:41 2023 Received: from localhost ([127.0.0.1]:38224 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQb3S-0000Kh-9q for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 16:35:41 -0500 Received: from mail-wr1-f41.google.com ([209.85.221.41]:39846) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <gvalkov@HIDDEN>) id 1pQb3B-0000KG-DQ for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 16:35:36 -0500 Received: by mail-wr1-f41.google.com with SMTP id a2so6319553wrd.6 for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 13:35:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1676064915; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=B3/l8ZP/b1QFcG2+/DaZJvU7CtJ2DhNMwopRvKTwRuM=; b=cQNPrATW1+L2bx9CbZ1TzpZCiGQSNhQwFS/SDks4ubR7iOpz00eJ4833T1wo+h50Hy X0IYIJumSQ2txzfQB2eXSVDqg4UV9Q/L+/zOUwQD+OdlpzRlSx4h26H4nkDD0uzYsIyv 0SU86UCEs+qj6AfwBeayUWjpFU9J7bLdNRZ/nGo/IifZMZSNVp2jd5c/e24wkDgQvytp oh/y1Ppf1tHmDVyQX6hMkoTEDgNuyvarZb+yemIMoXAR9QzKZaqvwlO9FXCpJSXTa+ET bwvUsbEFZq/guAhSaIPKVsmmr20PMkU66d3tUM8OcsTQbFJaNEfq+6E8llPNSIaei0to 77UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1676064915; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B3/l8ZP/b1QFcG2+/DaZJvU7CtJ2DhNMwopRvKTwRuM=; b=UpSqllIcLoZJ+A08GCA31w6kVWuxVoQEZV3jPlM2WnEkqWe6Wl3jM5A3P/UJhsDd65 MWmEj/oTXTWQOtHpciL/LLlGlmZlnkSKyFlA5xeY6pz0wESCf0vX/VfX/ZowizWwRyd5 YOMovgH6W4GhkDCqefwEz5DTd7Fe9N5PnfqAWqQcUfNkVZWY0+FcNSfr4WuzOaXBvaoC gxEx1DgpMcRUFxPn7vqzFABsamMz+wfRfrnNkgq8o9iGP6BtP2BsmsE5QQO9XJcr0kMg VKDvnqrRoGTLEdnExBwf+fZmvvXjp4Y1pUUAx9lw5hheyuFxYj3NiE7vpAkMSD+a6h+a 3mlA== X-Gm-Message-State: AO0yUKU9TovDc0Q8lrUbCvY9HWg6WbIbCP6YcWo7NrUZunuhtFtRwEaH ht/Dr2S+xT7McTo00g6Hh0o= X-Google-Smtp-Source: AK7set9TWJ+gNCSGj3CcZ8vPhz/TkK+2ziLOh6oaFt9t0dNuAey2kXS/0EXmzVDtzWgsB/eQREjOmQ== X-Received: by 2002:a05:6000:12cb:b0:2bf:e46c:8b89 with SMTP id l11-20020a05600012cb00b002bfe46c8b89mr11497397wrx.4.1676064915265; Fri, 10 Feb 2023 13:35:15 -0800 (PST) Received: from smtpclient.apple (212-39-89-169.ip.btc-net.bg. [212.39.89.169]) by smtp.gmail.com with ESMTPSA id m18-20020a5d6252000000b002c3dc4131f5sm4563512wrv.18.2023.02.10.13.35.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Feb 2023 13:35:14 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS From: George Valkov <gvalkov@HIDDEN> In-Reply-To: <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> Date: Fri, 10 Feb 2023 23:35:12 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <1D804E0D-EC28-4C3B-B765-C9CA8FF3E974@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> To: Paul Eggert <eggert@HIDDEN> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org, =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) > On 2023-02-10, at 10:45 PM, Paul Eggert <eggert@HIDDEN> wrote: >=20 > On 2/10/23 10:58, P=C3=A1draig Brady wrote: >> I was considering "touch"ing the timestamps after also, >> but it's better to just maintain them as we're >> pointing to the same data after all. >=20 > For POSIX conformance we must touch if the user has specified only = POSIX options (and has not specified -p). >=20 > And it's not just a POSIX conformance issue. Ordinary users will be = surprised if plain 'cp A B' creates a file B with a timestamp from last = year. >=20 > Likewise for B's modes. >=20 > There's another complication: recent macOS versions have CLONE_ACL, = and we're not using that. I personally prefer using CoW as default (no parameters), because it is = way more efficient: could save a lot of time and space, especially when long builds or large = files are involved. Of course I also understand that coreutils are expected to meet a = certain standard behaviour among all architectures. Since the source and it=E2=80=99s clone have = separate metadata, it should be possible to change it on the clone, to comply with = standards. On the other hand It feels more natural when the metadata is kept intact for the two = files. This is a good indication that no changes have occurred since the clone or copy, and = they are the same. This approach is also consistent with how macOS Finder copies files and = directories: all metadata is preserved. Here is a good example: I create a tarball on my OpenWRT router. Extract = on the Mac. Copy to the build directory and create new firmware. This preserves all = permissions and metadata. So there is no need for any special tools or command = arguments, it just works. In this scenario, the permissions are critical to preserve. Timestamp is = a good indication of when the file was last changed, setting a different value on the copy is = confusing and makes it very hard to track which files in a copied directory has changed = since. That=E2=80=99s just my personal opinion for default operation and may = not match standards, but it makes work natural and productive. Command line arguments should = be respected. I found this link in one of the mailing lists you sent, it explains what = was wrong with the original SEEK_DATA and SEEK_HOLE approach on macOS, and why we can=E2=80=99t relay on them: https://lists.gnu.org/archive/html/bug-gnulib/2018-09/msg00054.html Georgi Valkov httpstorm.com nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 20:45:25 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 15:45:25 2023 Received: from localhost ([127.0.0.1]:38211 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQaGr-0007Xe-Ff for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 15:45:25 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:41144) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pQaGo-0007XM-VT for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 15:45:24 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id D33C31600A9; Fri, 10 Feb 2023 12:45:16 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id MCUnXAClC5NB; Fri, 10 Feb 2023 12:45:16 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 2B5F71600AA; Fri, 10 Feb 2023 12:45:16 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 2B5F71600AA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676061916; bh=Eg3w2x711CK1J4RwhTCkrYp+bhOzFFwWc9W4DGeqCHA=; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type: Content-Transfer-Encoding; b=X2wLBXxtgKgTe2m4I53JKmx6rbc8el1nEi8uX7ZBdPoWm/12GsDUVYqSFDKsUTkwN V0ciafdL+jC7P9BHzgpDG13Fz+y6tkXUMzFcH2BmLbke62zUT1RKowe9/T7OP80E5S 3XBmKQymLO4DbL6hzZdAw1KmTDfcglE/03DCcVls= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id vm4yAecVZ-iW; Fri, 10 Feb 2023 12:45:16 -0800 (PST) Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 0B2D11600A9; Fri, 10 Feb 2023 12:45:16 -0800 (PST) Message-ID: <acbadedf-49de-3970-cb7b-84f6a4814bf8@HIDDEN> Date: Fri, 10 Feb 2023 12:45:10 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: =?UTF-8?Q?P=c3=a1draig_Brady?= <P@HIDDEN>, George Valkov <gvalkov@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN> <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) On 2/10/23 10:58, P=C3=A1draig Brady wrote: > I was considering "touch"ing the timestamps after also, > but it's better to just maintain them as we're > pointing to the same data after all. For POSIX conformance we must touch if the user has specified only POSIX=20 options (and has not specified -p). And it's not just a POSIX conformance issue. Ordinary users will be=20 surprised if plain 'cp A B' creates a file B with a timestamp from last=20 year. Likewise for B's modes. There's another complication: recent macOS versions have CLONE_ACL, and=20 we're not using that.
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 20:06:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 15:06:35 2023
Received: from localhost ([127.0.0.1]:38175 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQZfG-0006bb-IN
for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 15:06:35 -0500
Received: from mail-wr1-f48.google.com ([209.85.221.48]:45657)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pQZfD-0006bM-LI
for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 15:06:32 -0500
Received: by mail-wr1-f48.google.com with SMTP id h16so6102743wrz.12
for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 12:06:31 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20210112; t=1676059586;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:from:to:cc:subject:date:message-id:reply-to;
bh=Cb7F51dqwJOoHJhqMiMZf/9Wo/1ePy71HzmcUBebUKY=;
b=g1iNK/X0QXQB2k7iMEQKIHy2m2fiwKhhyPRKolH+l0yTg/hdU0tAJhwjM86/v36omk
ybj1729Np36UNbVqu0vwUYSJu6MNU1tAJVWBcHYpdbZBkutsDsCcr/KVkncDyqts5Usu
ANXXLaBlvF5E64bLv9Ih1dQdPAZ86ZUpPgo92Cy3uZdcmOS3cr6HdUZZ45H0lD/MyMQD
bsU8vncTVqddM0gJdyeIE3zQMklQu/eDOzGnS/NgvogVxVXylRmcczpKHhPYzN/uoD9f
65ddclt2NqMMLKKiffjIiStNMDkWE+ozDlvewDeUj8+HAPIJYfmFlWtTLnAtTaLn6CfJ
v1EQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112; t=1676059586;
h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
:from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=Cb7F51dqwJOoHJhqMiMZf/9Wo/1ePy71HzmcUBebUKY=;
b=gImnFa+knLeZL5bMjw3UgGmpjhsLud3oZFzwWvheFGusySRL9zbptR4Z+gocy9dfxd
j1I+tzCAvOMvm8GeE8Uo9ogPfJhKxevoUpGwUmIk/WlwgCID2yIf/JnPc+gTO66JZov+
BIcnXmols6bwE01JAA1KP16ShkF7gskHPoJv/YF0bR7gi2pbeoyUr4mMj1M/yahW+Xm8
1PS1OCmjxxOY6SUz25JuFyqfivY41fbZP6flsOdmlJFKouEKZtORpxMObxydQqfH3REg
cVudl16/L0R/fRs1tAvPF17M560Fo+9IVEAn9HXD1GGtrmNTYPY560xgaCtIJBTidA6Q
8UnQ==
X-Gm-Message-State: AO0yUKXgre/zfjfM6nucL92DqVpayP3LfHZaftqaz37UgZhbmDwGVlHv
n9QCd+RpYjp2QPB8RrPPKuM=
X-Google-Smtp-Source: AK7set8VdSVEMtpKfAfpevgXKkqKJCJ6MBbj/DIaqbfDcGaFnu7D+lMJLHKEpVY9yqIfBB2k3dm3dg==
X-Received: by 2002:a5d:4610:0:b0:2c5:4df4:685e with SMTP id
t16-20020a5d4610000000b002c54df4685emr35080wrq.0.1676059585587;
Fri, 10 Feb 2023 12:06:25 -0800 (PST)
Received: from smtpclient.apple (212-5-158-241.ip.btc-net.bg. [212.5.158.241])
by smtp.gmail.com with ESMTPSA id
q14-20020a5d574e000000b002bfb02153d1sm4427353wrw.45.2023.02.10.12.06.23
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Fri, 10 Feb 2023 12:06:24 -0800 (PST)
From: George Valkov <gvalkov@HIDDEN>
Message-Id: <F3D14ADC-0B93-4110-BF45-1BEBB1E3848D@HIDDEN>
Content-Type: multipart/mixed;
boundary="Apple-Mail=_21701657-1944-4F2F-B7D7-BA8F7488F004"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Date: Fri, 10 Feb 2023 22:06:22 +0200
In-Reply-To: <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
<2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
Cc: Paul Eggert <eggert@HIDDEN>, 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--Apple-Mail=_21701657-1944-4F2F-B7D7-BA8F7488F004
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
> On 2023-02-10, at 8:58 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 10/02/2023 17:24, George Valkov wrote:
>>> On 2023-02-10, at 4:02 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>>>=20
>>> On 10/02/2023 12:13, George Valkov wrote:
>>>>> On 2023-02-10, at 11:18 AM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>>>>>=20
>>>>> I'll apply the simple patch later I think.
>>>> I have an interesting idea: If I copy a large file, say the 16 GB =
disk image
>>>> where I compiled OpenWRT. So I copy this on the same filesystem, =
and check
>>>> disk usage before an after: no change. Also the copy is instant. =
That=E2=80=99s because
>>>> APFS supports copy-on-write. Initially both files share the same =
sectors on disk,
>>>> any changes made after that are copied to new sectors.
>>>> This is the most efficient way to copy files on APFS, and should =
produce no corruption.
>>>> Let=E2=80=99s implement it. I would assume there is a system cal =
that does the entire copy,
>>>> And we don=E2=80=99t need to read and write any data.
>>>> Does the trace contain any interesting calls that might be related =
to that?
>>>=20
>>> When you say "I copy a large file", is that with gcp or something =
else?
>> Finder: option + drag in the same directory
>>> Since coreutils 9.1 we try the CoW by default with fclonefileat()
>>> which is available since macOS 10.12 (2016).
>> I can confirms fclonefileat works on macOS 12.6.3 and solves the =
issue.
>> File attributes are also preserved. To clarify: these observations =
are from
>> the sample below. I don=E2=80=99t have internal experience with =
cloreutils, but I can
>> test it if you provide some description.
>>> Note that works only when src and dest are within the same file =
system,
>>> but that is the case for you if I'm reading your original report =
correctly.
>> Correct. fclonefileat only works on the same file system.
>> If we attempt to clone to another volume, fclonefileat fails 18 =
Cross-device link.
>> When building OpenWRT everything works on the same file system, so
>> fclonefileat is applicable.
>>> When I mentioned that earlier I thought your macOS version was too =
old to support that,
>>> but in fact your 12.6.3 should support fclonefileat() fine.
>> Thankfully it is supported. The latest security update came in =
January 2023.
>>> So that's another variable. Is that call failing completely for you?
>>> You might be quicker to add a couple of printfs around the
>>> fclonefileat() calls in the coreutils from latest git you compliled.
>>> Note there is new error handling related to that call in the latest =
git,
>>> compared to what was released in coreutils 9.1.
>> Coreutils tests on master
>> git clone git://git.savannah.gnu.org/coreutils.git
>> mv coreutils coreutils-clone
>> cd coreutils-clone
>> git submodule foreach git pull origin master
>> ./bootstrap
>> ./configure
>> make -j 16
>> git log
>> commit d374d32ccf12f8cf33c8f823d573498b7c8b27a4 (HEAD -> clone, =
origin/master, origin/HEAD, master)
>> cd ..
>> ./coreutils-clone/src/cp cc1 cc1-test
>> printf("HAVE_FCLONEFILEAT %u USE_XATTR %u\n", HAVE_FCLONEFILEAT, =
USE_XATTR);
>> HAVE_FCLONEFILEAT 1 USE_XATTR 0
>> int fc_flags =3D x->preserve_ownership ? 0 : CLONE_NOOWNERCOPY;
>> printf(
>> "data_copy_required %u x->reflink_mode =
%u\n"
>> "x->preserve_mode %u =
x->preserve_timestamps %u\n"
>> "x->preserve_ownership %u CLONE_NOOWNERCOPY =
%u\n",
>> data_copy_required, x->reflink_mode,
>> x->preserve_mode, x->preserve_timestamps,
>> x->preserve_ownership, CLONE_NOOWNERCOPY
>> );
>> if (data_copy_required && x->reflink_mode
>> && x->preserve_mode && x->preserve_timestamps
>> && (x->preserve_ownership || CLONE_NOOWNERCOPY))
>> {
>> int a =3D fclonefileat (source_desc, dst_dirfd, =
dst_relname, fc_flags);
>> int e =3D errno;
>> printf("fclonefileat %i %i\n", a, e);
>> if (a =3D=3D 0)
>> goto close_src_desc;
>> data_copy_required 1 x->reflink_mode 1
>> x->preserve_mode 0 x->preserve_timestamps 0
>> x->preserve_ownership 0 CLONE_NOOWNERCOPY 2
>> fclonefileat is not used because x->preserve_mode =3D 0, =
x->preserve_timestamps =3D 0
>> That design seems wrong. The fact than no one requested to preserve =
that data,
>> doesn=E2=80=99t mean preserving it via fclonefileat is a bad thing. I =
think we should
>> always call fclonefileat and proceed to other means if it fails. It =
is more efficient, since
>> it requires no additional space or data to be copied. And always =
produces a valid copy.
>>> Note also I don't see any fclonefileat() syscalls in your Finder =
logs at least.
>> Great news: fclonefileat works on the same file system, and if we
>> need to copy outside, then just disable sparse copy according to my =
patch.
>> One option is to start blindly with fclonefileat, and if that fails, =
fall back to
>> normal copy. If you can use the trace to find what Finder is doing, =
we can try it.
>> It might bring some improvements. Else, disable sparse and do a =
regular copy.
>> It was easier for me to write a small sample:
>> // clone.c
>> #include <stdio.h>
>> #include <fcntl.h>
>> #include <unistd.h>
>> #include <sys/clonefile.h>
>> int main(int argc, char ** argv)
>> {
>> int fd =3D open("cc1", O_RDONLY);
>> int dir =3D open("./", O_RDONLY);
>> int a =3D fclonefileat(fd, dir, "cc1-clone", 0);
>> close(fd);
>> close(dir);
>> printf("fd %i dir %i fclonefileat %i\n", fd, dir, a);
>> return 0;
>> }
>> ./coreutils/src/cp cc1 cc1-ori-before
>> gcc clone.c && ./a.out
>> ./coreutils/src/cp cc1 cc1-ori-after
>> ll cc1*
>> -rwxr-xr-x 1 g staff 27551296 Feb 10 17:35 cc1*
>> -rwxr-xr-x 1 g staff 27551296 Feb 10 17:35 cc1-clone*
>> -rwxr-xr-x 1 g staff 27551296 Feb 10 17:39 cc1-ori-after*
>> -rwxr-xr-x 1 g staff 27551296 Feb 10 17:39 cc1-ori-before*
>> sha1sum
>> 812cb0591b000f56bdf75c0f3f94c622e371b970 cc1
>> 812cb0591b000f56bdf75c0f3f94c622e371b970 cc1-clone
>> 812cb0591b000f56bdf75c0f3f94c622e371b970 cc1-ori-after
>> 7b447132f56f3b549ef62a4780945e82d26f7d2c cc1-ori-before
>> df -h
>> Filesystem Size Used Avail Capacity iused ifree %iused =
Mounted on
>> /dev/disk1s1 1.1Ti 1.0Ti 58Gi 95% 2400403 605165920 0% =
/System/Volumes/Data
>> # clone a large file 15 GB using fclonefileat
>> ./a.out
>> # copy a large file 15 GB using Finder: option + drag in the same =
directory
>> # both complete instantly
>> df -h
>> Filesystem Size Used Avail Capacity iused ifree %iused =
Mounted on
>> /dev/disk1s1 1.1Ti 1.0Ti 58Gi 95% 2400401 605207240 0% =
/System/Volumes/Data
>> ll -h /Users/g/vhd/OpenWRT.wrt3200*
>> -rw-r--r--@ 1 g staff 15G Feb 10 04:06 =
/Users/g/vhd/OpenWRT.wrt3200-clone.sparseimage
>> -rw-r--r--@ 1 g staff 15G Feb 10 04:06 =
/Users/g/vhd/OpenWRT.wrt3200-finder.sparseimage
>> -rw-r--r--@ 1 g staff 15G Feb 10 04:06 =
/Users/g/vhd/OpenWRT.wrt3200.sparseimage
>=20
> Cool that clears up the fclonefileat() questions.
> Attached is a patch to attempt the clone unless
> we've explicitly specified --no-preserve=3Dmode.
> I was considering "touch"ing the timestamps after also,
> but it's better to just maintain them as we're
> pointing to the same data after all.
>=20
> So with this patch we should operate quickly within a file system.
> With your original patch we should operate robustly otherwise.
Thank you, P=C3=A1driag! Nice teamwork. I added x-> and Tested-by to =
your patch.
The copy was instant and accurate. Both apply to cc1 as well as the 15 =
GB file.
Please let me know when there is a new release, so I can push it to =
OpenWRT.
--Apple-Mail=_21701657-1944-4F2F-B7D7-BA8F7488F004
Content-Disposition: attachment;
filename=0001-copy-on-macOS-try-COW-even-if-not-preserving-mode.patch
Content-Type: application/octet-stream;
x-unix-mode=0644;
name="0001-copy-on-macOS-try-COW-even-if-not-preserving-mode.patch"
Content-Transfer-Encoding: quoted-printable
=46rom=209470879845ae622bda07ff55d158ef212ac1fd83=20Mon=20Sep=2017=20=
00:00:00=202001=0AFrom:=20=3D?UTF-8?q?P=3DC3=3DA1draig=3D20Brady?=3D=20=
<P@HIDDEN>=0ADate:=20Fri,=2010=20Feb=202023=2018:48:05=20+0000=0A=
Subject:=20[PATCH]=20copy:=20on=20macOS=20try=20COW=20even=20if=20not=20=
preserving=20mode=0A=0A*=20src/copy.c=20(copy_reg):=20Try=20=
fclonefileat()=20unless=20we're=0Aexplicitly=20not=20preserving=20mode.=20=
=20Also=20don't=20key=20on=20timestamp=0Apreservation,=20as=20with=20COW=20=
it's=20more=20accurate=20anyway=20to=20keep=0Athe=20same=20timestamps=20=
as=20we're=20pointing=20to=20the=20same=20data.=0A*=20NEWS:=20Mention=20=
the=20change=20in=20behavior.=0AAddresses=20https://bugs.gnu.org/61386=0A=
=0ATested-by:=20Georgi=20Valkov=20<gvalkov@HIDDEN>=0A---=0A=20=
src/copy.c=20|=204=20+++-=0A=201=20file=20changed,=203=20insertions(+),=20=
1=20deletion(-)=0A=0Adiff=20--git=20a/src/copy.c=20b/src/copy.c=0Aindex=20=
dfbb557de..b1fa6e60a=20100644=0A---=20a/src/copy.c=0A+++=20b/src/copy.c=0A=
@@=20-1250,9=20+1250,11=20@@=20copy_reg=20(char=20const=20*src_name,=20=
char=20const=20*dst_name,=0A=20#=20endif=0A=20=20=20=20=20=20=20int=20=
fc_flags=20=3D=20x->preserve_ownership=20?=200=20:=20CLONE_NOOWNERCOPY;=0A=
=20=20=20=20=20=20=20if=20(data_copy_required=20&&=20x->reflink_mode=0A-=20=
=20=20=20=20=20=20=20=20=20&&=20x->preserve_mode=20&&=20=
x->preserve_timestamps=0A+=20=20=20=20=20=20=20=20=20=20&&=20!=20=
x->explicit_no_preserve_mode=0A=20=20=20=20=20=20=20=20=20=20=20&&=20=
(x->preserve_ownership=20||=20CLONE_NOOWNERCOPY))=0A=20=20=20=20=20=20=20=
=20=20{=0A+=20=20=20=20=20=20=20=20=20=20/*=20Note=20timestamps=20will=20=
be=20preserved=20irrespective=20of=0A+=20=20=20=20=20=20=20=20=20=20=20=20=
=20x->preserve_timestamps.=20*/=0A=20=20=20=20=20=20=20=20=20=20=20if=20=
(fclonefileat=20(source_desc,=20dst_dirfd,=20dst_relname,=20fc_flags)=20=
=3D=3D=200)=0A=20=20=20=20=20=20=20=20=20=20=20=20=20goto=20=
close_src_desc;=0A=20=20=20=20=20=20=20=20=20=20=20else=20if=20(!=20=
handle_clone_fail=20(dst_dirfd,=20dst_relname,=20src_name,=0A--=20=0A=
2.39.1=0A=0A=
--Apple-Mail=_21701657-1944-4F2F-B7D7-BA8F7488F004
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=us-ascii
./coreutils-clone/src/cp cc1 cc1-test
c9b79253a4b690c7be03ebc3187f1f09834f7472 cc1
c9b79253a4b690c7be03ebc3187f1f09834f7472 cc1-test
-rwxr-xr-x 1 g staff 27551296 Feb 10 21:45 cc1*
-rwxr-xr-x 1 g staff 27551296 Feb 10 21:45 cc1-test*
Build log in case you feel like fixing unrelated sprintf deprecated =
warnings later
to improve quality.
=
https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/build-l=
og-d374d32ccf12f8cf33c8f823d573498b7c8b27a4.txt
Cheers!
Georgi Valkov
httpstorm.com
nano RTOS
--Apple-Mail=_21701657-1944-4F2F-B7D7-BA8F7488F004--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 18:58:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 13:58:43 2023
Received: from localhost ([127.0.0.1]:38150 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQYba-0004qo-BE
for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 13:58:42 -0500
Received: from mail-wr1-f47.google.com ([209.85.221.47]:44628)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <pixelbeat@HIDDEN>) id 1pQYbY-0004qa-MN
for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 13:58:41 -0500
Received: by mail-wr1-f47.google.com with SMTP id bk16so5963608wrb.11
for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 10:58:40 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=in-reply-to:from:references:cc:to:content-language:subject
:user-agent:mime-version:date:message-id:sender:from:to:cc:subject
:date:message-id:reply-to;
bh=n+fWNuQDhmwcbY/yjhUkCh2QBP6KYpV8+AC+Y58zpOE=;
b=mk62sGA/Xf89T4XJXU7lN9/S82oDTF2lbhgDMeK/Iu+RltL+W4DPs9xSXzwlgf9BiS
T4p7SlKPmX1b8CMYaF0S/oRxLO4JKlzxTy1T6NxmMfrwkIWMZ443A0T1b+HC2JM0IBSS
cR5u0CGlYTM8yXAypfH1DTgdW4umK0ytXPXI+0iu7HdIQSFdw9Rik9UxZSy0nWL8mDlg
tlNxX7YuWVMUZ9YmhSfIsvUgw9EyvnoOiSocMMrTfVF/s/ionS2R1CRlksdwY4Q5QvTf
+oX8FctgltiJ7qcEBI7uMzD/h5xIsLxDgRITQ9My+SUVRlTlkrtagw8zbKVgbqDjxWWh
Tn/A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=in-reply-to:from:references:cc:to:content-language:subject
:user-agent:mime-version:date:message-id:sender:x-gm-message-state
:from:to:cc:subject:date:message-id:reply-to;
bh=n+fWNuQDhmwcbY/yjhUkCh2QBP6KYpV8+AC+Y58zpOE=;
b=1TZiclZ5PDSRp97xpBx8xd9kxt/rA5qBJ9ByLQOKBZy/sjFXVRs1GH7qXZ3v1fTgqC
IwpkVzj/SGpx8yjawxM5g6GHZ7k6H5RqyjZg8XangjrQB0HihDyWYzvOz8bNp/achWrY
L6YysfROWe4x6zshK7fUPunu/1zOQWhu+EOFuBope/bLHILL0U/rmEJ9VWSVM/f48IyN
CiAdT7OP2D3STcQnx0zcl3yDuBuraN+mthRbOxLDPOOADYPem7Tc0MH9QrTn3AZwO5PZ
D4ZKob2ev2mnPTz5lRlirKrwrAUEQmzpM+AgfKVLOMqIAJw3A6I4kZCR0lCQZMObKiPu
qlsw==
X-Gm-Message-State: AO0yUKU060QC4kcWsfPn8W7wGAFzLAL0OsrcELzuMEb2iDjQRCReRrdy
pXHytKllYl13FbW07bexlxY=
X-Google-Smtp-Source: AK7set8ZtW24RfH0c7Hye0SXkERp0gjDlwHF9tMixZmuANY1gdLn3mmpn9ISXx5pftXKq6/aUeqNhA==
X-Received: by 2002:adf:e408:0:b0:2bc:7fdd:9245 with SMTP id
g8-20020adfe408000000b002bc7fdd9245mr15692503wrm.5.1676055514784;
Fri, 10 Feb 2023 10:58:34 -0800 (PST)
Received: from [192.168.1.9]
(95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175])
by smtp.googlemail.com with ESMTPSA id
l3-20020a5d6683000000b002bdf3809f59sm4326909wru.38.2023.02.10.10.58.33
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Fri, 10 Feb 2023 10:58:34 -0800 (PST)
Content-Type: multipart/mixed; boundary="------------UDUlPbdePGYC8HFNqbhgB5Uo"
Message-ID: <2907ea4c-874b-c012-c02a-6fc2f97556f0@HIDDEN>
Date: Fri, 10 Feb 2023 18:58:33 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101
Thunderbird/109.0
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Content-Language: en-US
To: George Valkov <gvalkov@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
<6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
In-Reply-To: <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
X-Spam-Score: -0.9 (/)
X-Debbugs-Envelope-To: 61386
Cc: Paul Eggert <eggert@HIDDEN>, 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.9 (-)
This is a multi-part message in MIME format.
--------------UDUlPbdePGYC8HFNqbhgB5Uo
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
On 10/02/2023 17:24, George Valkov wrote:
>
>> On 2023-02-10, at 4:02 PM, Pádraig Brady <P@HIDDEN> wrote:
>>
>> On 10/02/2023 12:13, George Valkov wrote:
>>>> On 2023-02-10, at 11:18 AM, Pádraig Brady <P@HIDDEN> wrote:
>>>>
>>>> I'll apply the simple patch later I think.
>>> I have an interesting idea: If I copy a large file, say the 16 GB disk image
>>> where I compiled OpenWRT. So I copy this on the same filesystem, and check
>>> disk usage before an after: no change. Also the copy is instant. That’s because
>>> APFS supports copy-on-write. Initially both files share the same sectors on disk,
>>> any changes made after that are copied to new sectors.
>>> This is the most efficient way to copy files on APFS, and should produce no corruption.
>>> Let’s implement it. I would assume there is a system cal that does the entire copy,
>>> And we don’t need to read and write any data.
>>> Does the trace contain any interesting calls that might be related to that?
>>
>> When you say "I copy a large file", is that with gcp or something else?
>
> Finder: option + drag in the same directory
>
>> Since coreutils 9.1 we try the CoW by default with fclonefileat()
>> which is available since macOS 10.12 (2016).
>
> I can confirms fclonefileat works on macOS 12.6.3 and solves the issue.
> File attributes are also preserved. To clarify: these observations are from
> the sample below. I don’t have internal experience with cloreutils, but I can
> test it if you provide some description.
>
>
>> Note that works only when src and dest are within the same file system,
>> but that is the case for you if I'm reading your original report correctly.
>
> Correct. fclonefileat only works on the same file system.
> If we attempt to clone to another volume, fclonefileat fails 18 Cross-device link.
> When building OpenWRT everything works on the same file system, so
> fclonefileat is applicable.
>
>
>> When I mentioned that earlier I thought your macOS version was too old to support that,
>> but in fact your 12.6.3 should support fclonefileat() fine.
>
> Thankfully it is supported. The latest security update came in January 2023.
>
>
>> So that's another variable. Is that call failing completely for you?
>> You might be quicker to add a couple of printfs around the
>> fclonefileat() calls in the coreutils from latest git you compliled.
>> Note there is new error handling related to that call in the latest git,
>> compared to what was released in coreutils 9.1.
>
> Coreutils tests on master
> git clone git://git.savannah.gnu.org/coreutils.git
> mv coreutils coreutils-clone
> cd coreutils-clone
> git submodule foreach git pull origin master
> ./bootstrap
> ./configure
> make -j 16
> git log
> commit d374d32ccf12f8cf33c8f823d573498b7c8b27a4 (HEAD -> clone, origin/master, origin/HEAD, master)
> cd ..
>
> ./coreutils-clone/src/cp cc1 cc1-test
>
> printf("HAVE_FCLONEFILEAT %u USE_XATTR %u\n", HAVE_FCLONEFILEAT, USE_XATTR);
> HAVE_FCLONEFILEAT 1 USE_XATTR 0
>
> int fc_flags = x->preserve_ownership ? 0 : CLONE_NOOWNERCOPY;
> printf(
> "data_copy_required %u x->reflink_mode %u\n"
> "x->preserve_mode %u x->preserve_timestamps %u\n"
> "x->preserve_ownership %u CLONE_NOOWNERCOPY %u\n",
> data_copy_required, x->reflink_mode,
> x->preserve_mode, x->preserve_timestamps,
> x->preserve_ownership, CLONE_NOOWNERCOPY
> );
> if (data_copy_required && x->reflink_mode
> && x->preserve_mode && x->preserve_timestamps
> && (x->preserve_ownership || CLONE_NOOWNERCOPY))
> {
> int a = fclonefileat (source_desc, dst_dirfd, dst_relname, fc_flags);
> int e = errno;
> printf("fclonefileat %i %i\n", a, e);
> if (a == 0)
> goto close_src_desc;
>
> data_copy_required 1 x->reflink_mode 1
> x->preserve_mode 0 x->preserve_timestamps 0
> x->preserve_ownership 0 CLONE_NOOWNERCOPY 2
>
> fclonefileat is not used because x->preserve_mode = 0, x->preserve_timestamps = 0
> That design seems wrong. The fact than no one requested to preserve that data,
> doesn’t mean preserving it via fclonefileat is a bad thing. I think we should
> always call fclonefileat and proceed to other means if it fails. It is more efficient, since
> it requires no additional space or data to be copied. And always produces a valid copy.
>
>
>> Note also I don't see any fclonefileat() syscalls in your Finder logs at least.
>
> Great news: fclonefileat works on the same file system, and if we
> need to copy outside, then just disable sparse copy according to my patch.
> One option is to start blindly with fclonefileat, and if that fails, fall back to
> normal copy. If you can use the trace to find what Finder is doing, we can try it.
> It might bring some improvements. Else, disable sparse and do a regular copy.
>
> It was easier for me to write a small sample:
>
> // clone.c
> #include <stdio.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <sys/clonefile.h>
>
> int main(int argc, char ** argv)
> {
> int fd = open("cc1", O_RDONLY);
> int dir = open("./", O_RDONLY);
> int a = fclonefileat(fd, dir, "cc1-clone", 0);
> close(fd);
> close(dir);
>
> printf("fd %i dir %i fclonefileat %i\n", fd, dir, a);
>
> return 0;
> }
>
> ./coreutils/src/cp cc1 cc1-ori-before
> gcc clone.c && ./a.out
> ./coreutils/src/cp cc1 cc1-ori-after
>
> ll cc1*
> -rwxr-xr-x 1 g staff 27551296 Feb 10 17:35 cc1*
> -rwxr-xr-x 1 g staff 27551296 Feb 10 17:35 cc1-clone*
> -rwxr-xr-x 1 g staff 27551296 Feb 10 17:39 cc1-ori-after*
> -rwxr-xr-x 1 g staff 27551296 Feb 10 17:39 cc1-ori-before*
>
> sha1sum
> 812cb0591b000f56bdf75c0f3f94c622e371b970 cc1
> 812cb0591b000f56bdf75c0f3f94c622e371b970 cc1-clone
> 812cb0591b000f56bdf75c0f3f94c622e371b970 cc1-ori-after
> 7b447132f56f3b549ef62a4780945e82d26f7d2c cc1-ori-before
>
> df -h
> Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
> /dev/disk1s1 1.1Ti 1.0Ti 58Gi 95% 2400403 605165920 0% /System/Volumes/Data
>
> # clone a large file 15 GB using fclonefileat
> ./a.out
>
> # copy a large file 15 GB using Finder: option + drag in the same directory
> # both complete instantly
>
> df -h
> Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
> /dev/disk1s1 1.1Ti 1.0Ti 58Gi 95% 2400401 605207240 0% /System/Volumes/Data
>
> ll -h /Users/g/vhd/OpenWRT.wrt3200*
> -rw-r--r--@ 1 g staff 15G Feb 10 04:06 /Users/g/vhd/OpenWRT.wrt3200-clone.sparseimage
> -rw-r--r--@ 1 g staff 15G Feb 10 04:06 /Users/g/vhd/OpenWRT.wrt3200-finder.sparseimage
> -rw-r--r--@ 1 g staff 15G Feb 10 04:06 /Users/g/vhd/OpenWRT.wrt3200.sparseimage
Cool that clears up the fclonefileat() questions.
Attached is a patch to attempt the clone unless
we've explicitly specified --no-preserve=mode.
I was considering "touch"ing the timestamps after also,
but it's better to just maintain them as we're
pointing to the same data after all.
So with this patch we should operate quickly within a file system.
With your original patch we should operate robustly otherwise.
cheers,
Pádraig
--------------UDUlPbdePGYC8HFNqbhgB5Uo
Content-Type: text/x-patch; charset=UTF-8; name="macos-more-cow.patch"
Content-Disposition: attachment; filename="macos-more-cow.patch"
Content-Transfer-Encoding: base64
RnJvbSBmYzRmZmIzMmE4NjhlNmRiOGM4MDRiN2U1NmUzMTEwMjVmNDVmNmU3IE1vbiBTZXAg
MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/UD1DMz1BMWRyYWlnPTIwQnJhZHk/
PSA8UEBkcmFpZ0JyYWR5LmNvbT4KRGF0ZTogRnJpLCAxMCBGZWIgMjAyMyAxODo0ODowNSAr
MDAwMApTdWJqZWN0OiBbUEFUQ0hdIGNvcHk6IG9uIG1hY09TIHRyeSBDT1cgZXZlbiBpZiBu
b3QgcHJlc2VydmluZyBtb2RlCgoqIHNyYy9jb3B5LmMgKGNvcHlfcmVnKTogVHJ5IGZjbG9u
ZWZpbGVhdCgpIHVubGVzcyB3ZSdyZQpleHBsaWNpdGx5IG5vdCBwcmVzZXJ2aW5nIG1vZGUu
ICBBbHNvIGRvbid0IGtleSBvbiB0aW1lc3RhbXAKcHJlc2VydmF0aW9uLCBhcyB3aXRoIENP
VyBpdCdzIG1vcmUgYWNjdXJhdGUgYW55d2F5IHRvIGtlZXAKdGhlIHNhbWUgdGltZXN0YW1w
cyBhcyB3ZSdyZSBwb2ludGluZyB0byB0aGUgc2FtZSBkYXRhLgoqIE5FV1M6IE1lbnRpb24g
dGhlIGNoYW5nZSBpbiBiZWhhdmlvci4KQWRkcmVzc2VzIGh0dHBzOi8vYnVncy5nbnUub3Jn
LzYxMzg2Ci0tLQogc3JjL2NvcHkuYyB8IDQgKysrLQogMSBmaWxlIGNoYW5nZWQsIDMgaW5z
ZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQoKZGlmZiAtLWdpdCBhL3NyYy9jb3B5LmMgYi9z
cmMvY29weS5jCmluZGV4IGRmYmI1NTdkZS4uMWIyNjMxYzc4IDEwMDY0NAotLS0gYS9zcmMv
Y29weS5jCisrKyBiL3NyYy9jb3B5LmMKQEAgLTEyNTAsOSArMTI1MCwxMSBAQCBjb3B5X3Jl
ZyAoY2hhciBjb25zdCAqc3JjX25hbWUsIGNoYXIgY29uc3QgKmRzdF9uYW1lLAogIyBlbmRp
ZgogICAgICAgaW50IGZjX2ZsYWdzID0geC0+cHJlc2VydmVfb3duZXJzaGlwID8gMCA6IENM
T05FX05PT1dORVJDT1BZOwogICAgICAgaWYgKGRhdGFfY29weV9yZXF1aXJlZCAmJiB4LT5y
ZWZsaW5rX21vZGUKLSAgICAgICAgICAmJiB4LT5wcmVzZXJ2ZV9tb2RlICYmIHgtPnByZXNl
cnZlX3RpbWVzdGFtcHMKKyAgICAgICAgICAmJiAhIGV4cGxpY2l0X25vX3ByZXNlcnZlX21v
ZGUKICAgICAgICAgICAmJiAoeC0+cHJlc2VydmVfb3duZXJzaGlwIHx8IENMT05FX05PT1dO
RVJDT1BZKSkKICAgICAgICAgeworICAgICAgICAgIC8qIE5vdGUgdGltZXN0YW1wcyB3aWxs
IGJlIHByZXNlcnZlZCBpcnJlc3BlY3RpdmUgb2YKKyAgICAgICAgICAgICB4LT5wcmVzZXJ2
ZV90aW1lc3RhbXBzLiAqLwogICAgICAgICAgIGlmIChmY2xvbmVmaWxlYXQgKHNvdXJjZV9k
ZXNjLCBkc3RfZGlyZmQsIGRzdF9yZWxuYW1lLCBmY19mbGFncykgPT0gMCkKICAgICAgICAg
ICAgIGdvdG8gY2xvc2Vfc3JjX2Rlc2M7CiAgICAgICAgICAgZWxzZSBpZiAoISBoYW5kbGVf
Y2xvbmVfZmFpbCAoZHN0X2RpcmZkLCBkc3RfcmVsbmFtZSwgc3JjX25hbWUsCi0tIAoyLjI2
LjIKCg==
--------------UDUlPbdePGYC8HFNqbhgB5Uo--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 17:24:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 12:24:16 2023
Received: from localhost ([127.0.0.1]:38026 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQX8B-0002B5-ML
for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 12:24:16 -0500
Received: from mail-ej1-f44.google.com ([209.85.218.44]:44738)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pQX88-0002Ap-3D
for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 12:24:14 -0500
Received: by mail-ej1-f44.google.com with SMTP id hx15so17682889ejc.11
for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 09:24:12 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=tf2YE69EmpVDvSsA+wRok1wKhdHTAIjSfwb2GPk0K6I=;
b=AQB4Un5spTyZ/XI6eZ70rHInEmcfwoYN5b4G1JKcZoT/kxFYEG8xQNf9SmbJ1qc2wh
hKGblF+nljOsAvoS4JetfiALy1OLTeO8f0JEfvEemKZ9tc57eLgvmHxO0lN/lJLZz2j7
srPjbKUv044er7BKMbXLSPuKnDGThXRRr0T0srK6qLwWNPim61jboN0ntZA7S4SvFegG
QszfaOgy6JdzGzjk7sy0bfnYgU9Nna8AiiC4sAQfSTIqLmwW4MVpeB6rKu+nR1KYgdxJ
TJ2ecHtIo1DMLpWtf/NvBZUdqgLRdtIwv4xO09JuappSLFCh9G4PHvTyoSRBp9Q1pve9
pUJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=tf2YE69EmpVDvSsA+wRok1wKhdHTAIjSfwb2GPk0K6I=;
b=Uekk/BJwq88ieNOMm7mk97fP6cuIJHADkftL0n6zWuEUATgIc0XzRsBhQ9kpeRqsic
f/tiej62DZDXVAbtUQmMpmPZu8lTPoR5LhnnkXwxV1ES25GzqfWR4XPRyMAWlEnkcpZj
p3eDl3qhqVsphI0hC/ON/+X64DQrM0urUya3BDqd+SPMwgHAm2czZINFoGa9+o4M1SJ1
VBRZoF2Xx+j6NpAvBlVARqjFXVQbpDYEmRLI1QiVRKzH1CqaC2f0sJqp8NW/sZB52ztj
Sv2gVTvBGn7PDBDpzUBxyLtxfQGEMB/xKhRO93VIExYVW7nPZde2eGsahHlkTg7Bc0dS
h5Pg==
X-Gm-Message-State: AO0yUKVrqASYO24H4RW+kfrWh8Gs1mlhP+MbUE52EO5jojGUtSMt1ne+
RhKvdhMxBsizesKtKrWXq0w=
X-Google-Smtp-Source: AK7set9+1hk9kprX/A9ypYswPEIv6x41+L4Oupb5b9dpqTA5T33opBh6rj6iyR7m59Fi4KlcgTJwEA==
X-Received: by 2002:a17:906:2098:b0:889:3d10:ae93 with SMTP id
24-20020a170906209800b008893d10ae93mr15893592ejq.6.1676049845781;
Fri, 10 Feb 2023 09:24:05 -0800 (PST)
Received: from smtpclient.apple ([2a01:5a8:200:e28e:5d5e:cce3:c85a:54d9])
by smtp.gmail.com with ESMTPSA id
ov38-20020a170906fc2600b008af574e95d7sm802863ejb.27.2023.02.10.09.24.04
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Fri, 10 Feb 2023 09:24:04 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
Date: Fri, 10 Feb 2023 19:24:02 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <6878AF58-2F0E-4EE5-A72E-55263D45A8CF@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
<4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
<7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
Cc: Paul Eggert <eggert@HIDDEN>, 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
> On 2023-02-10, at 4:02 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 10/02/2023 12:13, George Valkov wrote:
>>> On 2023-02-10, at 11:18 AM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>>>=20
>>> I'll apply the simple patch later I think.
>> I have an interesting idea: If I copy a large file, say the 16 GB =
disk image
>> where I compiled OpenWRT. So I copy this on the same filesystem, and =
check
>> disk usage before an after: no change. Also the copy is instant. =
That=E2=80=99s because
>> APFS supports copy-on-write. Initially both files share the same =
sectors on disk,
>> any changes made after that are copied to new sectors.
>> This is the most efficient way to copy files on APFS, and should =
produce no corruption.
>> Let=E2=80=99s implement it. I would assume there is a system cal that =
does the entire copy,
>> And we don=E2=80=99t need to read and write any data.
>> Does the trace contain any interesting calls that might be related to =
that?
>=20
> When you say "I copy a large file", is that with gcp or something =
else?
Finder: option + drag in the same directory
> Since coreutils 9.1 we try the CoW by default with fclonefileat()
> which is available since macOS 10.12 (2016).
I can confirms fclonefileat works on macOS 12.6.3 and solves the issue.
File attributes are also preserved. To clarify: these observations are =
from
the sample below. I don=E2=80=99t have internal experience with =
cloreutils, but I can
test it if you provide some description.
> Note that works only when src and dest are within the same file =
system,
> but that is the case for you if I'm reading your original report =
correctly.
Correct. fclonefileat only works on the same file system.
If we attempt to clone to another volume, fclonefileat fails 18 =
Cross-device link.
When building OpenWRT everything works on the same file system, so
fclonefileat is applicable.
> When I mentioned that earlier I thought your macOS version was too old =
to support that,
> but in fact your 12.6.3 should support fclonefileat() fine.
Thankfully it is supported. The latest security update came in January =
2023.
> So that's another variable. Is that call failing completely for you?
> You might be quicker to add a couple of printfs around the
> fclonefileat() calls in the coreutils from latest git you compliled.
> Note there is new error handling related to that call in the latest =
git,
> compared to what was released in coreutils 9.1.
Coreutils tests on master
git clone git://git.savannah.gnu.org/coreutils.git
mv coreutils coreutils-clone
cd coreutils-clone
git submodule foreach git pull origin master
./bootstrap
./configure
make -j 16
git log
commit d374d32ccf12f8cf33c8f823d573498b7c8b27a4 (HEAD -> clone, =
origin/master, origin/HEAD, master)
cd ..
./coreutils-clone/src/cp cc1 cc1-test
printf("HAVE_FCLONEFILEAT %u USE_XATTR %u\n", HAVE_FCLONEFILEAT, =
USE_XATTR);
HAVE_FCLONEFILEAT 1 USE_XATTR 0
int fc_flags =3D x->preserve_ownership ? 0 : CLONE_NOOWNERCOPY;
printf(
"data_copy_required %u x->reflink_mode =
%u\n"
"x->preserve_mode %u =
x->preserve_timestamps %u\n"
"x->preserve_ownership %u CLONE_NOOWNERCOPY =
%u\n",
data_copy_required, x->reflink_mode,
x->preserve_mode, x->preserve_timestamps,
x->preserve_ownership, CLONE_NOOWNERCOPY
);
if (data_copy_required && x->reflink_mode
&& x->preserve_mode && x->preserve_timestamps
&& (x->preserve_ownership || CLONE_NOOWNERCOPY))
{
int a =3D fclonefileat (source_desc, dst_dirfd, =
dst_relname, fc_flags);
int e =3D errno;
printf("fclonefileat %i %i\n", a, e);
if (a =3D=3D 0)
goto close_src_desc;
data_copy_required 1 x->reflink_mode 1
x->preserve_mode 0 x->preserve_timestamps 0
x->preserve_ownership 0 CLONE_NOOWNERCOPY 2
fclonefileat is not used because x->preserve_mode =3D 0, =
x->preserve_timestamps =3D 0
That design seems wrong. The fact than no one requested to preserve that =
data,
doesn=E2=80=99t mean preserving it via fclonefileat is a bad thing. I =
think we should
always call fclonefileat and proceed to other means if it fails. It is =
more efficient, since
it requires no additional space or data to be copied. And always =
produces a valid copy.
> Note also I don't see any fclonefileat() syscalls in your Finder logs =
at least.
Great news: fclonefileat works on the same file system, and if we
need to copy outside, then just disable sparse copy according to my =
patch.
One option is to start blindly with fclonefileat, and if that fails, =
fall back to
normal copy. If you can use the trace to find what Finder is doing, we =
can try it.
It might bring some improvements. Else, disable sparse and do a regular =
copy.
It was easier for me to write a small sample:
// clone.c
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/clonefile.h>
int main(int argc, char ** argv)
{
int fd =3D open("cc1", O_RDONLY);
int dir =3D open("./", O_RDONLY);
int a =3D fclonefileat(fd, dir, "cc1-clone", 0);
close(fd);
close(dir);
printf("fd %i dir %i fclonefileat %i\n", fd, dir, a);
return 0;
}
./coreutils/src/cp cc1 cc1-ori-before
gcc clone.c && ./a.out
./coreutils/src/cp cc1 cc1-ori-after
ll cc1*
-rwxr-xr-x 1 g staff 27551296 Feb 10 17:35 cc1*
-rwxr-xr-x 1 g staff 27551296 Feb 10 17:35 cc1-clone*
-rwxr-xr-x 1 g staff 27551296 Feb 10 17:39 cc1-ori-after*
-rwxr-xr-x 1 g staff 27551296 Feb 10 17:39 cc1-ori-before*
sha1sum
812cb0591b000f56bdf75c0f3f94c622e371b970 cc1
812cb0591b000f56bdf75c0f3f94c622e371b970 cc1-clone
812cb0591b000f56bdf75c0f3f94c622e371b970 cc1-ori-after
7b447132f56f3b549ef62a4780945e82d26f7d2c cc1-ori-before
df -h
Filesystem Size Used Avail Capacity iused ifree %iused =
Mounted on
/dev/disk1s1 1.1Ti 1.0Ti 58Gi 95% 2400403 605165920 0% =
/System/Volumes/Data
# clone a large file 15 GB using fclonefileat
./a.out
# copy a large file 15 GB using Finder: option + drag in the same =
directory
# both complete instantly
df -h
Filesystem Size Used Avail Capacity iused ifree %iused =
Mounted on
/dev/disk1s1 1.1Ti 1.0Ti 58Gi 95% 2400401 605207240 0% =
/System/Volumes/Data
ll -h /Users/g/vhd/OpenWRT.wrt3200*
-rw-r--r--@ 1 g staff 15G Feb 10 04:06 =
/Users/g/vhd/OpenWRT.wrt3200-clone.sparseimage
-rw-r--r--@ 1 g staff 15G Feb 10 04:06 =
/Users/g/vhd/OpenWRT.wrt3200-finder.sparseimage
-rw-r--r--@ 1 g staff 15G Feb 10 04:06 =
/Users/g/vhd/OpenWRT.wrt3200.sparseimage
Cheers mate!
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 14:02:32 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 09:02:32 2023 Received: from localhost ([127.0.0.1]:34501 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQTyx-0004dX-Vr for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 09:02:32 -0500 Received: from mail-wm1-f49.google.com ([209.85.128.49]:37530) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pQTyt-0004dF-Gs for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 09:02:30 -0500 Received: by mail-wm1-f49.google.com with SMTP id bg13-20020a05600c3c8d00b003d9712b29d2so6353224wmb.2 for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 06:02:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=V6PkSTOfLsmQ6s/LgZ+hruoHDPepIbP4wcxsT3ZF/Yc=; b=XvzVsuzjU3UCH5EyZkrGDKPhgA3rTwOIu/wnkfv1O1DUru4d5XKGY1Iq3RFzaYCTpg 03RvFRjsJgRf4Zv909F2lbpIwXJfha3km/NnHissOWnl7cgCufQQWjrF2B/4Ug/rshjo zvGyHAPMbyPN13ZrWXhzZrgSbMPKit4uwW17jkyCT73OzCZ97+aUTayMdVpe/xzT/uwT IYpJ1J008RlvPujaqb6VB3/aaWikmHxPP9ylZKCHamQrENPOOxJe+OlC3uKm2Qdp1Tja 4RsDCxQ+HjAofDYuCVScjLC2qJCw9/wwvrEjCnj+DYni3hYlZXRwkQlYzxNvXNxoj0ib BhGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=V6PkSTOfLsmQ6s/LgZ+hruoHDPepIbP4wcxsT3ZF/Yc=; b=TyF5ua+McwPXYGRXaO/VcG3oiHISIgJR/zD7RwXCsEFXam1PzhhXK5yWFodLuhEKbB TC3GDJumIC2ZXMKRzfJJFng2MNMwURQVVoXzGPIJAhAPggqkH7BjVmK1vLZ8riSO8foK lTNGqztjaNqZbtIPYNTN3Rlrw6SsG1OhRUSt2ciCHW+fMoHiyFxl11MthOA+qZY6ZfyQ pmtLvW1oVCzodAWX8t6TjK9L2WMnAewUp3sJ6uPHu5TpYEDAW82V/r119iOJdk9E7Dqf 1ITSSRJBaxfnTPwbPk3M94Xl5CjsKTGcWAasjHKLbMDpL/pztREf8CJJF5oggT/cnZGG j+WA== X-Gm-Message-State: AO0yUKVcheBkd5QY9d/GrS588NtF64hupIiqcqa+mRYULbWQ8cMSrndf uNskQN+ZfLVRNzU36e3I1FmxfP5Rezautw== X-Google-Smtp-Source: AK7set92hjprO7wxSYmFsrhW/Mf2xGwkjy7svMIJYqjnQ9fV62WErIZuGCO53Er8WsfQXCzYzLPbOg== X-Received: by 2002:a05:600c:1685:b0:3db:2e06:4091 with SMTP id k5-20020a05600c168500b003db2e064091mr14701258wmn.37.1676037741598; Fri, 10 Feb 2023 06:02:21 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id az10-20020a05600c600a00b003dc3f07c876sm8570487wmb.46.2023.02.10.06.02.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 10 Feb 2023 06:02:21 -0800 (PST) Message-ID: <7b7f50d7-e50a-6272-0fec-038ff17ec61f@HIDDEN> Date: Fri, 10 Feb 2023 14:02:19 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, Paul Eggert <eggert@HIDDEN> References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 Cc: 61386 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 10/02/2023 12:13, George Valkov wrote: > >> On 2023-02-10, at 11:18 AM, Pádraig Brady <P@HIDDEN> wrote: >> >> I'll apply the simple patch later I think. > > I have an interesting idea: If I copy a large file, say the 16 GB disk image > where I compiled OpenWRT. So I copy this on the same filesystem, and check > disk usage before an after: no change. Also the copy is instant. That’s because > APFS supports copy-on-write. Initially both files share the same sectors on disk, > any changes made after that are copied to new sectors. > This is the most efficient way to copy files on APFS, and should produce no corruption. > Let’s implement it. I would assume there is a system cal that does the entire copy, > And we don’t need to read and write any data. > Does the trace contain any interesting calls that might be related to that? When you say "I copy a large file", is that with gcp or something else? Since coreutils 9.1 we try the CoW by default with fclonefileat() which is available since macOS 10.12 (2016). Note that works only when src and dest are within the same file system, but that is the case for you if I'm reading your original report correctly. When I mentioned that earlier I thought your macOS version was too old to support that, but in fact your 12.6.3 should support fclonefileat() fine. So that's another variable. Is that call failing completely for you? You might be quicker to add a couple of printfs around the fclonefileat() calls in the coreutils from latest git you compliled. Note there is new error handling related to that call in the latest git, compared to what was released in coreutils 9.1. Note also I don't see any fclonefileat() syscalls in your Finder logs at least. cheers, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 12:13:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 07:13:33 2023
Received: from localhost ([127.0.0.1]:34445 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQSHV-0007xk-Dh
for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 07:13:33 -0500
Received: from mail-wm1-f44.google.com ([209.85.128.44]:42659)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pQSHS-0007xU-L9
for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 07:13:32 -0500
Received: by mail-wm1-f44.google.com with SMTP id
j29-20020a05600c1c1d00b003dc52fed235so3925756wms.1
for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 04:13:30 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=sEt1aOmc2gwNO1yT8oYCdlTopo5zPRb79EadAxfqdG0=;
b=h0dOflHX2jQ7pBMbneCvldxsap4toPOts+HZmTHOBTAAe8i6VqvRwMnLlzMMJXm5XM
stF3Rjh0FUBU5DWoUhURbxkPf0Svwx0tmyCRFyTmXgbmOPXqVqYzF1tF7D/9EYhCMKEW
h1PhvX5wzlw1GEnqo48xejjBWF4ClQ1wjsEsxSgcWYm5pXZoP4V29RYl4RaihiDHJ+tm
cvrUku8xO0L1gXUnJvpte7XDJVpe4JmXE6u6X9xK74DQ4Q/dyFrHFJpP397QV9BfKHC7
L9BZovKy/t35wj7NdadcfFSBZwgw5zZmVMRbap6RN7kqpf2j5nank05BIYynCeQnCkj3
sYSw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=sEt1aOmc2gwNO1yT8oYCdlTopo5zPRb79EadAxfqdG0=;
b=pUv0DPtfYeqV+SYoxkprDBDYTGBMuJZFdmFfvlxSIiSyAQKACxGdUURB7dJPkSlqXN
wFhwfBc+GgJXnsS97aZsIuXoVo7Rscm73gvy3YCNiwRdog2P/oE55uyXXKo26Tw9CbHr
efXQiyHx8iuJ5hoeLE6HUuX2DfCwMlORM3O7XyqD1/I8g8W7NMJcezFbDFom/7EyPwTU
msHywkaefG0G4BJJbRezZtw0hrblfmAzwVSJkgQIg3+TY5LV75VJuJn2xzoqGd4jNEro
40lKa/ykB6ZM5poOT6XCZ098rSqeOydSvzvdlbpg3chOH3lr6TE0rwOekItmCf2NAAJP
aiiA==
X-Gm-Message-State: AO0yUKXtXZen09E0SRpoQ/kIRQso07/sRl3TjTGBi6UMr+uv8VwVS9fe
vxljNJwfrpbgBQcBsJ1bfqE=
X-Google-Smtp-Source: AK7set+72/vj7CdGv5ZroLsouA7XBgpHvqhtXuV3cJGiF147LpI+yJXYG1DArImfSKMvWyYmsq5pqg==
X-Received: by 2002:a05:600c:3ca3:b0:3dc:5ae4:c13d with SMTP id
bg35-20020a05600c3ca300b003dc5ae4c13dmr17122960wmb.4.1676031204524;
Fri, 10 Feb 2023 04:13:24 -0800 (PST)
Received: from smtpclient.apple (212-39-89-145.ip.btc-net.bg. [212.39.89.145])
by smtp.gmail.com with ESMTPSA id
v18-20020a5d4a52000000b002c3f210e8c0sm3614764wrs.19.2023.02.10.04.13.23
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Fri, 10 Feb 2023 04:13:24 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
Date: Fri, 10 Feb 2023 14:13:20 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <4DDCA557-BF65-4D58-A07D-7A959EEB477B@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN>
<6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>,
Paul Eggert <eggert@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
> On 2023-02-10, at 11:18 AM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 10/02/2023 03:57, Paul Eggert wrote:
>> On 2/9/23 01:20, George Valkov wrote:
>>> -#ifdef SEEK_HOLE
>>> +#if defined(SEEK_HOLE) && !defined(__APPLE__)
>> Instead of always disabling the SEEK_HOLE optimization, how about =
doing
>> it only on APFS files? Something like the attached, perhaps (this is
>> against Savannah master).
>> If APFS is pretty much universal now on Apple platforms, this patch =
is
>> overkill and we should use your much-simpler patch.
Hi Paul,
I compiled your patch against git://git.savannah.gnu.org/coreutils.git
and it fixes the problem. But as I mention later, I think we should not =
use it.
Thank you!
> Thanks for doing that Paul.
> I think the simpler patch is best though
> since APFS is the default macOS file system since 2017,
> and upon upgrade earlier file systems are upgraded to that.
I agree. macOS supports HFS+ and APFS.
According to Wikipedia, HFS+ does not support sparse files.
On APFS we should not use SEEK_DATA since it is broken.
So the simple patch is better.
> Also SEEK_DATA doesn't seem to be a well supported feature
> on macOS anyway as there was also this issue:
> https://bugs.gnu.org/51857
>=20
> I'll apply the simple patch later I think.
I have an interesting idea: If I copy a large file, say the 16 GB disk =
image
where I compiled OpenWRT. So I copy this on the same filesystem, and =
check
disk usage before an after: no change. Also the copy is instant. =
That=E2=80=99s because
APFS supports copy-on-write. Initially both files share the same sectors =
on disk,
any changes made after that are copied to new sectors.
This is the most efficient way to copy files on APFS, and should produce =
no corruption.
Let=E2=80=99s implement it. I would assume there is a system cal that =
does the entire copy,
And we don=E2=80=99t need to read and write any data.
Does the trace contain any interesting calls that might be related to =
that?
Regarding dtruss, on macOS there is integrity protection which prevents =
you
from debugging applications that came with macOS such as fined. I read
that in order to do this, integrity protection has to be disabled from =
recovery,
then copy the application to a non-system location, e.g. ~/ and finally
remove code signature. That=E2=80=99s why the traces I provided are =
limited, and it gets
tricky to do all of this, considering Finder runs all the time and the =
system
will restart it if it gets killed. But we need to run the copy. With =
that said, if you want,
I will give it a try.
dtruss [-acdefholLs] [-t syscall] { -p PID | -n name | command | -W name =
}
-p PID # examine this PID
-n name # examine this process name
-t syscall # examine this syscall only
-W name # wait for a process matching this name
-a # print all details
-c # print syscall counts
-d # print relative times (us)
-e # print elapsed times (us)
-f # follow children
-l # force printing pid/lwpid
-o # print on cpu times
-s # print stack backtraces
-L # don't print pid/lwpid
-b bufsize # dynamic variable buf size
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 09:18:54 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 04:18:54 2023 Received: from localhost ([127.0.0.1]:34330 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQPYU-0002jp-4s for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 04:18:54 -0500 Received: from mail-wm1-f48.google.com ([209.85.128.48]:33405) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pQPYS-0002ja-Cw for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 04:18:52 -0500 Received: by mail-wm1-f48.google.com with SMTP id l21-20020a05600c1d1500b003dfe462b7e4so5305001wms.0 for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 01:18:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=U4ctm0ufvSa9V48Zw48O5IJQ70G1HTdIO29SXuK6/QM=; b=L+RCBo91aI9cOOd+8WuyGjGGJqw8CB71Ntn8+xXpPLzDoAd2g4/txXGLhwIDAEihoy 8m10z/KMPNpefnmpooJnEyAWGRMrtn3YouCwGDsj6uC2i9ZFcIIKhTv/EaBCjduSz0cT 28xCYIjm4qpDk40U2r6jNbUenVVTKZiwKZYGLbMfDEvw0ZJ4rcZ7rSLZuHopZlLKroVY +IAEsbfiqKGxp6YGmgxTv/BdUdhTB4/WznUhH9turSY9S/x/buYlYEZWvil2I8bNc5w0 C9klu3Q2qKiXqo+LSG8dtrRZEU4mcECTbQUS/ZGOV+inTV5s8RylBTo3OJilzFkfo4lD BHDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=U4ctm0ufvSa9V48Zw48O5IJQ70G1HTdIO29SXuK6/QM=; b=Kuo5xY4FDWP5/ZZQzcwXcVpV2ySoO3sBaZk9zrvKXVrh5ik9dlRV1yJFEXTaH7T9sF 1K93sI0Re0zKPJXj7ck2iUc/Kz0PzD9JFyPyyfWu/yE99ZeDZi+go7+8WfgVtYlzdN9N UaONGuMwLeHKYZ2cJ1AfDzeQk/hMeAZnJMJwsqGc+i9IyVMWYoYHJZaUyS8/w+KoDlS3 fXsNVqBwobRP4UE6IcRbLDEazfOaQniTjWZ300jfroHS0UDAObBUlcHgLBT2PQVRdUfb xRd7DEzlXtQu+HaXN7wCB6ajyAbXKExPdcz/mJuzOVLR40IkQx5BjB1Y4lmdcNIntO3z Bipw== X-Gm-Message-State: AO0yUKXAI78OSJprLbv3RGGEpbcD9LvXydLu6hNBlSFaDeIblvJn6TkH 60WxHaszT01c4SYe196sojIVd1XA5duj+w== X-Google-Smtp-Source: AK7set83s6Z4TXXoGLLr6gHMqOKayVRRjOLb4MVFq3SBGwLpI6f3Q8S1ImKJcLEou3JlQl94Qt0DHQ== X-Received: by 2002:a05:600c:32a7:b0:3dd:e86e:8805 with SMTP id t39-20020a05600c32a700b003dde86e8805mr12463051wmp.5.1676020726716; Fri, 10 Feb 2023 01:18:46 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id n2-20020a05600c3b8200b003dfe659f9b1sm13746375wms.3.2023.02.10.01.18.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 10 Feb 2023 01:18:46 -0800 (PST) Message-ID: <6c94b60d-08f1-ba46-6173-03caba917643@HIDDEN> Date: Fri, 10 Feb 2023 09:18:45 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: Paul Eggert <eggert@HIDDEN>, George Valkov <gvalkov@HIDDEN>, 61386 <at> debbugs.gnu.org References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 10/02/2023 03:57, Paul Eggert wrote: > On 2/9/23 01:20, George Valkov wrote: >> -#ifdef SEEK_HOLE >> +#if defined(SEEK_HOLE) && !defined(__APPLE__) > > Instead of always disabling the SEEK_HOLE optimization, how about doing > it only on APFS files? Something like the attached, perhaps (this is > against Savannah master). > > If APFS is pretty much universal now on Apple platforms, this patch is > overkill and we should use your much-simpler patch. Thanks for doing that Paul. I think the simpler patch is best though since APFS is the default macOS file system since 2017, and upon upgrade earlier file systems are upgraded to that. Also SEEK_DATA doesn't seem to be a well supported feature on macOS anyway as there was also this issue: https://bugs.gnu.org/51857 I'll apply the simple patch later I think. cheers, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 09:14:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 10 04:14:09 2023
Received: from localhost ([127.0.0.1]:34326 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQPTt-0002cr-9X
for submit <at> debbugs.gnu.org; Fri, 10 Feb 2023 04:14:09 -0500
Received: from mail-wm1-f47.google.com ([209.85.128.47]:33647)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <pixelbeat@HIDDEN>) id 1pQPTr-0002ce-WD
for 61386 <at> debbugs.gnu.org; Fri, 10 Feb 2023 04:14:08 -0500
Received: by mail-wm1-f47.google.com with SMTP id
l21-20020a05600c1d1500b003dfe462b7e4so5299359wms.0
for <61386 <at> debbugs.gnu.org>; Fri, 10 Feb 2023 01:14:07 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=content-transfer-encoding:in-reply-to:from:references:cc:to
:content-language:subject:user-agent:mime-version:date:message-id
:sender:from:to:cc:subject:date:message-id:reply-to;
bh=VrNqDlrUeHF2I4CrS64cY6JtDnhEhogYFAdrLkfIB2k=;
b=q04UNJWcJZ7nSe0BkiDmQ9BlvOfG161j/P1WD3h6SRDwDaxkXwsWNRGvEWO6G7G7Ng
9iDzENMiOQKq/b2BUTL4y4gdOKUrSruruIkopFKlhBs+1VLKTJVnvRDQWcAZGwQtO3hQ
gcduXz/Vo3GLGcKyahjHz6ZmzHucqTrDGtviGRKpva4bMal889Y3aygywSB6998ccosJ
AtrQ6pUqKdOJnCCKahP8e+fR4IhK6ZQXHdnx/pv0ggIU6ElhNAp/HhNVSgL6/70+MSL/
BelTv4MJQ+tv9kiSmwuDhA5SoQD5K7ca+n3Vjjqpg5VI8NMQG+ZnwcOPlHXbWsNIzNPM
YViA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=content-transfer-encoding:in-reply-to:from:references:cc:to
:content-language:subject:user-agent:mime-version:date:message-id
:sender:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=VrNqDlrUeHF2I4CrS64cY6JtDnhEhogYFAdrLkfIB2k=;
b=xeC5JkHorFbeK0BByTBiX5LaaBNq2I/a/goYpOxseMtWWDELMwoQBYx8y4BHW+FyE1
rHXDe9cVpIxpuqROCvE/H0d6BaIdF4fpAcaUJC2DtceDP6RycD+R+Dy934GSmh8gRkbi
Iiu5zoZBeACW8CDKpN+32n6yH9ZynbQW/bANiHHCcMcNojc3GAOsRKuYxerKc5PzLv8J
0sZUDYqJ5S5iFcEWhZdtdAP17D0M8RkrinhD3guG1rKhCNJu2c0H/sQWGLLbj9fNU73w
N1goPOv8rZucskc7E96JqjgBhG8gLBmuQal4m1t3cU7UWx4we/RHRo/yZ+TTt+febYeU
FLoA==
X-Gm-Message-State: AO0yUKXqMwxFRdVc4lnB6HtAtbVTW9NQlA5wnjFeKT13SLaAFnkw4wbo
68g/Is0Zyf8kJxBn8c8eWeo=
X-Google-Smtp-Source: AK7set91FSRt+gTtbjAvLNOv7BZtLvBcBGX/uX1mo0FOrZXehJlTIqu1dBrrSHdzzu/22RHn09wWtQ==
X-Received: by 2002:a05:600c:43c4:b0:3dc:557f:6129 with SMTP id
f4-20020a05600c43c400b003dc557f6129mr12180316wmn.2.1676020441787;
Fri, 10 Feb 2023 01:14:01 -0800 (PST)
Received: from [192.168.1.9]
(95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175])
by smtp.googlemail.com with ESMTPSA id
n2-20020a05600c3b8200b003dfe659f9b1sm13746375wms.3.2023.02.10.01.14.01
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Fri, 10 Feb 2023 01:14:01 -0800 (PST)
Message-ID: <5b2cefc2-3b9a-b84b-1dd6-1b3ac396a48a@HIDDEN>
Date: Fri, 10 Feb 2023 09:14:00 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101
Thunderbird/109.0
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Content-Language: en-US
To: George Valkov <gvalkov@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<675808b7-58af-6419-7e91-372e556c2ad2@HIDDEN>
<21A1241D-B890-4796-926F-E8712E2A807B@HIDDEN>
<cddf680c-5544-4da8-f8fa-dbaee0c4aaaf@HIDDEN>
<BAC6C40E-ED03-4121-97C4-9B0E53191F61@HIDDEN>
<9dffe2e2-287e-b7ce-e6f4-b9a56aa0bdbc@HIDDEN>
<5FAEB850-8A6C-4C45-8D34-3E011CD49112@HIDDEN>
From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
In-Reply-To: <5FAEB850-8A6C-4C45-8D34-3E011CD49112@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.9 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.9 (-)
On 10/02/2023 01:03, George Valkov wrote:
>
>
>> On 2023-02-09, at 11:35 PM, Pádraig Brady <P@HIDDEN> wrote:
>>
>> On 09/02/2023 17:23, George Valkov wrote:
>>>> On 2023-02-09, at 6:32 PM, Pádraig Brady <P@HIDDEN> wrote:
>>>>
>>>> On 09/02/2023 15:57, George Valkov wrote:
>>>>>> On 2023-02-09, at 1:56 PM, Pádraig Brady <P@HIDDEN> wrote:
>>>>>>
>>>>>> On 09/02/2023 09:20, George Valkov wrote:
>>>>>>> Due to a bug in macOS, sparse copies are corrupted on virtual disks
>>>>>>> formatted with APFS. HFS is not affected. Affected are coreutils
>>>>>>> install, and gcp when compiled with SEEK_HOLE, as well as macOS Finder.
>>>>>>> While reading the entire file returns valid data, scanning for
>>>>>>> allocated segments may return holes where valid data is present.
>>>>>>> In this case a sparse copy does not contain these segments and returns
>>>>>>> zeroes instead. Once the virtual disk is dismounted and then
>>>>>>> mounted again, a sparse copy produces correct results.
>>>>>>> This breaks OpenWRT build on macOS. Details:
>>>>>>> https://github.com/openwrt/openwrt/pull/11960
>>>>>>> https://github.com/openwrt/openwrt/pull/11960#issuecomment-1423185579
>>>>>>> Signed-off-by: Georgi Valkov <gvalkov@HIDDEN>
>>>>>>> ---
>>>>>>> src/copy.c | 2 +-
>>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>> diff --git a/src/copy.c b/src/copy.c
>>>>>>> index e16fedb28..4f56138a6 100644
>>>>>>> --- a/src/copy.c
>>>>>>> +++ b/src/copy.c
>>>>>>> @@ -1065,7 +1065,7 @@ infer_scantype (int fd, struct stat const *sb,
>>>>>>> return PLAIN_SCANTYPE;
>>>>>>> }
>>>>>>> -#ifdef SEEK_HOLE
>>>>>>> +#if defined(SEEK_HOLE) && !defined(__APPLE__)
>>>>>>> scan_inference->ext_start = lseek (fd, 0, SEEK_DATA);
>>>>>>> if (0 <= scan_inference->ext_start || errno == ENXIO)
>>>>>>> return LSEEK_SCANTYPE;
>>>>>>
>>>>>>
>>>>>> Thanks for the detailed report.
>>>>>> The patch might very well be appropriate.
>>>>> Hi! Let’s test the ideas you have first, and fall-back to the patch.
>>>>> In October 2021 macOS Finder was also affected, and that points directly at Apple.
>>>>> I tested again today, they have fixed Finder. After performing a copy in Finder,
>>>>> coreutils cp produces a good copy. I have to run
>>>>> make toolchain/gcc/initial/{clean,compile} -j 16
>>>>> before I can reproduce it again with the same file.
>>>>> So while Apple didn't fix the underlaying issue with APFS, they did
>>>>> provide a solution for Finder. And we can make coreutils work correctly too.
>>>>
>>>> That suggests that Finder may have sync'd the file.
>>>> Now syncing has overheads of course, so not an option to take lightly.
>>> #include "stdio.h"
>>> #include "unistd.h"
>>> #include "fcntl.h"
>>> int main(int argc, char ** argv)
>>> {
>>> sync();
>>> int fd = open("cc1", O_RDWR);
>>> //int fd = open("/Users/g/vhd/coreutils.sparseimage", O_RDWR);
>>> int a = fdatasync(fd);
>>> int b = fsync(fd);
>>> int c = close(fd);
>>> printf("fdatasync %u fsync %u close %u\n", a, b, c);
>>> return 0;
>>> }
>>> fdatasync 0 fsync 0 close 0
>>> I agree. It takes about one second to run this code.
>>> All calls are successful. The sparse copy after that is still corrupted.
>>> I also tried doing this on the sparse image while it is mounted.
>>
>> Thanks for confirming the sync doesn't help.
>>
>>>> We may be safer just doing the normal copy on __APPLE__ as per your orig patch.
>>>> A dtruss of Finder might be instructive BTW.
>>> Yes, sync is definitely not good for performance. What is ‘dtruss'?
>>
>> That would show the system calls that Finder is using to perform the copy.
>> Perhaps Finder is also just avoiding the SEEK_DATA on APFS?
>
> I ran 3 traces, let me know if they are useful? If not, I will have to disable
> integrity protection, and collect new data tomorrow. I got errors:
> dtruss: system integrity protection is on, some features will not be available
> https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/dtruss/
>
> You did not mention which flags you want with dtruss?
>
>
>> Without further info, I'll apply your original avoidance patch for __APPLE__.
>
> It’s up to you, as long as you have ideas, I can test them.
> I can also invite you on Parsec or Anydesk. Working together is more efficient.
I've only used dtruss a couple of times (with sudo)
as I'm not a macOS user, sorry.
Those 3 traces only had a single lseek() each,
so I'm assuming for now that Finder is not using SEEK_DATA.
Will apply your patch later.
thanks,
Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 03:57:55 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 09 22:57:55 2023 Received: from localhost ([127.0.0.1]:34049 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQKXr-000285-0V for submit <at> debbugs.gnu.org; Thu, 09 Feb 2023 22:57:55 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:46454) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pQKXo-00027r-BA for 61386 <at> debbugs.gnu.org; Thu, 09 Feb 2023 22:57:53 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 675FF160076; Thu, 9 Feb 2023 19:57:46 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 2i8a2Av0UQr3; Thu, 9 Feb 2023 19:57:45 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 0F167160080; Thu, 9 Feb 2023 19:57:45 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 0F167160080 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1676001465; bh=kcd9hG3qivPFGXUmjuCmCVkdD0bRYCAFvduTkr34Hww=; h=Content-Type:Message-ID:Date:MIME-Version:Subject:To:From; b=uPRfsDDBXbtVZtUbM3lzdcXpegM+l5KRl6r9e80diAscUeJ3ipPz3dGMl+Vb9sLJx BPo/ZJxE3vAYd1H14+zn4Dr9Zhs0QwtKl+P1mQLyw4uklaNQmynaQ9+jhfUqp2uJQq 1BfwnlYbM/Bs4TYvMEWGP4JoJoEjI5geqDc8wwuQ= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ZLxjdXGGG5tv; Thu, 9 Feb 2023 19:57:44 -0800 (PST) Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id D08A8160076; Thu, 9 Feb 2023 19:57:44 -0800 (PST) Content-Type: multipart/mixed; boundary="------------0hRobjBNEAcnpZrMUNXpL8GD" Message-ID: <4513f574-38ef-a7d4-fa16-2a4d27d0fd4f@HIDDEN> Date: Thu, 9 Feb 2023 19:57:44 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, 61386 <at> debbugs.gnu.org References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 61386 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) This is a multi-part message in MIME format. --------------0hRobjBNEAcnpZrMUNXpL8GD Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2/9/23 01:20, George Valkov wrote: > -#ifdef SEEK_HOLE > +#if defined(SEEK_HOLE) && !defined(__APPLE__) Instead of always disabling the SEEK_HOLE optimization, how about doing it only on APFS files? Something like the attached, perhaps (this is against Savannah master). If APFS is pretty much universal now on Apple platforms, this patch is overkill and we should use your much-simpler patch. --------------0hRobjBNEAcnpZrMUNXpL8GD Content-Type: text/x-patch; charset=UTF-8; name="0001-cp-work-around-APFS-bug.patch" Content-Disposition: attachment; filename="0001-cp-work-around-APFS-bug.patch" Content-Transfer-Encoding: base64 RnJvbSBiZDhjOTMxZmFkZTdmY2Y5NmMwMmU0NDkyOTEyZjI3YTJkMWM3OTViIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBUaHUsIDkgRmViIDIwMjMgMTk6NTM6NTkgLTA4MDAKU3ViamVjdDogW1BBVENI XSBjcDogd29yayBhcm91bmQgQVBGUyBidWcKClNlZSA8aHR0cHM6Ly9idWdzLmdudS5vcmcv NjEzODY+LgoqIHNyYy9jb3B5LmMgKFNFRUtfSE9MRSAmJiBfX0FQUExFX18pOgpJbmNsdWRl IDxzeXMvbW91bnQuaD4sIGZvciBzdGF0ZnMuCihpbmZlcl9zY2FudHlwZSkgW1NFRUtfSE9M RSAmJiBfX0FQUExFX18pOgpEb24ndCB1c2UgU0VFS19IT0xFIG9uIEFQRlMsIGFzIEFQRlMg aXMgYnVnZ3kuCi0tLQogc3JjL2NvcHkuYyB8IDI0ICsrKysrKysrKysrKysrKysrKysrKysr KwogMSBmaWxlIGNoYW5nZWQsIDI0IGluc2VydGlvbnMoKykKCmRpZmYgLS1naXQgYS9zcmMv Y29weS5jIGIvc3JjL2NvcHkuYwppbmRleCBkZmJiNTU3ZGUuLjk1MTUxZmFjMCAxMDA2NDQK LS0tIGEvc3JjL2NvcHkuYworKysgYi9zcmMvY29weS5jCkBAIC02NSw2ICs2NSwxMCBAQAog I2luY2x1ZGUgInllc25vLmgiCiAjaW5jbHVkZSAic2VsaW51eC5oIgogCisjaWYgZGVmaW5l ZCBTRUVLX0hPTEUgJiYgZGVmaW5lZCBfX0FQUExFX18KKyMgaW5jbHVkZSA8c3lzL21vdW50 Lmg+CisjZW5kaWYKKwogI2lmbmRlZiBVU0VfWEFUVFIKICMgZGVmaW5lIFVTRV9YQVRUUiBm YWxzZQogI2VuZGlmCkBAIC0xMDYzLDYgKzEwNjcsMjYgQEAgaW5mZXJfc2NhbnR5cGUgKGlu dCBmZCwgc3RydWN0IHN0YXQgY29uc3QgKnNiLAogICAgIHJldHVybiBQTEFJTl9TQ0FOVFlQ RTsKIAogI2lmZGVmIFNFRUtfSE9MRQorIyBpZmRlZiBfX0FQUExFX18KKyAgLyogV29yayBh cm91bmQgQVBGUyBidWcgPGh0dHBzOi8vZGViYnVncy5nbnUub3JnLzYxMzg2PiBieSBub3Qg dXNpbmcKKyAgICAgU0VFS19EQVRBIG9uIEFQRlMgZmlsZXMuICBUbyBhdm9pZCBmc3RhdGZz IGNhbGxzIGluIGNvbW1vbiBjYXNlcywKKyAgICAgQVBGU19ERVZbMF0gY2FjaGVzIHRoZSBt b3N0IHJlY2VudCBub24tQVBGUyBzdF9kZXYsIGFuZCBBUEZTX0RFVlsxXQorICAgICBjYWNo ZXMgdGhlIG1vc3QgcmVjZW50IEFQRlMgc3RfZGV2LiAgWmVybyBjYWNoZXMgYXJlIGludmFs aWQuICAqLworICBzdGF0aWMgZGV2X3QgYXBmc19kZXZbMl07CisgIGlmICghIChzYi0+c3Rf ZGV2ICYmIHNiLT5zdF9kZXYgPT0gYXBmc19kZXZbMF0pKQorICAgIHsKKyAgICAgIGlmIChz Yi0+c3RfZGV2ICYmIHNiLT5zdF9kZXYgPT0gYXBmc19kZXZbMV0pCisgICAgICAgIHJldHVy biBaRVJPX1NDQU5UWVBFOworICAgICAgc3RydWN0IHN0YXRmcyBmczsKKyAgICAgIGlmIChm c3RhdGZzIChmZCwgJmZzKSAhPSAwKQorICAgICAgICByZXR1cm4gRVJST1JfU0NBTlRZUEU7 CisgICAgICBib29sIGFwZnMgPSBzdHJjbXAgKGZzLmZfZnN0eXBlbmFtZSwgImFwZnMiKSA9 PSAwOworICAgICAgaWYgKHNiLT5zdF9kZXYpCisgICAgICAgIGFwZnNfZGV2W2FwZnNdID0g c2ItPnN0X2RldjsKKyAgICAgIGlmIChhcGZzKQorICAgICAgICByZXR1cm4gWkVST19TQ0FO VFlQRTsKKyAgICB9CisjIGVuZGlmCiAgIG9mZl90IGV4dF9zdGFydCA9IGxzZWVrIChmZCwg MCwgU0VFS19EQVRBKTsKICAgaWYgKDAgPD0gZXh0X3N0YXJ0IHx8IGVycm5vID09IEVOWElP KQogICAgIHsKLS0gCjIuMzkuMQoK --------------0hRobjBNEAcnpZrMUNXpL8GD--
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 10 Feb 2023 01:17:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 09 20:17:25 2023
Received: from localhost ([127.0.0.1]:33915 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQI2W-00063H-G2
for submit <at> debbugs.gnu.org; Thu, 09 Feb 2023 20:17:25 -0500
Received: from mail-wr1-f53.google.com ([209.85.221.53]:41478)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pQHp4-0005gk-DY
for 61386 <at> debbugs.gnu.org; Thu, 09 Feb 2023 20:03:31 -0500
Received: by mail-wr1-f53.google.com with SMTP id bu23so3602999wrb.8
for <61386 <at> debbugs.gnu.org>; Thu, 09 Feb 2023 17:03:30 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20210112; t=1675991004;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=oqhST07axLDIXh38/8I6RC9Bu6n8r813s8VwwTvoJn0=;
b=CvtlidxGrRLvrcG2joH9fzJM9LXnFlR3EBnkUQI76ARXd+5JKGc+GWJXQLGRNTjDmp
4vijG0Mu6AHbAcxIH2qo49BTVdTXpu/5lZtUx0CjqKBBkE9pDy2ML4tmKeavlsA07DBv
ZWXxIHtt78EFewBNLJ5XhqZq8pAVKD4nC83Kl7lGP+6A0M/hdYEumSnQ1zffu/LBR7Pj
N8ldNvnfzPJWl1wzlHleyqZmz9iCbk3+3VPrPzh81IW1OgQ20A1WM01T8MEkEkjXehDF
U0uP82Z88ff+wEo1cc/RrTdzELtlkWtXDTYZf5YXrWbu9Wh9VqENasT98koPVCRG1uzm
HwkQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112; t=1675991004;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=oqhST07axLDIXh38/8I6RC9Bu6n8r813s8VwwTvoJn0=;
b=T4BzaUysZgJxKoec79jJjysDafztfqOpSXC3HEPyVcw3I4/IxyI8vIt9jY7xq0YF3L
lZcJAPZrEeMOS2xGIWmhd+CUK4w9XjTmcxlGoLmh7XVi10veznJ7MhJvFOKfFm6zyidK
+qMnuC7hTWT5iYBTnSkAj9lwz/sdXj7trMBpsP+nAmV682nQtIURMePo6KixSqlI6DUv
XKkTUg6rDETUfdMxFGUVKfT9p7CbzP0Hw5jhTHKtkuSulgk8l7pdtYpRaVszzrZwBBmO
2cT1LbRyH6q0y4N/GC6OECwoEhnu7PenFi/+jJYRjNEPMuHqnx/rwOEQOemweFfaUeeN
g7LA==
X-Gm-Message-State: AO0yUKVtVmgtE3qewukfJkjzQLP3NQmknofmQi7VQU1TFXZO9+wYw0oq
ais8f2KtwLGJrYbt2fSbUU0=
X-Google-Smtp-Source: AK7set+uajOSZ32Wj6xbBK2xq92dOvxinlMBBzY98lMQDf3JQ+pfBOC3yo0uEOSb8tedC7eM2KTAIQ==
X-Received: by 2002:a5d:5443:0:b0:2c4:80a:e851 with SMTP id
w3-20020a5d5443000000b002c4080ae851mr3480734wrv.5.1675991004077;
Thu, 09 Feb 2023 17:03:24 -0800 (PST)
Received: from smtpclient.apple (212-39-89-145.ip.btc-net.bg. [212.39.89.145])
by smtp.gmail.com with ESMTPSA id
i14-20020adff30e000000b00241fab5a296sm2348856wro.40.2023.02.09.17.03.22
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Thu, 09 Feb 2023 17:03:23 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <9dffe2e2-287e-b7ce-e6f4-b9a56aa0bdbc@HIDDEN>
Date: Fri, 10 Feb 2023 03:03:21 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <5FAEB850-8A6C-4C45-8D34-3E011CD49112@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<675808b7-58af-6419-7e91-372e556c2ad2@HIDDEN>
<21A1241D-B890-4796-926F-E8712E2A807B@HIDDEN>
<cddf680c-5544-4da8-f8fa-dbaee0c4aaaf@HIDDEN>
<BAC6C40E-ED03-4121-97C4-9B0E53191F61@HIDDEN>
<9dffe2e2-287e-b7ce-e6f4-b9a56aa0bdbc@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
X-Mailman-Approved-At: Thu, 09 Feb 2023 20:17:23 -0500
Cc: 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
> On 2023-02-09, at 11:35 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 09/02/2023 17:23, George Valkov wrote:
>>> On 2023-02-09, at 6:32 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>>>=20
>>> On 09/02/2023 15:57, George Valkov wrote:
>>>>> On 2023-02-09, at 1:56 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>>>>>=20
>>>>> On 09/02/2023 09:20, George Valkov wrote:
>>>>>> Due to a bug in macOS, sparse copies are corrupted on virtual =
disks
>>>>>> formatted with APFS. HFS is not affected. Affected are coreutils
>>>>>> install, and gcp when compiled with SEEK_HOLE, as well as macOS =
Finder.
>>>>>> While reading the entire file returns valid data, scanning for
>>>>>> allocated segments may return holes where valid data is present.
>>>>>> In this case a sparse copy does not contain these segments and =
returns
>>>>>> zeroes instead. Once the virtual disk is dismounted and then
>>>>>> mounted again, a sparse copy produces correct results.
>>>>>> This breaks OpenWRT build on macOS. Details:
>>>>>> https://github.com/openwrt/openwrt/pull/11960
>>>>>> =
https://github.com/openwrt/openwrt/pull/11960#issuecomment-1423185579
>>>>>> Signed-off-by: Georgi Valkov <gvalkov@HIDDEN>
>>>>>> ---
>>>>>> src/copy.c | 2 +-
>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>> diff --git a/src/copy.c b/src/copy.c
>>>>>> index e16fedb28..4f56138a6 100644
>>>>>> --- a/src/copy.c
>>>>>> +++ b/src/copy.c
>>>>>> @@ -1065,7 +1065,7 @@ infer_scantype (int fd, struct stat const =
*sb,
>>>>>> return PLAIN_SCANTYPE;
>>>>>> }
>>>>>> -#ifdef SEEK_HOLE
>>>>>> +#if defined(SEEK_HOLE) && !defined(__APPLE__)
>>>>>> scan_inference->ext_start =3D lseek (fd, 0, SEEK_DATA);
>>>>>> if (0 <=3D scan_inference->ext_start || errno =3D=3D ENXIO)
>>>>>> return LSEEK_SCANTYPE;
>>>>>=20
>>>>>=20
>>>>> Thanks for the detailed report.
>>>>> The patch might very well be appropriate.
>>>> Hi! Let=E2=80=99s test the ideas you have first, and fall-back to =
the patch.
>>>> In October 2021 macOS Finder was also affected, and that points =
directly at Apple.
>>>> I tested again today, they have fixed Finder. After performing a =
copy in Finder,
>>>> coreutils cp produces a good copy. I have to run
>>>> make toolchain/gcc/initial/{clean,compile} -j 16
>>>> before I can reproduce it again with the same file.
>>>> So while Apple didn't fix the underlaying issue with APFS, they did
>>>> provide a solution for Finder. And we can make coreutils work =
correctly too.
>>>=20
>>> That suggests that Finder may have sync'd the file.
>>> Now syncing has overheads of course, so not an option to take =
lightly.
>> #include "stdio.h"
>> #include "unistd.h"
>> #include "fcntl.h"
>> int main(int argc, char ** argv)
>> {
>> sync();
>> int fd =3D open("cc1", O_RDWR);
>> //int fd =3D open("/Users/g/vhd/coreutils.sparseimage", O_RDWR);
>> int a =3D fdatasync(fd);
>> int b =3D fsync(fd);
>> int c =3D close(fd);
>> printf("fdatasync %u fsync %u close %u\n", a, b, c);
>> return 0;
>> }
>> fdatasync 0 fsync 0 close 0
>> I agree. It takes about one second to run this code.
>> All calls are successful. The sparse copy after that is still =
corrupted.
>> I also tried doing this on the sparse image while it is mounted.
>=20
> Thanks for confirming the sync doesn't help.
>=20
>>> We may be safer just doing the normal copy on __APPLE__ as per your =
orig patch.
>>> A dtruss of Finder might be instructive BTW.
>> Yes, sync is definitely not good for performance. What is =E2=80=98dtru=
ss'?
>=20
> That would show the system calls that Finder is using to perform the =
copy.
> Perhaps Finder is also just avoiding the SEEK_DATA on APFS?
I ran 3 traces, let me know if they are useful? If not, I will have to =
disable
integrity protection, and collect new data tomorrow. I got errors:
dtruss: system integrity protection is on, some features will not be =
available
=
https://httpstorm.com/share/.openwrt/test/2023-02-06_coreutils-9.1/dtruss/=
You did not mention which flags you want with dtruss?
> Without further info, I'll apply your original avoidance patch for =
__APPLE__.
It=E2=80=99s up to you, as long as you have ideas, I can test them.
I can also invite you on Parsec or Anydesk. Working together is more =
efficient.
Cheers, mate!
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 9 Feb 2023 21:35:36 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 09 16:35:36 2023
Received: from localhost ([127.0.0.1]:33858 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQEZs-00008e-En
for submit <at> debbugs.gnu.org; Thu, 09 Feb 2023 16:35:36 -0500
Received: from mail-wm1-f45.google.com ([209.85.128.45]:46847)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <pixelbeat@HIDDEN>) id 1pQEZq-00008O-Qo
for 61386 <at> debbugs.gnu.org; Thu, 09 Feb 2023 16:35:35 -0500
Received: by mail-wm1-f45.google.com with SMTP id
bg5-20020a05600c3c8500b003e00c739ce4so2643721wmb.5
for <61386 <at> debbugs.gnu.org>; Thu, 09 Feb 2023 13:35:34 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=content-transfer-encoding:in-reply-to:from:references:cc:to
:content-language:subject:user-agent:mime-version:date:message-id
:sender:from:to:cc:subject:date:message-id:reply-to;
bh=9a6S7aFmir1+5zt/6dcLiaAum3uEjz0Wlhi9R6ymJO8=;
b=f9TSi0ZEr6xlYShEGEUYbVeH2op78BhZie3OBy+o2TETd9AXdF4xL2KA3ByrsQi8PB
LXedld2oeChzAU5/ckjIeqVOP7jv56O7/avm5rCigd7C42LgNZ2DdUULKsvlj+nUBZwn
l8J+kxAvvZ+cEh4kCE+S19xRF5azpNEFbhszcqowuIkeyQtbMjA/iPCbrmhZab+9Gsme
TnkLaFeb7Elp5nxUcubH3Mfz6VxWp+2DKGM51fa5WUN+M4GX7lTID69aQ1eTla6DG/lm
2mnXxXgLJZqNfSYyueNSPuhIM1JfcfNebqgFyEvGjmSSsKHabWPTXx3ais5U9i3P0ioK
pbgA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=content-transfer-encoding:in-reply-to:from:references:cc:to
:content-language:subject:user-agent:mime-version:date:message-id
:sender:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=9a6S7aFmir1+5zt/6dcLiaAum3uEjz0Wlhi9R6ymJO8=;
b=7H3ST7np1AkGCAmyzkTsNY1RmwIEL273qlHo1MBH19d0S/m178GIsZqRyKqkdSyb37
J2lB1RVYxwDKgTwo8h7uLZ1hxQj/EUswOsCTqmr6b9TD9bpX1lne6Fk2YnaLPuf30S2t
Ozkm+W6b0NgDVLES/skt6cD29ONvk3fJADAKgcr8Ug2lDHWWqdn4Ulkl1NsSpnOF/i1S
p3YAwYW1GsQPGcfUljQEwb8Qa5LZq5OThBj6RowyT6iA6vycjGLDh99RoBGn06ZBGbh8
AzCEdlF3oKjqNKfKhXqZyAD1QDe2bb84rndAYeVlCRl/jgW/Gq26iW2ljQMBTOJgwU5P
1tTA==
X-Gm-Message-State: AO0yUKXvEwMRGLHEAh//Zf0JV9ADMsUxP6DkYqDE1SPr596mbd8/xSlY
lLxRcE73XexJftYqj9EomQQ=
X-Google-Smtp-Source: AK7set/Sii4wf/m8PX5f833e54rb1Cc3p9s3nAzDHUm7ZTvUSJEQmFQ/ITyuyzDggBCgV/wfMVi4Iw==
X-Received: by 2002:a05:600c:13c8:b0:3da:516:19ed with SMTP id
e8-20020a05600c13c800b003da051619edmr10742890wmg.29.1675978528690;
Thu, 09 Feb 2023 13:35:28 -0800 (PST)
Received: from [192.168.1.9]
(95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175])
by smtp.googlemail.com with ESMTPSA id
j6-20020a05600c42c600b003df245cd853sm2685018wme.44.2023.02.09.13.35.27
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Thu, 09 Feb 2023 13:35:28 -0800 (PST)
Message-ID: <9dffe2e2-287e-b7ce-e6f4-b9a56aa0bdbc@HIDDEN>
Date: Thu, 9 Feb 2023 21:35:27 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101
Thunderbird/109.0
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Content-Language: en-US
To: George Valkov <gvalkov@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<675808b7-58af-6419-7e91-372e556c2ad2@HIDDEN>
<21A1241D-B890-4796-926F-E8712E2A807B@HIDDEN>
<cddf680c-5544-4da8-f8fa-dbaee0c4aaaf@HIDDEN>
<BAC6C40E-ED03-4121-97C4-9B0E53191F61@HIDDEN>
From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
In-Reply-To: <BAC6C40E-ED03-4121-97C4-9B0E53191F61@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.9 (/)
X-Debbugs-Envelope-To: 61386
Cc: 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.9 (-)
On 09/02/2023 17:23, George Valkov wrote:
>
>
>> On 2023-02-09, at 6:32 PM, Pádraig Brady <P@HIDDEN> wrote:
>>
>> On 09/02/2023 15:57, George Valkov wrote:
>>>> On 2023-02-09, at 1:56 PM, Pádraig Brady <P@HIDDEN> wrote:
>>>>
>>>> On 09/02/2023 09:20, George Valkov wrote:
>>>>> Due to a bug in macOS, sparse copies are corrupted on virtual disks
>>>>> formatted with APFS. HFS is not affected. Affected are coreutils
>>>>> install, and gcp when compiled with SEEK_HOLE, as well as macOS Finder.
>>>>> While reading the entire file returns valid data, scanning for
>>>>> allocated segments may return holes where valid data is present.
>>>>> In this case a sparse copy does not contain these segments and returns
>>>>> zeroes instead. Once the virtual disk is dismounted and then
>>>>> mounted again, a sparse copy produces correct results.
>>>>> This breaks OpenWRT build on macOS. Details:
>>>>> https://github.com/openwrt/openwrt/pull/11960
>>>>> https://github.com/openwrt/openwrt/pull/11960#issuecomment-1423185579
>>>>> Signed-off-by: Georgi Valkov <gvalkov@HIDDEN>
>>>>> ---
>>>>> src/copy.c | 2 +-
>>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>> diff --git a/src/copy.c b/src/copy.c
>>>>> index e16fedb28..4f56138a6 100644
>>>>> --- a/src/copy.c
>>>>> +++ b/src/copy.c
>>>>> @@ -1065,7 +1065,7 @@ infer_scantype (int fd, struct stat const *sb,
>>>>> return PLAIN_SCANTYPE;
>>>>> }
>>>>> -#ifdef SEEK_HOLE
>>>>> +#if defined(SEEK_HOLE) && !defined(__APPLE__)
>>>>> scan_inference->ext_start = lseek (fd, 0, SEEK_DATA);
>>>>> if (0 <= scan_inference->ext_start || errno == ENXIO)
>>>>> return LSEEK_SCANTYPE;
>>>>
>>>>
>>>> Thanks for the detailed report.
>>>> The patch might very well be appropriate.
>>> Hi! Let’s test the ideas you have first, and fall-back to the patch.
>>> In October 2021 macOS Finder was also affected, and that points directly at Apple.
>>> I tested again today, they have fixed Finder. After performing a copy in Finder,
>>> coreutils cp produces a good copy. I have to run
>>> make toolchain/gcc/initial/{clean,compile} -j 16
>>> before I can reproduce it again with the same file.
>>> So while Apple didn't fix the underlaying issue with APFS, they did
>>> provide a solution for Finder. And we can make coreutils work correctly too.
>>
>> That suggests that Finder may have sync'd the file.
>> Now syncing has overheads of course, so not an option to take lightly.
>
> #include "stdio.h"
> #include "unistd.h"
> #include "fcntl.h"
>
> int main(int argc, char ** argv)
> {
> sync();
>
> int fd = open("cc1", O_RDWR);
> //int fd = open("/Users/g/vhd/coreutils.sparseimage", O_RDWR);
> int a = fdatasync(fd);
> int b = fsync(fd);
> int c = close(fd);
>
> printf("fdatasync %u fsync %u close %u\n", a, b, c);
>
> return 0;
> }
>
> fdatasync 0 fsync 0 close 0
>
> I agree. It takes about one second to run this code.
> All calls are successful. The sparse copy after that is still corrupted.
> I also tried doing this on the sparse image while it is mounted.
Thanks for confirming the sync doesn't help.
>> We may be safer just doing the normal copy on __APPLE__ as per your orig patch.
>> A dtruss of Finder might be instructive BTW.
>
> Yes, sync is definitely not good for performance. What is ‘dtruss'?
That would show the system calls that Finder is using to perform the copy.
Perhaps Finder is also just avoiding the SEEK_DATA on APFS?
Without further info, I'll apply your original avoidance patch for __APPLE__.
cheers,
Pádraig.
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 9 Feb 2023 17:38:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 09 12:38:39 2023
Received: from localhost ([127.0.0.1]:33615 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQAsY-0008OJ-Ni
for submit <at> debbugs.gnu.org; Thu, 09 Feb 2023 12:38:39 -0500
Received: from mail-wr1-f42.google.com ([209.85.221.42]:33766)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pQAeT-0007z4-Uc
for 61386 <at> debbugs.gnu.org; Thu, 09 Feb 2023 12:24:06 -0500
Received: by mail-wr1-f42.google.com with SMTP id j23so2560421wra.0
for <61386 <at> debbugs.gnu.org>; Thu, 09 Feb 2023 09:24:05 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=CJGa51ThdTVfDv3LsTWQdsTmuSH3otSsPfXWm9pWBzM=;
b=NQOF4DFM3m/GEgwKOPgvY1o2K+TBqTfV8zCI6Q0Y/FNaGCgCBOSbb8RQBRNJRXDbNq
SWUC+lQ4GuyCFDr1MnMKanduTOvp37MqEJwLIUPNxssybHHDpUUuFAmYkWIz53D2niJt
rG2wFIdRAMEcSjgmSOXIH7Vkp3hUnXIMJbqw1uXfbeamnmcyDFnp+WQNIZ1py5AqRslE
/CuuX9U4kK6sDp50zUlzzGrkVeOcW9UgIJq+qquuQX7+KsF5NiwR3lUgkYpmw6UEpRkQ
VLtVWRh2qEwLr7wBw5+qV4z/Zl0rBqg/jtFeOj79Eyve5WYVJ1Ee5ORLjsyjtri+0SIp
91yg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:references:message-id:content-transfer-encoding:cc:date
:in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=CJGa51ThdTVfDv3LsTWQdsTmuSH3otSsPfXWm9pWBzM=;
b=fLdWhGVZQ0H2Z1i4BYoZ5ZdNGZ/ka5W6CN8Ty3tUI8O9gP8op8hnFCRjUsuaFu3tff
oyXU32x13hrz/3wGNlekhcn7lgOkynkv2hraJQ0DxB5rjugZQVtP4nwvIIVL4iOWDXVf
ZZogkFY0J56rtATRn04RlbfwBOpUpCaZrn2IuirIgVn60XendwkV75Ve7TxxRYzKUXiL
S1oyAEIrTQN4wTyJSaid8nGeA8KhNpsMvJgY2VvOHfIlmY25gf9qnrRivkYmNCBbxLg0
lES/FR9gzYpiA/6/szo7aOsjvkd+HFZhMLSGBhGRv/eQXxG1Mr/wI6in8GObb6ndhgx7
R6EQ==
X-Gm-Message-State: AO0yUKXw5swTr/qxGmzy9/NMQet+/pbT28xprRFOCudl56HsHKOugRhL
z/bkF3MOFk6Ww2lldxsB4+M=
X-Google-Smtp-Source: AK7set8gBh0SgaukxhltgYBkrhO1NOIs8jYSqZpoxpDy8/yHV0jYU/ckAmcRASquIO0fy3adQuFiQw==
X-Received: by 2002:adf:f586:0:b0:2bf:d52a:2f39 with SMTP id
f6-20020adff586000000b002bfd52a2f39mr7912507wro.5.1675963439741;
Thu, 09 Feb 2023 09:23:59 -0800 (PST)
Received: from smtpclient.apple (212-39-89-145.ip.btc-net.bg. [212.39.89.145])
by smtp.gmail.com with ESMTPSA id
d15-20020a5d6dcf000000b002c3f9404c45sm1721703wrz.7.2023.02.09.09.23.58
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Thu, 09 Feb 2023 09:23:58 -0800 (PST)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
From: George Valkov <gvalkov@HIDDEN>
In-Reply-To: <cddf680c-5544-4da8-f8fa-dbaee0c4aaaf@HIDDEN>
Date: Thu, 9 Feb 2023 19:23:57 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <BAC6C40E-ED03-4121-97C4-9B0E53191F61@HIDDEN>
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<675808b7-58af-6419-7e91-372e556c2ad2@HIDDEN>
<21A1241D-B890-4796-926F-E8712E2A807B@HIDDEN>
<cddf680c-5544-4da8-f8fa-dbaee0c4aaaf@HIDDEN>
To: =?utf-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
X-Mailer: Apple Mail (2.3696.120.41.1.2)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 61386
X-Mailman-Approved-At: Thu, 09 Feb 2023 12:38:37 -0500
Cc: 61386 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
> On 2023-02-09, at 6:32 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>=20
> On 09/02/2023 15:57, George Valkov wrote:
>>> On 2023-02-09, at 1:56 PM, P=C3=A1draig Brady <P@HIDDEN> =
wrote:
>>>=20
>>> On 09/02/2023 09:20, George Valkov wrote:
>>>> Due to a bug in macOS, sparse copies are corrupted on virtual disks
>>>> formatted with APFS. HFS is not affected. Affected are coreutils
>>>> install, and gcp when compiled with SEEK_HOLE, as well as macOS =
Finder.
>>>> While reading the entire file returns valid data, scanning for
>>>> allocated segments may return holes where valid data is present.
>>>> In this case a sparse copy does not contain these segments and =
returns
>>>> zeroes instead. Once the virtual disk is dismounted and then
>>>> mounted again, a sparse copy produces correct results.
>>>> This breaks OpenWRT build on macOS. Details:
>>>> https://github.com/openwrt/openwrt/pull/11960
>>>> =
https://github.com/openwrt/openwrt/pull/11960#issuecomment-1423185579
>>>> Signed-off-by: Georgi Valkov <gvalkov@HIDDEN>
>>>> ---
>>>> src/copy.c | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>> diff --git a/src/copy.c b/src/copy.c
>>>> index e16fedb28..4f56138a6 100644
>>>> --- a/src/copy.c
>>>> +++ b/src/copy.c
>>>> @@ -1065,7 +1065,7 @@ infer_scantype (int fd, struct stat const =
*sb,
>>>> return PLAIN_SCANTYPE;
>>>> }
>>>> -#ifdef SEEK_HOLE
>>>> +#if defined(SEEK_HOLE) && !defined(__APPLE__)
>>>> scan_inference->ext_start =3D lseek (fd, 0, SEEK_DATA);
>>>> if (0 <=3D scan_inference->ext_start || errno =3D=3D ENXIO)
>>>> return LSEEK_SCANTYPE;
>>>=20
>>>=20
>>> Thanks for the detailed report.
>>> The patch might very well be appropriate.
>> Hi! Let=E2=80=99s test the ideas you have first, and fall-back to the =
patch.
>> In October 2021 macOS Finder was also affected, and that points =
directly at Apple.
>> I tested again today, they have fixed Finder. After performing a copy =
in Finder,
>> coreutils cp produces a good copy. I have to run
>> make toolchain/gcc/initial/{clean,compile} -j 16
>> before I can reproduce it again with the same file.
>> So while Apple didn't fix the underlaying issue with APFS, they did
>> provide a solution for Finder. And we can make coreutils work =
correctly too.
>=20
> That suggests that Finder may have sync'd the file.
> Now syncing has overheads of course, so not an option to take lightly.
#include "stdio.h"
#include "unistd.h"
#include "fcntl.h"
int main(int argc, char ** argv)
{
sync();
int fd =3D open("cc1", O_RDWR);
//int fd =3D open("/Users/g/vhd/coreutils.sparseimage", O_RDWR);
int a =3D fdatasync(fd);
int b =3D fsync(fd);
int c =3D close(fd);
printf("fdatasync %u fsync %u close %u\n", a, b, c);
return 0;
}
fdatasync 0 fsync 0 close 0
I agree. It takes about one second to run this code.
All calls are successful. The sparse copy after that is still corrupted.
I also tried doing this on the sparse image while it is mounted.
> We may be safer just doing the normal copy on __APPLE__ as per your =
orig patch.
> A dtruss of Finder might be instructive BTW.
Yes, sync is definitely not good for performance. What is =E2=80=98dtruss'=
?
> Why I suspect syncing is that old fiemap code we used to have in copy
> needed the FIEMAP_FLAG_SYNC flag to avoid bugs like this on some file =
systems.
I=E2=80=99m not familiar with the coreutils code. Check an old commit.
>>> We avoided copy_file_range() for a long time on Linux for example
>>> until it became usable.
>>>=20
>>> Thanks for confirming the latest git is also impacted.
>> Yes, I tested on master today.
>>> I see you're on macOS 12.
>>> macOS 13 supports fclonefileat() which may
>>> avoid the issue, or also be impacted?
>> Yes, I use macOS 12. There are too many complains about 13.
>> I can boot macOS 13 recovery, so that might be an option,
>> if I can setup a working build environment.
>> Can you invite someone with macOS 13 to test the new API?
>> Github has hosted runners. But I=E2=80=99ve never used one. And I =
think they use macOS 12.
>=20
> OK let's assume fclonefileat() on macOS 13 is fine.
> That's just a thing to consider if one is testing on macOS 13,
> rather than jumping through hoops to test it.
>=20
>>> I wonder might an fdatasync(fd) avoid your issue,
>>> which we might do if defined(__APPLE__)?
>> Send me patches, and I will test them.
>> #ifdef SEEK_HOLE
>> fdatasync(fd); // this did not work, is fd writable? Do we need to =
call it on the disk image?
>> scan_inference->ext_start =3D lseek (fd, 0, SEEK_DATA);
>> if (0 <=3D scan_inference->ext_start || errno =3D=3D ENXIO)
>> return LSEEK_SCANTYPE;
>=20
>=20
> Oh right interesting. When you say doesn't work, do you mean the sync
> fails, or just doesn't help. Note the separate coreutils sync util
> only opens with write mode on AIX and Cygwin.
fdatasync returns 0, which means success
The sparse copy after that is still corrupted.
> Relatedly you could just try that external `sync file && cp file dest` =
also to test the sync hypothesis.
I tried these on cc1 and on the sparse image file, the copy still gets =
corrupted.
sync -d cc1
sync -f cc1
sync
> If the sync doesn't work, or requires write mode
> then that might be enough to decide to just avoid SEEK_DATA on =
__APPLE__ for now.
manpage: On some UNIX systems (but not Linux), fd must be a writable =
file descriptor.
I=E2=80=99m not sure about macOS. The call succeeds if we open with =
O_RDONLY,
so I guess we don=E2=80=99t need a writable file descriptor.
>> It would be best if we test on an Intel Mac with T2 chip. I use MBP =
2019 16=E2=80=9D.
>> That security chip and the encrypted physical disk with APFS might =
also affect our results.
>> Georgi Valkov
>> httpstorm.com
>> nano RTOS
>> off-topic: can you please point me at API to take a disk offline or =
lock it for exclusive access?
>> I wrote a disk cloning tool and I have that functionality on Windows, =
I=E2=80=99d like to add it to Linux, BSD and macOS.
>=20
>=20
> Note it's best to keep this on list.
I apologise, I accidentally used replay instead of replay all.
Is there a safe way to add the group once a message is sent?
Good luck!
Georgi Valkov
httpstorm.com
nano RTOS
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at 61386) by debbugs.gnu.org; 9 Feb 2023 16:32:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 09 11:32:24 2023
Received: from localhost ([127.0.0.1]:33473 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQ9qR-0004AD-SN
for submit <at> debbugs.gnu.org; Thu, 09 Feb 2023 11:32:24 -0500
Received: from mail-wm1-f53.google.com ([209.85.128.53]:39854)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <pixelbeat@HIDDEN>) id 1pQ9qQ-0004A1-74
for 61386 <at> debbugs.gnu.org; Thu, 09 Feb 2023 11:32:22 -0500
Received: by mail-wm1-f53.google.com with SMTP id
hn2-20020a05600ca38200b003dc5cb96d46so4280169wmb.4
for <61386 <at> debbugs.gnu.org>; Thu, 09 Feb 2023 08:32:22 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=content-transfer-encoding:in-reply-to:from:references:to
:content-language:subject:user-agent:mime-version:date:message-id
:sender:from:to:cc:subject:date:message-id:reply-to;
bh=M8e/A8Z6US8HBItQ5eDCfyjTHBFBAxffDVtP9VVJLLU=;
b=T336h9dp3+MFuVndQLQE2pFwMpfQhiU+gbPJnHCUMVUC9RZsYregxZz6KwD6HAJiB1
qBLPb8C7qFR/OUP1If75oR0bGKfz/jPYt4bREOBvc8jgysYRKBYQZS1zkBuYzNPxPDCI
DI03e7DMyvtw3zXh8519gtel/PhyYU96fm9p32iU4cvBN9BFnlpX/zx20X5HDYA22pmE
Cj078sOaYNHyjuIq2ri4u7985R2C0EdBMchzfeJHp9ZAKw4lFTDvYqFDrbPKugg02u7X
aGsGpxhKe/Qt4sdDwA1dTlEJjnjR06OGbbudgfFB+24BTKGp2fLWRYVcVzH7v5ghfjm/
yIBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=content-transfer-encoding:in-reply-to:from:references:to
:content-language:subject:user-agent:mime-version:date:message-id
:sender:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=M8e/A8Z6US8HBItQ5eDCfyjTHBFBAxffDVtP9VVJLLU=;
b=kQKji5BjlBQInlH8H7KBRllB+SoOZ6Mr7IVnp4/EAIjX6I/M8CTwt9NM/ouA4JPwFu
tfjVHr2tqpo4l4kcrlUI9/knJeNEiNmWykN+TjDKFSz2Ao3LO+8TJVoj6GM3s5ieHR5d
7g1Muc6CMJjykF1tTmO0zQey4Zm85jANo7to6gMxhpnRVUzGDm0qI5hk8SnYnGinr1jR
bCGrD4XzQXX9IDl8g40d7UN1xsmg+A6nnyckcmVOz7aypxHESwMEWHV5vGTeo2X532lX
N5IdLv0AXSmLDAsg21ScjvW1SLVlXjBKdOjFMbipcyyaR2ojgqGq0EgEBpyFlYX6uB4k
b4MQ==
X-Gm-Message-State: AO0yUKV9FBz0DdJIvtqJpBEt1bJNUu8Ccdcs66ZCYUrZKlKPJ5l/Dz/o
lqwLPv1vz5OWNpAkv2PpT+I=
X-Google-Smtp-Source: AK7set/SbhkPnH0YoXUM+F+gfMyeweV896BRKjM1t73DwpvWLMPUj1+QT90c1/M1fxfloLAuPo7/eA==
X-Received: by 2002:a05:600c:331e:b0:3dc:5823:d6c0 with SMTP id
q30-20020a05600c331e00b003dc5823d6c0mr10240777wmp.6.1675960336226;
Thu, 09 Feb 2023 08:32:16 -0800 (PST)
Received: from [192.168.1.9]
(95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175])
by smtp.googlemail.com with ESMTPSA id
h2-20020a05600c350200b003dc434b39c7sm9815822wmq.0.2023.02.09.08.32.15
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Thu, 09 Feb 2023 08:32:15 -0800 (PST)
Message-ID: <cddf680c-5544-4da8-f8fa-dbaee0c4aaaf@HIDDEN>
Date: Thu, 9 Feb 2023 16:32:15 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101
Thunderbird/109.0
Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS
Content-Language: en-US
To: George Valkov <gvalkov@HIDDEN>, 61386 <at> debbugs.gnu.org
References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
<675808b7-58af-6419-7e91-372e556c2ad2@HIDDEN>
<21A1241D-B890-4796-926F-E8712E2A807B@HIDDEN>
From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN>
In-Reply-To: <21A1241D-B890-4796-926F-E8712E2A807B@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.9 (/)
X-Debbugs-Envelope-To: 61386
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.9 (-)
On 09/02/2023 15:57, George Valkov wrote:
>
>
>> On 2023-02-09, at 1:56 PM, Pádraig Brady <P@HIDDEN> wrote:
>>
>> On 09/02/2023 09:20, George Valkov wrote:
>>> Due to a bug in macOS, sparse copies are corrupted on virtual disks
>>> formatted with APFS. HFS is not affected. Affected are coreutils
>>> install, and gcp when compiled with SEEK_HOLE, as well as macOS Finder.
>>> While reading the entire file returns valid data, scanning for
>>> allocated segments may return holes where valid data is present.
>>> In this case a sparse copy does not contain these segments and returns
>>> zeroes instead. Once the virtual disk is dismounted and then
>>> mounted again, a sparse copy produces correct results.
>>> This breaks OpenWRT build on macOS. Details:
>>> https://github.com/openwrt/openwrt/pull/11960
>>> https://github.com/openwrt/openwrt/pull/11960#issuecomment-1423185579
>>> Signed-off-by: Georgi Valkov <gvalkov@HIDDEN>
>>> ---
>>> src/copy.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>> diff --git a/src/copy.c b/src/copy.c
>>> index e16fedb28..4f56138a6 100644
>>> --- a/src/copy.c
>>> +++ b/src/copy.c
>>> @@ -1065,7 +1065,7 @@ infer_scantype (int fd, struct stat const *sb,
>>> return PLAIN_SCANTYPE;
>>> }
>>> -#ifdef SEEK_HOLE
>>> +#if defined(SEEK_HOLE) && !defined(__APPLE__)
>>> scan_inference->ext_start = lseek (fd, 0, SEEK_DATA);
>>> if (0 <= scan_inference->ext_start || errno == ENXIO)
>>> return LSEEK_SCANTYPE;
>>
>>
>> Thanks for the detailed report.
>> The patch might very well be appropriate.
>
> Hi! Let’s test the ideas you have first, and fall-back to the patch.
>
> In October 2021 macOS Finder was also affected, and that points directly at Apple.
> I tested again today, they have fixed Finder. After performing a copy in Finder,
> coreutils cp produces a good copy. I have to run
> make toolchain/gcc/initial/{clean,compile} -j 16
> before I can reproduce it again with the same file.
>
> So while Apple didn't fix the underlaying issue with APFS, they did
> provide a solution for Finder. And we can make coreutils work correctly too.
That suggests that Finder may have sync'd the file.
Now syncing has overheads of course, so not an option to take lightly.
We may be safer just doing the normal copy on __APPLE__ as per your orig patch.
A dtruss of Finder might be instructive BTW.
Why I suspect syncing is that old fiemap code we used to have in copy
needed the FIEMAP_FLAG_SYNC flag to avoid bugs like this on some file systems.
>> We avoided copy_file_range() for a long time on Linux for example
>> until it became usable.
>>
>> Thanks for confirming the latest git is also impacted.
>
> Yes, I tested on master today.
>
>> I see you're on macOS 12.
>> macOS 13 supports fclonefileat() which may
>> avoid the issue, or also be impacted?
>
> Yes, I use macOS 12. There are too many complains about 13.
> I can boot macOS 13 recovery, so that might be an option,
> if I can setup a working build environment.
> Can you invite someone with macOS 13 to test the new API?
> Github has hosted runners. But I’ve never used one. And I think they use macOS 12.
OK let's assume fclonefileat() on macOS 13 is fine.
That's just a thing to consider if one is testing on macOS 13,
rather than jumping through hoops to test it.
>> I wonder might an fdatasync(fd) avoid your issue,
>> which we might do if defined(__APPLE__)?
>
> Send me patches, and I will test them.
>
> #ifdef SEEK_HOLE
> fdatasync(fd); // this did not work, is fd writable? Do we need to call it on the disk image?
> scan_inference->ext_start = lseek (fd, 0, SEEK_DATA);
> if (0 <= scan_inference->ext_start || errno == ENXIO)
> return LSEEK_SCANTYPE;
Oh right interesting. When you say doesn't work, do you mean the sync
fails, or just doesn't help. Note the separate coreutils sync util
only opens with write mode on AIX and Cygwin.
Relatedly you could just try that external `sync file && cp file dest` also to test the sync hypothesis.
If the sync doesn't work, or requires write mode
then that might be enough to decide to just avoid SEEK_DATA on __APPLE__ for now.
> It would be best if we test on an Intel Mac with T2 chip. I use MBP 2019 16”.
> That security chip and the encrypted physical disk with APFS might also affect our results.
>
> Georgi Valkov
> httpstorm.com
> nano RTOS
>
> off-topic: can you please point me at API to take a disk offline or lock it for exclusive access?
> I wrote a disk cloning tool and I have that functionality on Windows, I’d like to add it to Linux, BSD and macOS.
Note it's best to keep this on list.
cheers,
Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.Received: (at 61386) by debbugs.gnu.org; 9 Feb 2023 11:56:16 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 09 06:56:16 2023 Received: from localhost ([127.0.0.1]:57745 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pQ5XE-0005W9-7t for submit <at> debbugs.gnu.org; Thu, 09 Feb 2023 06:56:16 -0500 Received: from mail-wm1-f47.google.com ([209.85.128.47]:37420) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pixelbeat@HIDDEN>) id 1pQ5X9-0005Vo-PK for 61386 <at> debbugs.gnu.org; Thu, 09 Feb 2023 06:56:15 -0500 Received: by mail-wm1-f47.google.com with SMTP id bg13-20020a05600c3c8d00b003d9712b29d2so3682190wmb.2 for <61386 <at> debbugs.gnu.org>; Thu, 09 Feb 2023 03:56:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=leRrvZWopItkoFn1zJZYJjHd5q62uvesKX3eJ5A22a8=; b=ZSCsyUQP2dn/zn/XO6n2EFSCTFj6rbsP85TwKuhMkcRGeYiRIu7jbEaqKV8CUpOfaE BQQ1RtOAAAoAzE4/p8Vorz/YswHEdfbsnjar6XAN+q9Nd11KRU2lcK+a5OybodSlrppw VtzXfi6ArY4luDMxCVAdB5aymOdr51cNg0UmgfcfF4shDDZaq6QPtxFaVcFHrLQOXcku vZieoRnWWODpCvyh4Hm99PnsSr9MNQmtgYeRGGAVQH2XULOrcjknBqqBcsBtHgIvKcHl N/DdQB9Y7lAT8QvYZpgMawBobfdSbRV5/2mpIsmomjzYtaSQjT0ZtYzebo9/7WNlqLcM RKIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=leRrvZWopItkoFn1zJZYJjHd5q62uvesKX3eJ5A22a8=; b=OPVtzioB0uC1KKNYam01Vv2EuTftXGnGVIfeqYyxPm+HRCtbItvROjjQOD/AA5/IU1 FXp2l0w4eOVPPtTMDeh3HdS1p/S77yS8Y5zchGDb/SHjAYnRIEEnJUeqKcD7FDIc6r5R OPXP8h5dzyhm3E4xOt7U8tBmM9jOOlt6vldULb3y70/TwTVUEdPF6sZTsxkCujaZb7VV T/+vlpE5t+43CKgs0QF+8PQJGFeSYRTeE5+u5Ek7c/rQLxERysxmo3UeZzHegeyLczoG OyWPDWkd3+xYrOS9fHxbUdXLTZpCZAiS+qvz684ukp+iB4rL4ZEYFB3bnt6Y22yaHGY7 R2Aw== X-Gm-Message-State: AO0yUKVndXcqQv5hYuKjvElWMnLMZHW8OF9rgsCeVnGyrVV4tK+PGFk5 uDVRXHDBrTejaSBjHEXDh+8= X-Google-Smtp-Source: AK7set+GRp0CbL2xsJ6pntkHDPrnuF5AAZl9mZd5KIjOEdCQwC63bp1hWKYeZgT6tRDJ0y+ABDmbrg== X-Received: by 2002:a05:600c:1716:b0:3df:f7cc:4da2 with SMTP id c22-20020a05600c171600b003dff7cc4da2mr9642922wmn.16.1675943765811; Thu, 09 Feb 2023 03:56:05 -0800 (PST) Received: from [192.168.1.9] (95-44-90-175-dynamic.agg2.lod.rsl-rtd.eircom.net. [95.44.90.175]) by smtp.googlemail.com with ESMTPSA id j40-20020a05600c1c2800b003dc4480df80sm5360396wms.34.2023.02.09.03.56.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 09 Feb 2023 03:56:05 -0800 (PST) Message-ID: <675808b7-58af-6419-7e91-372e556c2ad2@HIDDEN> Date: Thu, 9 Feb 2023 11:56:04 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 Subject: Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS Content-Language: en-US To: George Valkov <gvalkov@HIDDEN>, 61386 <at> debbugs.gnu.org References: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> From: =?UTF-8?Q?P=C3=A1draig_Brady?= <P@HIDDEN> In-Reply-To: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.9 (/) X-Debbugs-Envelope-To: 61386 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.9 (-) On 09/02/2023 09:20, George Valkov wrote: > Due to a bug in macOS, sparse copies are corrupted on virtual disks > formatted with APFS. HFS is not affected. Affected are coreutils > install, and gcp when compiled with SEEK_HOLE, as well as macOS Finder. > > While reading the entire file returns valid data, scanning for > allocated segments may return holes where valid data is present. > In this case a sparse copy does not contain these segments and returns > zeroes instead. Once the virtual disk is dismounted and then > mounted again, a sparse copy produces correct results. > > This breaks OpenWRT build on macOS. Details: > https://github.com/openwrt/openwrt/pull/11960 > https://github.com/openwrt/openwrt/pull/11960#issuecomment-1423185579 > > Signed-off-by: Georgi Valkov <gvalkov@HIDDEN> > --- > src/copy.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/copy.c b/src/copy.c > index e16fedb28..4f56138a6 100644 > --- a/src/copy.c > +++ b/src/copy.c > @@ -1065,7 +1065,7 @@ infer_scantype (int fd, struct stat const *sb, > return PLAIN_SCANTYPE; > } > > -#ifdef SEEK_HOLE > +#if defined(SEEK_HOLE) && !defined(__APPLE__) > scan_inference->ext_start = lseek (fd, 0, SEEK_DATA); > if (0 <= scan_inference->ext_start || errno == ENXIO) > return LSEEK_SCANTYPE; Thanks for the detailed report. The patch might very well be appropriate. We avoided copy_file_range() for a long time on Linux for example until it became usable. Thanks for confirming the latest git is also impacted. I see you're on macOS 12. macOS 13 supports fclonefileat() which may avoid the issue, or also be impacted? I wonder might an fdatasync(fd) avoid your issue, which we might do if defined(__APPLE__)? cheers, Pádraig
bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
Received: (at submit) by debbugs.gnu.org; 9 Feb 2023 09:35:50 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 09 04:35:50 2023
Received: from localhost ([127.0.0.1]:57532 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pQ3LJ-0001XW-1J
for submit <at> debbugs.gnu.org; Thu, 09 Feb 2023 04:35:49 -0500
Received: from lists.gnu.org ([209.51.188.17]:42938)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <gvalkov@HIDDEN>) id 1pQ36j-00018S-U0
for submit <at> debbugs.gnu.org; Thu, 09 Feb 2023 04:20:46 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <gvalkov@HIDDEN>) id 1pQ36j-0001Oo-NW
for bug-coreutils@HIDDEN; Thu, 09 Feb 2023 04:20:45 -0500
Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.90_1) (envelope-from <gvalkov@HIDDEN>) id 1pQ36h-0004bU-O5
for bug-coreutils@HIDDEN; Thu, 09 Feb 2023 04:20:45 -0500
Received: by mail-wr1-x42d.google.com with SMTP id ba1so1087817wrb.5
for <bug-coreutils@HIDDEN>; Thu, 09 Feb 2023 01:20:42 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:date:message-id:subject:mime-version:content-transfer-encoding
:from:from:to:cc:subject:date:message-id:reply-to;
bh=VVJIC/7TkyeMO0DtL/QrW1Ybiy+yM4vkNcfNnHfokPo=;
b=X+jgwfHyzr4KAW4hIua92YVH58/h+UX/8q0AVlfpsiF6T8ZT1h5HpCMdhrG3KXV+zq
hDQuBOOdYtJh06k7Sh3xam6PVS7Us17fuqysG/8D3BnSBDQ5Zq97kX7ui0lCFxabIJFl
zOFx1b7r8ZQRpct3HHBlxv/sj6JAU/F/i/00FGaJ1oYleKx9f+L/oEQyZ8vG8BqqGeef
ESu7dcMgki9gMBH+POaX9WXH/seYTILX5lATf+t/xWsI127K4mSPwUN98riPN6815ARm
mM6l5XygTmM9KX+O4qHhvDMi2syoSAbSPbgMu1Jm9ND30OxVTV5xomCkJhhKw774XgTg
PMbg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:date:message-id:subject:mime-version:content-transfer-encoding
:from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=VVJIC/7TkyeMO0DtL/QrW1Ybiy+yM4vkNcfNnHfokPo=;
b=pIR7LoAls0RYvmPcRsMloZvdxphHgc6WfIBXYY15My5QqztsC3Cd+qjcWkAvlkftec
fcZEocxhFvYzKYXW2/c1OcAhYaG1ZJvMBinMtCq/Oq8O54aHFTh4wQye6bDg0mnbXyJv
t3LQtyYFld7pVO/Smd0cG97tHZGUMkl7ALBbxa3veMiwFaC/LK0XD50ZrNBP641LyBrU
+MyzAVbtDEYjs4ZTBu9vXx9AKqf7iRQjBnqNeTkJxuBbHLxUdOlvIe/S2yNojb65CTkw
HvFIgDvQkEY0NB5Wi5HRyxabFZnxR1lUZlZM9hIExEhdCUtxTg+rpW22s6P77H142CI1
mRiQ==
X-Gm-Message-State: AO0yUKU2gBroDWNDQeVoLBc3QuA9MbvuFtTsfAosi2YulcqgGgdD3/UE
XGvPJ43CuWyaY1/b79aI8kNecWmgaew=
X-Google-Smtp-Source: AK7set/GhTpBLsBla8OwTwe/0SOVXOENCH8Bg4d1dax6adfP1PsdDZdI2S7cb1uhw0I8UvYuu3KxHw==
X-Received: by 2002:a5d:4b4c:0:b0:2bf:b9b1:9568 with SMTP id
w12-20020a5d4b4c000000b002bfb9b19568mr7115225wrs.2.1675934440276;
Thu, 09 Feb 2023 01:20:40 -0800 (PST)
Received: from smtpclient.apple (212-39-89-145.ip.btc-net.bg. [212.39.89.145])
by smtp.gmail.com with ESMTPSA id
s11-20020adfdb0b000000b002c3e1e1dcd7sm730727wri.104.2023.02.09.01.20.38
for <bug-coreutils@HIDDEN>
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Thu, 09 Feb 2023 01:20:39 -0800 (PST)
From: George Valkov <gvalkov@HIDDEN>
Content-Type: text/plain;
charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\))
Subject: [PATCH] cp,mv,install: Disable sparse copy on macOS
Message-Id: <9A894CE0-00EC-44A2-A387-14BEF8BC8FCE@HIDDEN>
Date: Thu, 9 Feb 2023 11:20:37 +0200
To: bug-coreutils@HIDDEN
X-Mailer: Apple Mail (2.3696.120.41.1.2)
Received-SPF: pass client-ip=2a00:1450:4864:20::42d;
envelope-from=gvalkov@HIDDEN; helo=mail-wr1-x42d.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Thu, 09 Feb 2023 04:35:46 -0500
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)
Due to a bug in macOS, sparse copies are corrupted on virtual disks
formatted with APFS. HFS is not affected. Affected are coreutils
install, and gcp when compiled with SEEK_HOLE, as well as macOS Finder.
While reading the entire file returns valid data, scanning for
allocated segments may return holes where valid data is present.
In this case a sparse copy does not contain these segments and returns
zeroes instead. Once the virtual disk is dismounted and then
mounted again, a sparse copy produces correct results.
This breaks OpenWRT build on macOS. Details:
https://github.com/openwrt/openwrt/pull/11960
https://github.com/openwrt/openwrt/pull/11960#issuecomment-1423185579
Signed-off-by: Georgi Valkov <gvalkov@HIDDEN>
---
src/copy.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/copy.c b/src/copy.c
index e16fedb28..4f56138a6 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -1065,7 +1065,7 @@ infer_scantype (int fd, struct stat const *sb,
return PLAIN_SCANTYPE;
}
=20
-#ifdef SEEK_HOLE
+#if defined(SEEK_HOLE) && !defined(__APPLE__)
scan_inference->ext_start =3D lseek (fd, 0, SEEK_DATA);
if (0 <=3D scan_inference->ext_start || errno =3D=3D ENXIO)
return LSEEK_SCANTYPE;
--=20
2.39.1
The issue is most likely within APFS, So we need to bring it to Apple's =
attention.
Since it is triggered by building `gcc`, it would be helpful
if they can shine some light how their tools are build
and what trigger this, in order to create PoC code.
**steps to reproduce**
_Create a sparse disk image with APFS_
- open Disk Utility
- press Command-N to create a new virtual disk
- Image Format: sparse disk image
- save as: ~/vhd/test
- Name: test
- Size: 50 GB
- Format: APFS (Case-sensitive)
- Encryption: none
- Partitions: Single partition - GUID Partition Map
- Save
- the image is mounted under `/Volumes/test`
_clone and build coreutils_
``` bash
cd /Volumes/test
# patched with sparse support disabled on macOS
git clone https://github.com/httpstorm/coreutils.git
mv coreutils coreutils-no-sparse
cd coreutils-no-sparse
git checkout macos_disable-sparse-copy
git submodule foreach git pull origin master
./bootstrap
make -j 16
cd ..
# original code
git clone https://github.com/coreutils/coreutils.git
cd coreutils
git submodule foreach git pull origin master
./bootstrap
make -j 16
```
_download OpenWRT and build gcc_
``` bash
cd /Volumes/test
git clone https://github.com/openwrt/openwrt.git
cd openwrt
./scripts/feeds update -a
./scripts/feeds install -a
make defconfig
make menuconfig
# Target System (Marvell EBU Armada)
# Target Profile (Linksys WRT3200ACM)
# Save, Exit
make tools/compile -j 16
make toolchain/gcc/initial/compile -j 16
```
proof of concept
``` bash
cd /Volumes/test/openwrt
cd =
build_dir/toolchain-arm_cortex-a9+vfpv3-d16_gcc-12.2.0_musl_eabi/gcc-12.2.=
0-initial/gcc
/Volumes/test/coreutils/src/cp cc1 cc1-ori
./cc1-ori --help
-bash: ./cc1-ori: cannot execute binary file: Exec format error
/Volumes/test/coreutils-no-sparse/src/cp cc1 cc1-fix
./cc1-fix --help
# works correctly
sha1sum cc1
# fcf5d9458f54d5588efc5a76a89388925ca04eda cc1
sha1sum cc1-fix=20
# fcf5d9458f54d5588efc5a76a89388925ca04eda cc1-fix
sha1sum cc1-ori
# 7b447132f56f3b549ef62a4780945e82d26f7d2c cc1-ori
```
Kind regards,
Georgi Valkov
httpstorm.com
nano RTOS
George Valkov <gvalkov@HIDDEN>:bug-coreutils@HIDDEN.
Full text available.bug-coreutils@HIDDEN:bug#61386; Package coreutils.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.