GNU bug report logs - #61161
package-input-rewriting causes different ordering of grafts

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 30 Jan 2023 01:17:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 29 20:17:48 2023
Received: from localhost ([127.0.0.1]:45643 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1pMIns-00005f-CJ
	for submit <at> debbugs.gnu.org; Sun, 29 Jan 2023 20:17:48 -0500
Received: from lists.gnu.org ([209.51.188.17]:55100)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <akira@HIDDEN>) id 1pMInq-00005X-4o
 for submit <at> debbugs.gnu.org; Sun, 29 Jan 2023 20:17:46 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <akira@HIDDEN>)
 id 1pMInp-00008z-Qd
 for bug-guix@HIDDEN; Sun, 29 Jan 2023 20:17:45 -0500
Received: from ms11p00im-qufo17281401.me.com ([17.58.38.51])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <akira@HIDDEN>)
 id 1pMInn-0006GK-Pc
 for bug-guix@HIDDEN; Sun, 29 Jan 2023 20:17:45 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akirakyle.com;
 s=sig1; t=1675041461;
 bh=5rr+y4prxqYDcPcgiRlBqK4kkCkFKB8cY04pJ9D5Sio=;
 h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type;
 b=e5NZ5VBSbmTTMVy5U0A+VL+JNeAOjSetuW2Z3uLSPtfppMWeJ7fLRFZMNceqSX+f8
 K75BVdbZ6X17DQAzJ5AL8TmEuxsJ5rgNFoAmZ8aO2m2hS+P8zlO8qsNutJNIZUfWcR
 C31dD8Fj+9rAXKhSAn1r5EmWjuNQhhUUGnK39TRWdMej79JLDpKFHg1XhFc0rmU1jP
 NZJ/zovI/znwWI9lK7rSD+rQ6flnFQfP46mGY28L/rHWfUhJj2VblpXqZ1XLicDHmG
 hpevJ/utPA8Fx9WBzyiZjBbyMGMjnlviTLQgWXxN7x1KKjNX9iqZJ7yqTmJ0s5Wwmy
 IP0Ck5p05L07g==
Received: from data (ms11p00im-dlb-asmtpmailmevip.me.com [17.57.154.19])
 by ms11p00im-qufo17281401.me.com (Postfix) with ESMTPSA id D3D19BA03EE
 for <bug-guix@HIDDEN>; Mon, 30 Jan 2023 01:17:40 +0000 (UTC)
User-agent: mu4e 1.9.16; emacs 29.0.50
From: Akira Kyle <akira@HIDDEN>
To: bug-guix@HIDDEN
Subject: package-input-rewriting causes different ordering of grafts
Date: Sun, 29 Jan 2023 17:59:57 -0700
Message-ID: <875ycox1l8.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; format=flowed
X-Proofpoint-ORIG-GUID: zy-L2FKCIeHxRZVjefUqoi1NYekoErCR
X-Proofpoint-GUID: zy-L2FKCIeHxRZVjefUqoi1NYekoErCR
X-Proofpoint-Virus-Version: =?UTF-8?Q?vendor=3Dfsecure_engine=3D1.1.170-22c6f66c430a71ce266a39bfe25bc?=
 =?UTF-8?Q?2903e8d5c8f:6.0.138,18.0.572,17.0.605.474.0000000_definitions?=
 =?UTF-8?Q?=3D2020-02-14=5F11:2020-02-14=5F02,2020-02-14=5F11,2020-01-23?=
 =?UTF-8?Q?=5F02_signatures=3D0?=
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=534
 phishscore=0 mlxscore=0
 spamscore=0 clxscore=1030 bulkscore=0 adultscore=0 suspectscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2209130000 definitions=main-2301300010
Received-SPF: pass client-ip=17.58.38.51; envelope-from=akira@HIDDEN;
 helo=ms11p00im-qufo17281401.me.com
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)


I think this may be a strange, subtle bug and hopefully this is 
sufficient to reproduce.
This was all done with guix commit 
da9af8c72fe427e3eeb718b53c599139a8958d61.

Consider the following:

(use-modules (guix packages)
             (guix gexp)
             (guix monads)
             (guix store)
             (guix git)
             (guix download)
             (guix git-download)
             (gnu packages compression)
             (gnu packages emacs-xyz))

(define (package-commit pkg commit checksum)
  "Return a package variant using the given commit and sha256."
  (package
    (inherit pkg)
    (name (package-name pkg))
    (version (substring commit 0 7))
    (source
     (origin
       (method git-fetch)
       (uri (git-reference
             (url (git-checkout-url (git-reference->git-checkout
                                     (origin-uri (package-source 
                                     pkg)))))
             (commit commit)))
       (sha256 (base32 checksum))
       (file-name (git-file-name name version))))))

(define emacs-magit-latest 
  (package-commit emacs-magit
                  "45be64e5f1ef0d03156c53543eb198829177a934"
                  "19b1d4200x1rhffmxf834kh55nj2zjps09hkkhw9hrsdig3nlfl9"))

(define emacs-compat-latest
  (package
    (inherit emacs-compat)
    (version "29.1.3.0")
    (source (origin
              (method url-fetch)
              (uri (string-append "https://elpa.gnu.org/packages/"
                                  "compat-" version ".tar.lz"))
              (sha256
               (base32
                "0jnk81rg5w6zhf413nf3j72i2mr8vfdms55gahrdx28727w8gfj8"))))
    (native-inputs (modify-inputs (package-native-inputs 
    emacs-compat)
                         (prepend lzip)))))

(define with-replacements
  (package-input-rewriting `((,emacs-compat 
  . ,emacs-compat-latest))))


(list
 emacs-magit-latest
 (with-replacements emacs-magit-latest)
 )

emacs-magit-latest depends on
/gnu/store/xrwyv6s8gsazgb2hqxxvh3h6d9cvkxal-git-2.39.1.drv

whereas (with-replacements emacs-magit-latest)` depends on
/gnu/store/x2gd1dsqvpas4rwvlxdmj7p9w2d2ba49-git-2.39.1.drv

Which is surprising since I didn't change anything about the git 
package.

Following this a bit further reveals that the difference comes 
down to the grafts being applied to python where 
emacs-magit-latest depends on
/gnu/store/848hfdz4xj91jail0y8lfj22dkgb3d12-python-3.9.9-builder


whereas (with-replacements emacs-magit-latest)` depends on
/gnu/store/f3b6wlkwnnp2zfm20br6hbc9i0pipi24-python-3.9.9-builder


Inspecting these derivations, the difference between them seems to 
be that in the former has zlib first in the `mapping` field of the 
graft while in the latter zlib is last. My guess is this is due to 
the fact that the emacs-compat override adds lzip to its 
native-inputs and causes a different traversal order for grafting 
zlib. But this is my first time attempting to look at the grafting 
machinery in detail, so hopefully the experts can figure out if 
this really is a bug and how to fix it!

I don't have a concrete example where this causes issues in 
practice, since here the two packages would conflict anyways due 
to different compat versions, but it seems like that's mostly a 
consequence of emacs-packages having propagated dependencies. At 
the very least it would seem to cause unnecessary duplication in 
the store.