Package: guix;
Reported by: Mark H Weaver <mhw <at> netris.org>
Date: Sun, 1 Mar 2015 05:07:01 UTC
Severity: important
Tags: patch
Done: ludo <at> gnu.org (Ludovic Courtès)
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19973 in the body.
You can then email your comments to 19973 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-guix <at> gnu.org:bug#19973; Package guix.
(Sun, 01 Mar 2015 05:07:01 GMT) Full text and rfc822 format available.Mark H Weaver <mhw <at> netris.org>:bug-guix <at> gnu.org.
(Sun, 01 Mar 2015 05:07:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Mark H Weaver <mhw <at> netris.org> To: bug-guix <at> gnu.org Subject: Grafts break debug outputs Date: Sun, 01 Mar 2015 00:06:11 -0500
Grafts break debug outputs, or at least that's my guess.
Here's what I see:
mhw <at> jojen:~$ guix build guile
guix build: warning: ambiguous package specification `guile'
guix build: warning: choosing guile-2.0.11 from gnu/packages/guile.scm:110:2
/gnu/store/3lhr8q28q6f59774di9av7ncy09jd55d-guile-2.0.11
/gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug
mhw <at> jojen:~$ guix package -I guile
guile 2.0.11 out /gnu/store/3lhr8q28q6f59774di9av7ncy09jd55d-guile-2.0.11
guile 2.0.11 debug /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug
mhw <at> jojen:~$ ls -l .guix-profile/lib/debug/gnu/store/
total 8
lrwxrwxrwx 12 root guixbuild 128 Dec 31 1969 122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11 -> /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug/lib/debug/gnu/store/122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11
lrwxrwxrwx 15 root guixbuild 127 Dec 31 1969 3g20rdmnavpblsmgppyl8jhg67nidhjk-glibc-2.20 -> /gnu/store/hrny2whqg9c3m0klyfpbmmcyiir9yf8m-gcc-toolchain-4.9.2/lib/debug/gnu/store/3g20rdmnavpblsmgppyl8jhg67nidhjk-glibc-2.20
I guess GDB can't find the debugging information because
122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11 is the name of the guile
directory *before* grafting.
To work around this, I created my own ~/mydebug directory:
mhw <at> jojen:~/mydebug/gnu/store$ ln -s /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug/lib/debug/gnu/store/122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11 3lhr8q28q6f59774di9av7ncy09jd55d-guile-2.0.11
and then pointed GDB at ~/mydebug and then debugging works again.
Mark
bug-guix <at> gnu.org:bug#19973; Package guix.
(Sun, 01 Mar 2015 05:25:02 GMT) Full text and rfc822 format available.Message #8 received at 19973 <at> debbugs.gnu.org (full text, mbox):
From: Mark H Weaver <mhw <at> netris.org> To: 19973 <at> debbugs.gnu.org Subject: Re: bug#19973: Grafts break debug outputs Date: Sun, 01 Mar 2015 00:24:57 -0500
Mark H Weaver <mhw <at> netris.org> writes:
> Grafts break debug outputs, or at least that's my guess.
> Here's what I see:
>
> mhw <at> jojen:~$ guix build guile
> guix build: warning: ambiguous package specification `guile'
> guix build: warning: choosing guile-2.0.11 from gnu/packages/guile.scm:110:2
> /gnu/store/3lhr8q28q6f59774di9av7ncy09jd55d-guile-2.0.11
> /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug
> mhw <at> jojen:~$ guix package -I guile
> guile 2.0.11 out /gnu/store/3lhr8q28q6f59774di9av7ncy09jd55d-guile-2.0.11
> guile 2.0.11 debug /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug
> mhw <at> jojen:~$ ls -l .guix-profile/lib/debug/gnu/store/
> total 8
> lrwxrwxrwx 12 root guixbuild 128 Dec 31 1969
> 122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11 ->
> /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug/lib/debug/gnu/store/122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11
> lrwxrwxrwx 15 root guixbuild 127 Dec 31 1969
> 3g20rdmnavpblsmgppyl8jhg67nidhjk-glibc-2.20 ->
> /gnu/store/hrny2whqg9c3m0klyfpbmmcyiir9yf8m-gcc-toolchain-4.9.2/lib/debug/gnu/store/3g20rdmnavpblsmgppyl8jhg67nidhjk-glibc-2.20
>
> I guess GDB can't find the debugging information because
> 122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11 is the name of the guile
> directory *before* grafting.
>
> To work around this, I created my own ~/mydebug directory:
>
> mhw <at> jojen:~/mydebug/gnu/store$ ln -s
> /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug/lib/debug/gnu/store/122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11
> 3lhr8q28q6f59774di9av7ncy09jd55d-guile-2.0.11
>
> and then pointed GDB at ~/mydebug and then debugging works again.
I've since discovered that because our grafting is not done properly,
for now we need symbolic links for both the pre-grafted and post-grafted
directories.
I installed libgc:debug and then created a proper symlink in ~/mydebug,
but because our guile is still linked against the pre-grafted libgc, we
need a symbolic link with the pre-grafted name. So here's what I have
now:
mhw <at> jojen:~/mydebug/gnu/store$ ls -l
total 12
lrwxrwxrwx 1 mhw mhw 128 Mar 1 00:00 3lhr8q28q6f59774di9av7ncy09jd55d-guile-2.0.11 -> /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug/lib/debug/gnu/store/122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11
lrwxrwxrwx 1 mhw mhw 126 Mar 1 00:18 h6byyxpii73xlbhiq5lxfq3n6qzqwsif-libgc-7.4.2 -> /gnu/store/k0xjv8l196j4nxgbz8dxz6xz7kbysaix-libgc-7.4.2-debug/lib/debug/gnu/store/zshjm1ilbzz0z0zpxr17h13jnmilga7l-libgc-7.4.2
lrwxrwxrwx 1 mhw mhw 126 Mar 1 00:21 zshjm1ilbzz0z0zpxr17h13jnmilga7l-libgc-7.4.2 -> /gnu/store/k0xjv8l196j4nxgbz8dxz6xz7kbysaix-libgc-7.4.2-debug/lib/debug/gnu/store/zshjm1ilbzz0z0zpxr17h13jnmilga7l-libgc-7.4.2
Mark
bug-guix <at> gnu.org:bug#19973; Package guix.
(Tue, 07 Mar 2017 21:55:01 GMT) Full text and rfc822 format available.Message #11 received at 19973 <at> debbugs.gnu.org (full text, mbox):
From: ludo <at> gnu.org (Ludovic Courtès) To: Mark H Weaver <mhw <at> netris.org> Cc: 19973 <at> debbugs.gnu.org Subject: Re: bug#19973: Grafts break debug outputs Date: Tue, 07 Mar 2017 22:54:39 +0100
Mark H Weaver <mhw <at> netris.org> skribis: > mhw <at> jojen:~$ guix build guile > guix build: warning: ambiguous package specification `guile' > guix build: warning: choosing guile-2.0.11 from gnu/packages/guile.scm:110:2 > /gnu/store/3lhr8q28q6f59774di9av7ncy09jd55d-guile-2.0.11 > /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug > mhw <at> jojen:~$ guix package -I guile > guile 2.0.11 out /gnu/store/3lhr8q28q6f59774di9av7ncy09jd55d-guile-2.0.11 > guile 2.0.11 debug /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug > mhw <at> jojen:~$ ls -l .guix-profile/lib/debug/gnu/store/ > total 8 > lrwxrwxrwx 12 root guixbuild 128 Dec 31 1969 122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11 -> /gnu/store/rgv3fvy6xqp6966rfh8v6fv7m48abcbh-guile-2.0.11-debug/lib/debug/gnu/store/122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11 > lrwxrwxrwx 15 root guixbuild 127 Dec 31 1969 3g20rdmnavpblsmgppyl8jhg67nidhjk-glibc-2.20 -> /gnu/store/hrny2whqg9c3m0klyfpbmmcyiir9yf8m-gcc-toolchain-4.9.2/lib/debug/gnu/store/3g20rdmnavpblsmgppyl8jhg67nidhjk-glibc-2.20 > > I guess GDB can't find the debugging information because > 122jv790mv2mlnylbrbzav65vghbw93n-guile-2.0.11 is the name of the guile > directory *before* grafting. I wonder if the problem you described here still exists today. However, one very likely problem is that .debug files include a CRC of the binary they correspond to (info "(gdb) Separate Debug Files"), and that CRC is not updated when we graft things. We could change our grafting code to specifically address this problem (using (guix elf) & co.). Similar issue with build IDs: <https://bugs.gnu.org/25752>. Ludo’.
bug-guix <at> gnu.org:bug#19973; Package guix.
(Fri, 10 Mar 2017 13:15:02 GMT) Full text and rfc822 format available.Message #14 received at 19973 <at> debbugs.gnu.org (full text, mbox):
From: ludo <at> gnu.org (Ludovic Courtès) To: Mark H Weaver <mhw <at> netris.org> Cc: 19973 <at> debbugs.gnu.org Subject: Re: bug#19973: Grafts break debug outputs Date: Fri, 10 Mar 2017 14:14:13 +0100
ludo <at> gnu.org (Ludovic Courtès) skribis: > However, one very likely problem is that .debug files include a CRC of > the binary they correspond to (info "(gdb) Separate Debug Files"), and > that CRC is not updated when we graft things. It manifests like this in GDB: --8<---------------cut here---------------start------------->8--- warning: the debug information found in "/home/ludo/.guix-profile/lib/debug//gnu/store/y8ppqsxiki39n4mqpb4mab6bgwqsnnp7-libgc-7.4.2/lib/libgc.so.debug" does not match "/gnu/store/y8ppqsxiki39n4mqpb4mab6bgwqsnnp7-libgc-7.4.2/lib/libgc.so.1" (CRC mismatch). --8<---------------cut here---------------end--------------->8--- Ludo’.
ludo <at> gnu.org (Ludovic Courtès)
to control <at> debbugs.gnu.org.
(Thu, 26 Jul 2018 15:22:01 GMT) Full text and rfc822 format available.bug-guix <at> gnu.org:bug#19973; Package guix.
(Tue, 21 Aug 2018 20:55:02 GMT) Full text and rfc822 format available.Message #19 received at 19973 <at> debbugs.gnu.org (full text, mbox):
From: ludo <at> gnu.org (Ludovic Courtès) To: Mark H Weaver <mhw <at> netris.org> Cc: Timothy Sample <samplet <at> ngyro.com>, 19973 <at> debbugs.gnu.org Subject: Re: bug#19973: Grafts break debug outputs Date: Tue, 21 Aug 2018 22:53:57 +0200
Hello! ludo <at> gnu.org (Ludovic Courtès) skribis: > However, one very likely problem is that .debug files include a CRC of > the binary they correspond to (info "(gdb) Separate Debug Files"), and > that CRC is not updated when we graft things. > > We could change our grafting code to specifically address this problem > (using (guix elf) & co.). The patches below address this by adding code to update the CRC in the ‘.gnu_debuglink’ section upon grafting. It uses a simple hook mechanism, a simplified version of what Timothy proposed. The ‘.gnu_debuglink’ hook does nothing for packages that lack a “debug” output. For packages that have a “debug” output, only ELF files from bin/, lib/, etc. are considered. Overall the run-time cost should be low. I tested it with guile-ssh on current master (commit bab4dc58f7278e481c7eb8c6f954d6f1751deb23). When running: guix build guile-ssh you see messages like this: updating '.gnu_debuglink' CRC in '/gnu/store/523lgdnmr4x0q7cnn8wr7f2zzcnlvrpp-guile-ssh-0.11.3/lib/guile/2.2/site-ccache/ssh/tunnel.go' updating '.gnu_debuglink' CRC in '/gnu/store/523lgdnmr4x0q7cnn8wr7f2zzcnlvrpp-guile-ssh-0.11.3/lib/guile/2.2/site-ccache/ssh/version.go' updating '.gnu_debuglink' CRC in '/gnu/store/523lgdnmr4x0q7cnn8wr7f2zzcnlvrpp-guile-ssh-0.11.3/lib/libguile-ssh.so.11.0.0' The thing about .go files isn’t very useful (the ‘strip’ phase should ignore them in the first place probably). However I’ve confirmed that gdb is happy with the grafted libguile-ssh.so: if you add breakpoints with “rb guile_ssh” and then call one of the guile-ssh procedures from the REPL, you see that you get proper debugging info and all (don’t forget to “set debug-file-directory” upfront.) That’s it! It’s a relief to fix a 3-year old bug. :-) Next we can fix build IDs similarly (see <https://bugs.gnu.org/25752>), and maybe the Racket CRC issue that Timothy and Chris looked at recently, and maybe the Java manifest issue as well (is it still relevant?). Thanks, Ludo’.
bug-guix <at> gnu.org:bug#19973; Package guix.
(Tue, 21 Aug 2018 20:58:02 GMT) Full text and rfc822 format available.Message #22 received at 19973 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: 19973 <at> debbugs.gnu.org Cc: Timothy Sample <samplet <at> ngyro.com>, Ludovic Courtès <ludo <at> gnu.org> Subject: [PATCH 1/2] grafts: Add high-level 'graft' procedure on the build side. Date: Tue, 21 Aug 2018 22:56:38 +0200
* guix/build/graft.scm (graft): New procedure.
* guix/grafts.scm (graft-derivation/shallow)[build]: Use it instead of
inline code.
---
guix/build/graft.scm | 21 +++++++++++++++++++--
guix/grafts.scm | 13 ++-----------
2 files changed, 21 insertions(+), 13 deletions(-)
diff --git a/guix/build/graft.scm b/guix/build/graft.scm
index e567bff4f..8d79e8a50 100644
--- a/guix/build/graft.scm
+++ b/guix/build/graft.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2014, 2015, 2016 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2014, 2015, 2016, 2018 Ludovic Courtès <ludo <at> gnu.org>
;;; Copyright © 2016 Mark H Weaver <mhw <at> netris.org>
;;;
;;; This file is part of GNU Guix.
@@ -27,7 +27,8 @@
#:use-module (srfi srfi-1) ; list library
#:use-module (srfi srfi-26) ; cut and cute
#:export (replace-store-references
- rewrite-directory))
+ rewrite-directory
+ graft))
;;; Commentary:
;;;
@@ -321,4 +322,20 @@ file name pairs."
#:directories? #t))
(rename-matching-files output mapping))
+(define* (graft old-outputs new-outputs mapping
+ #:key (log-port (current-output-port)))
+ "Apply the grafts described by MAPPING on OLD-OUTPUTS, leading to
+NEW-OUTPUTS. MAPPING must be a list of file name pairs; OLD-OUTPUTS and
+NEW-OUTPUTS are lists of output name/file name pairs."
+ (for-each (lambda (input output)
+ (format log-port "grafting '~a' -> '~a'...~%" input output)
+ (force-output)
+ (rewrite-directory input output mapping))
+ (match old-outputs
+ (((names . files) ...)
+ files))
+ (match new-outputs
+ (((names . files) ...)
+ files))))
+
;;; graft.scm ends here
diff --git a/guix/grafts.scm b/guix/grafts.scm
index d6b0e93e8..4b10b3efd 100644
--- a/guix/grafts.scm
+++ b/guix/grafts.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2014, 2015, 2016, 2017 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2014, 2015, 2016, 2017, 2018 Ludovic Courtès <ludo <at> gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -117,16 +117,7 @@ are not recursively applied to dependencies of DRV."
(cons (assoc-ref old-outputs name)
file)))
%outputs))))
- (for-each (lambda (input output)
- (format #t "grafting '~a' -> '~a'...~%" input output)
- (force-output)
- (rewrite-directory input output mapping))
- (match old-outputs
- (((names . files) ...)
- files))
- (match %outputs
- (((names . files) ...)
- files))))))
+ (graft old-outputs %outputs mapping))))
(define add-label
(cut cons "x" <>))
--
2.18.0
bug-guix <at> gnu.org:bug#19973; Package guix.
(Tue, 21 Aug 2018 20:58:02 GMT) Full text and rfc822 format available.Message #25 received at 19973 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: 19973 <at> debbugs.gnu.org Cc: Timothy Sample <samplet <at> ngyro.com>, Ludovic Courtès <ludo <at> gnu.org> Subject: [PATCH 2/2] grafts: Add (guix build debug-link) and use it. Date: Tue, 21 Aug 2018 22:56:39 +0200
Fixes <https://bugs.gnu.org/19973>.
Reported by Mark H Weaver <mhw <at> netris.org>.
* guix/build/debug-link.scm: New file.
* guix/build/graft.scm (%graft-hooks): New variable.
(graft): Add #:hooks and honor it.
* guix/grafts.scm (graft-derivation/shallow): Add (guix build
debug-link) and (guix elf) to #:modules.
* tests/debug-link.scm: New file.
* Makefile.am (MODULES): Add guix/build/debug-link.scm.
(SCM_TESTS): Add tests/debug-link.scm.
---
Makefile.am | 2 +
guix/build/debug-link.scm | 210 ++++++++++++++++++++++++++++++++++++++
guix/build/graft.scm | 14 ++-
guix/grafts.scm | 4 +-
tests/debug-link.scm | 154 ++++++++++++++++++++++++++++
5 files changed, 381 insertions(+), 3 deletions(-)
create mode 100644 guix/build/debug-link.scm
create mode 100644 tests/debug-link.scm
diff --git a/Makefile.am b/Makefile.am
index bdf1c63ce..324674a60 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -171,6 +171,7 @@ MODULES = \
guix/build/svn.scm \
guix/build/syscalls.scm \
guix/build/gremlin.scm \
+ guix/build/debug-link.scm \
guix/build/emacs-utils.scm \
guix/build/java-utils.scm \
guix/build/lisp-utils.scm \
@@ -363,6 +364,7 @@ SCM_TESTS = \
tests/syscalls.scm \
tests/inferior.scm \
tests/gremlin.scm \
+ tests/debug-link.scm \
tests/bournish.scm \
tests/lint.scm \
tests/publish.scm \
diff --git a/guix/build/debug-link.scm b/guix/build/debug-link.scm
new file mode 100644
index 000000000..9167737fb
--- /dev/null
+++ b/guix/build/debug-link.scm
@@ -0,0 +1,210 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix build debug-link)
+ #:use-module (guix elf)
+ #:use-module ((guix build utils)
+ #:select (find-files elf-file? make-file-writable))
+ #:use-module (rnrs bytevectors)
+ #:use-module (rnrs io ports)
+ #:use-module (srfi srfi-1)
+ #:use-module (system foreign)
+ #:use-module (ice-9 match)
+ #:export (debuglink-crc32
+ elf-debuglink
+ set-debuglink-crc
+
+ graft-debug-links))
+
+;;; Commentary:
+;;;
+;;; This module provides tools to deal with '.gnu_debuglink' sections in ELF
+;;; files. These sections are created by 'objcopy --add-gnu-debuglink' to
+;;; create separate debug files (info "(gdb) Separate Debug Files").
+;;;
+;;; The main facility of this module is 'graft-debug-links', which allows us
+;;; to update the CRC that appears in '.gnu_debuglink' sections when grafting,
+;;; such that separate debug files remain usable after grafting. Failing to
+;;; do that, GDB would complain about CRC mismatch---see
+;;; <https://bugs.gnu.org/19973>.
+;;;
+;;; Code:
+
+(define %crc32-table
+ ;; CRC table taken from "(gdb) Separate Debug Files".
+ ;; TODO: Wouldn't it be nice to generate it "from source" with a macro?
+ #(#x00000000 #x77073096 #xee0e612c #x990951ba #x076dc419
+ #x706af48f #xe963a535 #x9e6495a3 #x0edb8832 #x79dcb8a4
+ #xe0d5e91e #x97d2d988 #x09b64c2b #x7eb17cbd #xe7b82d07
+ #x90bf1d91 #x1db71064 #x6ab020f2 #xf3b97148 #x84be41de
+ #x1adad47d #x6ddde4eb #xf4d4b551 #x83d385c7 #x136c9856
+ #x646ba8c0 #xfd62f97a #x8a65c9ec #x14015c4f #x63066cd9
+ #xfa0f3d63 #x8d080df5 #x3b6e20c8 #x4c69105e #xd56041e4
+ #xa2677172 #x3c03e4d1 #x4b04d447 #xd20d85fd #xa50ab56b
+ #x35b5a8fa #x42b2986c #xdbbbc9d6 #xacbcf940 #x32d86ce3
+ #x45df5c75 #xdcd60dcf #xabd13d59 #x26d930ac #x51de003a
+ #xc8d75180 #xbfd06116 #x21b4f4b5 #x56b3c423 #xcfba9599
+ #xb8bda50f #x2802b89e #x5f058808 #xc60cd9b2 #xb10be924
+ #x2f6f7c87 #x58684c11 #xc1611dab #xb6662d3d #x76dc4190
+ #x01db7106 #x98d220bc #xefd5102a #x71b18589 #x06b6b51f
+ #x9fbfe4a5 #xe8b8d433 #x7807c9a2 #x0f00f934 #x9609a88e
+ #xe10e9818 #x7f6a0dbb #x086d3d2d #x91646c97 #xe6635c01
+ #x6b6b51f4 #x1c6c6162 #x856530d8 #xf262004e #x6c0695ed
+ #x1b01a57b #x8208f4c1 #xf50fc457 #x65b0d9c6 #x12b7e950
+ #x8bbeb8ea #xfcb9887c #x62dd1ddf #x15da2d49 #x8cd37cf3
+ #xfbd44c65 #x4db26158 #x3ab551ce #xa3bc0074 #xd4bb30e2
+ #x4adfa541 #x3dd895d7 #xa4d1c46d #xd3d6f4fb #x4369e96a
+ #x346ed9fc #xad678846 #xda60b8d0 #x44042d73 #x33031de5
+ #xaa0a4c5f #xdd0d7cc9 #x5005713c #x270241aa #xbe0b1010
+ #xc90c2086 #x5768b525 #x206f85b3 #xb966d409 #xce61e49f
+ #x5edef90e #x29d9c998 #xb0d09822 #xc7d7a8b4 #x59b33d17
+ #x2eb40d81 #xb7bd5c3b #xc0ba6cad #xedb88320 #x9abfb3b6
+ #x03b6e20c #x74b1d29a #xead54739 #x9dd277af #x04db2615
+ #x73dc1683 #xe3630b12 #x94643b84 #x0d6d6a3e #x7a6a5aa8
+ #xe40ecf0b #x9309ff9d #x0a00ae27 #x7d079eb1 #xf00f9344
+ #x8708a3d2 #x1e01f268 #x6906c2fe #xf762575d #x806567cb
+ #x196c3671 #x6e6b06e7 #xfed41b76 #x89d32be0 #x10da7a5a
+ #x67dd4acc #xf9b9df6f #x8ebeeff9 #x17b7be43 #x60b08ed5
+ #xd6d6a3e8 #xa1d1937e #x38d8c2c4 #x4fdff252 #xd1bb67f1
+ #xa6bc5767 #x3fb506dd #x48b2364b #xd80d2bda #xaf0a1b4c
+ #x36034af6 #x41047a60 #xdf60efc3 #xa867df55 #x316e8eef
+ #x4669be79 #xcb61b38c #xbc66831a #x256fd2a0 #x5268e236
+ #xcc0c7795 #xbb0b4703 #x220216b9 #x5505262f #xc5ba3bbe
+ #xb2bd0b28 #x2bb45a92 #x5cb36a04 #xc2d7ffa7 #xb5d0cf31
+ #x2cd99e8b #x5bdeae1d #x9b64c2b0 #xec63f226 #x756aa39c
+ #x026d930a #x9c0906a9 #xeb0e363f #x72076785 #x05005713
+ #x95bf4a82 #xe2b87a14 #x7bb12bae #x0cb61b38 #x92d28e9b
+ #xe5d5be0d #x7cdcefb7 #x0bdbdf21 #x86d3d2d4 #xf1d4e242
+ #x68ddb3f8 #x1fda836e #x81be16cd #xf6b9265b #x6fb077e1
+ #x18b74777 #x88085ae6 #xff0f6a70 #x66063bca #x11010b5c
+ #x8f659eff #xf862ae69 #x616bffd3 #x166ccf45 #xa00ae278
+ #xd70dd2ee #x4e048354 #x3903b3c2 #xa7672661 #xd06016f7
+ #x4969474d #x3e6e77db #xaed16a4a #xd9d65adc #x40df0b66
+ #x37d83bf0 #xa9bcae53 #xdebb9ec5 #x47b2cf7f #x30b5ffe9
+ #xbdbdf21c #xcabac28a #x53b39330 #x24b4a3a6 #xbad03605
+ #xcdd70693 #x54de5729 #x23d967bf #xb3667a2e #xc4614ab8
+ #x5d681b02 #x2a6f2b94 #xb40bbe37 #xc30c8ea1 #x5a05df1b
+ #x2d02ef8d))
+
+(define (debuglink-crc32 port)
+ "Compute the 32-bit CRC used in in '.gnu_debuglink' over the data read from
+PORT and return it." ;(info "(gdb) Separate Debug Files")
+ (let loop ((crc #xffffffff))
+ (let ((byte (get-u8 port)))
+ (if (eof-object? byte)
+ (logand (lognot crc) #xffffffff)
+ (let* ((index (logand (logxor crc byte) #xff))
+ (lhs (vector-ref %crc32-table index)))
+ (loop (logxor lhs (ash crc -8))))))))
+
+(define (section-contents elf section) ;XXX: copied from linux-modules.scm
+ "Return the contents of SECTION in ELF as a bytevector."
+ (let* ((contents (make-bytevector (elf-section-size section))))
+ (bytevector-copy! (elf-bytes elf) (elf-section-offset section)
+ contents 0
+ (elf-section-size section))
+ contents))
+
+(define null-terminated-bytevector->string
+ (compose pointer->string bytevector->pointer))
+
+(define (elf-debuglink elf)
+ "Return two values: the '.gnu_debuglink' file name of ELF and its CRC.
+Return #f for both if ELF lacks a '.gnu_debuglink' section."
+ (let ((section (elf-section-by-name elf ".gnu_debuglink")))
+ (if section
+ (let ((size (elf-section-size section))
+ (bv (section-contents elf section))
+ (endianness (elf-byte-order elf)))
+ (values (null-terminated-bytevector->string bv)
+ (bytevector-u32-ref bv (- size 4) endianness)))
+ (values #f #f))))
+
+(define (elf-debuglink-crc-offset elf)
+ "Return the offset of the '.gnu_debuglink' 32-bit CRC, or #f if ELF lacks a
+'.gnu_debuglink' section."
+ (let ((section (elf-section-by-name elf ".gnu_debuglink")))
+ (and section
+ (+ (elf-section-offset section)
+ (elf-section-size section)
+ -4))))
+
+(define (set-debuglink-crc file debug-file)
+ "Compute the CRC of DEBUG-FILE and set it as the '.gnu_debuglink' CRC in
+FILE."
+ (let* ((elf (parse-elf (call-with-input-file file get-bytevector-all)))
+ (offset (elf-debuglink-crc-offset elf)))
+ (and offset
+ (let* ((crc (call-with-input-file debug-file debuglink-crc32))
+ (bv (make-bytevector 4)))
+ (bytevector-u32-set! bv 0 crc (elf-byte-order elf))
+ (let ((port (open file O_RDWR)))
+ (set-port-position! port offset)
+ (put-bytevector port bv)
+ (close-port port))))))
+
+
+;;;
+;;; Updating debuglink CRC.
+;;;
+
+(define (find-elf-files outputs)
+ "Return the list of ELF files found in OUTPUTS, a list of top-level store
+directories."
+ (define directories
+ (append-map (lambda (output)
+ (list (string-append output "/bin")
+ (string-append output "/sbin")
+ (string-append output "/lib")
+ (string-append output "/libexec")))
+ outputs))
+
+ (append-map (lambda (directory)
+ (filter elf-file?
+ (with-error-to-port (%make-void-port "w")
+ (lambda ()
+ (find-files directory)))))
+ directories))
+
+(define* (graft-debug-links old-outputs new-outputs mapping
+ #:key (log-port (current-error-port)))
+ "Update the '.gnu_debuglink' CRCs found in ELF files of NEW-OUTPUTS,
+provided NEW-OUTPUTS contains a \"debug\" output, such that those CRCs match
+those of the corresponding '.debug' files found in the \"debug\" output.
+
+This procedure is meant to be used as a \"grafting hook\" by (guix build
+graft)."
+ (match (assoc-ref new-outputs "debug")
+ (#f #t) ;nothing to do
+ (debug-directory
+ (let ((files (find-elf-files (filter-map (match-lambda
+ (("debug" . _)
+ #f)
+ ((name . directory)
+ directory))
+ new-outputs))))
+ (for-each (lambda (file)
+ (let ((debug (string-append debug-directory
+ "/lib/debug" file ".debug")))
+ (when (file-exists? debug)
+ (format log-port
+ "updating '.gnu_debuglink' CRC in '~a'~%"
+ file)
+ (make-file-writable file)
+ (set-debuglink-crc file debug))))
+ files)))))
diff --git a/guix/build/graft.scm b/guix/build/graft.scm
index 8d79e8a50..c119ee71d 100644
--- a/guix/build/graft.scm
+++ b/guix/build/graft.scm
@@ -19,6 +19,7 @@
(define-module (guix build graft)
#:use-module (guix build utils)
+ #:use-module (guix build debug-link)
#:use-module (rnrs bytevectors)
#:use-module (ice-9 vlist)
#:use-module (ice-9 match)
@@ -322,8 +323,13 @@ file name pairs."
#:directories? #t))
(rename-matching-files output mapping))
+(define %graft-hooks
+ ;; Default list of hooks run after grafting.
+ (list graft-debug-links))
+
(define* (graft old-outputs new-outputs mapping
- #:key (log-port (current-output-port)))
+ #:key (log-port (current-output-port))
+ (hooks %graft-hooks))
"Apply the grafts described by MAPPING on OLD-OUTPUTS, leading to
NEW-OUTPUTS. MAPPING must be a list of file name pairs; OLD-OUTPUTS and
NEW-OUTPUTS are lists of output name/file name pairs."
@@ -336,6 +342,10 @@ NEW-OUTPUTS are lists of output name/file name pairs."
files))
(match new-outputs
(((names . files) ...)
- files))))
+ files)))
+ (for-each (lambda (hook)
+ (hook old-outputs new-outputs mapping
+ #:log-port log-port))
+ hooks))
;;; graft.scm ends here
diff --git a/guix/grafts.scm b/guix/grafts.scm
index 4b10b3efd..f303e925f 100644
--- a/guix/grafts.scm
+++ b/guix/grafts.scm
@@ -130,7 +130,9 @@ are not recursively applied to dependencies of DRV."
#:system system
#:guile-for-build guile
#:modules '((guix build graft)
- (guix build utils))
+ (guix build utils)
+ (guix build debug-link)
+ (guix elf))
#:inputs `(,@(map (lambda (out)
`("x" ,drv ,out))
outputs)
diff --git a/tests/debug-link.scm b/tests/debug-link.scm
new file mode 100644
index 000000000..a57f360b2
--- /dev/null
+++ b/tests/debug-link.scm
@@ -0,0 +1,154 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (test-debug-link)
+ #:use-module (guix elf)
+ #:use-module (guix build utils)
+ #:use-module (guix build debug-link)
+ #:use-module (guix gexp)
+ #:use-module (guix store)
+ #:use-module (guix tests)
+ #:use-module (guix monads)
+ #:use-module (guix derivations)
+ #:use-module (gnu packages bootstrap)
+ #:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-64)
+ #:use-module (rnrs io ports)
+ #:use-module (ice-9 match))
+
+(define %guile-executable
+ (match (false-if-exception (readlink "/proc/self/exe"))
+ ((? string? program)
+ (and (file-exists? program) (elf-file? program)
+ program))
+ (_
+ #f)))
+
+(define read-elf
+ (compose parse-elf get-bytevector-all))
+
+(define %store
+ (open-connection-for-tests))
+
+(define-syntax-rule (test-assertm name exp)
+ (test-assert name
+ (run-with-store %store exp
+ #:guile-for-build (%guile-for-build))))
+
+
+(test-begin "debug-link")
+
+(unless %guile-executable (test-skip 1))
+(test-assert "elf-debuglink"
+ (let ((elf (call-with-input-file %guile-executable read-elf)))
+ (match (call-with-values (lambda () (elf-debuglink elf)) list)
+ ((#f #f) ;no '.gnu_debuglink' section
+ (pk 'no-debuglink #t))
+ (((? string? file) (? integer? crc))
+ (string-suffix? ".debug" file)))))
+
+(test-assertm "elf-debuglink"
+ ;; Check whether we can compute the CRC just like objcopy, and whether we
+ ;; can retrieve it.
+ (let* ((code (plain-file "test.c" "int main () { return 42; }"))
+ (exp (with-imported-modules '((guix build utils)
+ (guix build debug-link)
+ (guix elf))
+ #~(begin
+ (use-modules (guix build utils)
+ (guix build debug-link)
+ (guix elf)
+ (rnrs io ports))
+
+ (define read-elf
+ (compose parse-elf get-bytevector-all))
+
+ (setenv "PATH" (string-join '(#$%bootstrap-gcc
+ #$%bootstrap-binutils)
+ "/bin:" 'suffix))
+ (invoke "gcc" "-O0" "-g" #$code "-o" "exe")
+ (copy-file "exe" "exe.debug")
+ (invoke "strip" "--only-keep-debug" "exe.debug")
+ (invoke "strip" "--strip-debug" "exe")
+ (invoke "objcopy" "--add-gnu-debuglink=exe.debug"
+ "exe")
+ (call-with-values (lambda ()
+ (elf-debuglink
+ (call-with-input-file "exe"
+ read-elf)))
+ (lambda (file crc)
+ (call-with-output-file #$output
+ (lambda (port)
+ (let ((expected (call-with-input-file "exe.debug"
+ debuglink-crc32)))
+ (write (list file (= crc expected))
+ port))))))))))
+ (mlet* %store-monad ((drv (gexp->derivation "debuglink" exp))
+ (x (built-derivations (list drv))))
+ (call-with-input-file (derivation->output-path drv)
+ (lambda (port)
+ (return (match (read port)
+ (("exe.debug" #t) #t)
+ (x (pk 'fail x #f)))))))))
+
+(test-assertm "set-debuglink-crc"
+ ;; Check whether 'set-debuglink-crc' successfully updates the CRC.
+ (let* ((code (plain-file "test.c" "int main () { return 42; }"))
+ (debug (plain-file "exe.debug" "a"))
+ (exp (with-imported-modules '((guix build utils)
+ (guix build debug-link)
+ (guix elf))
+ #~(begin
+ (use-modules (guix build utils)
+ (guix build debug-link)
+ (guix elf)
+ (rnrs io ports))
+
+ (define read-elf
+ (compose parse-elf get-bytevector-all))
+
+ (setenv "PATH" (string-join '(#$%bootstrap-gcc
+ #$%bootstrap-binutils)
+ "/bin:" 'suffix))
+ (invoke "gcc" "-O0" "-g" #$code "-o" "exe")
+ (copy-file "exe" "exe.debug")
+ (invoke "strip" "--only-keep-debug" "exe.debug")
+ (invoke "strip" "--strip-debug" "exe")
+ (invoke "objcopy" "--add-gnu-debuglink=exe.debug"
+ "exe")
+ (set-debuglink-crc "exe" #$debug)
+ (call-with-values (lambda ()
+ (elf-debuglink
+ (call-with-input-file "exe"
+ read-elf)))
+ (lambda (file crc)
+ (call-with-output-file #$output
+ (lambda (port)
+ (write (list file crc) port)))))))))
+ (mlet* %store-monad ((drv (gexp->derivation "debuglink" exp))
+ (x (built-derivations (list drv))))
+ (call-with-input-file (derivation->output-path drv)
+ (lambda (port)
+ (return (match (read port)
+ (("exe.debug" crc)
+ (= crc (debuglink-crc32 (open-input-string "a"))))
+ (x
+ (pk 'fail x #f)))))))))
+
+(test-end "debug-link")
--
2.18.0
ludo <at> gnu.org (Ludovic Courtès)
to control <at> debbugs.gnu.org.
(Tue, 21 Aug 2018 21:30:02 GMT) Full text and rfc822 format available.bug-guix <at> gnu.org:bug#19973; Package guix.
(Wed, 22 Aug 2018 14:05:02 GMT) Full text and rfc822 format available.Message #30 received at 19973 <at> debbugs.gnu.org (full text, mbox):
From: Timothy Sample <samplet <at> ngyro.com> To: ludo <at> gnu.org (Ludovic Courtès) Cc: Mark H Weaver <mhw <at> netris.org>, 19973 <at> debbugs.gnu.org Subject: Re: bug#19973: Grafts break debug outputs Date: Wed, 22 Aug 2018 10:04:07 -0400
Hi Ludo, ludo <at> gnu.org (Ludovic Courtès) writes: > Hello! > > ludo <at> gnu.org (Ludovic Courtès) skribis: > >> However, one very likely problem is that .debug files include a CRC of >> the binary they correspond to (info "(gdb) Separate Debug Files"), and >> that CRC is not updated when we graft things. >> >> We could change our grafting code to specifically address this problem >> (using (guix elf) & co.). > > The patches below address this by adding code to update the CRC in the > ‘.gnu_debuglink’ section upon grafting. It uses a simple hook > mechanism, a simplified version of what Timothy proposed. > > The ‘.gnu_debuglink’ hook does nothing for packages that lack a “debug” > output. For packages that have a “debug” output, only ELF files from > bin/, lib/, etc. are considered. Overall the run-time cost should be > low. > > I tested it with guile-ssh on current master (commit > bab4dc58f7278e481c7eb8c6f954d6f1751deb23). When running: > > guix build guile-ssh > > you see messages like this: > > updating '.gnu_debuglink' CRC in > '/gnu/store/523lgdnmr4x0q7cnn8wr7f2zzcnlvrpp-guile-ssh-0.11.3/lib/guile/2.2/site-ccache/ssh/tunnel.go' > updating '.gnu_debuglink' CRC in > '/gnu/store/523lgdnmr4x0q7cnn8wr7f2zzcnlvrpp-guile-ssh-0.11.3/lib/guile/2.2/site-ccache/ssh/version.go' > updating '.gnu_debuglink' CRC in > '/gnu/store/523lgdnmr4x0q7cnn8wr7f2zzcnlvrpp-guile-ssh-0.11.3/lib/libguile-ssh.so.11.0.0' > > The thing about .go files isn’t very useful (the ‘strip’ phase should > ignore them in the first place probably). However I’ve confirmed that > gdb is happy with the grafted libguile-ssh.so: if you add breakpoints > with “rb guile_ssh” and then call one of the guile-ssh procedures from > the REPL, you see that you get proper debugging info and all (don’t > forget to “set debug-file-directory” upfront.) > > That’s it! It’s a relief to fix a 3-year old bug. :-) > > Next we can fix build IDs similarly (see <https://bugs.gnu.org/25752>), > and maybe the Racket CRC issue that Timothy and Chris looked at > recently, and maybe the Java manifest issue as well (is it still > relevant?). The only concern I have is the level at which the hooks operate. In my draft patch¹ I had the hooks running both on the client side and the build side. This made it possible to get a bit more information about the derivation being grafted. If everything happens at the build level based on outputs, we will only be able to look at the structures and names of the outputs. That being said, this is probably okay. The Racket hook will just have to check for “share/racket” to determine if it runs (and fail safely if anything is amiss). > Thanks, > Ludo’. -- Tim ¹ https://lists.gnu.org/archive/html/guix-devel/2018-08/msg00113.html
bug-guix <at> gnu.org:bug#19973; Package guix.
(Thu, 23 Aug 2018 15:48:01 GMT) Full text and rfc822 format available.Message #33 received at 19973 <at> debbugs.gnu.org (full text, mbox):
From: ludo <at> gnu.org (Ludovic Courtès) To: Timothy Sample <samplet <at> ngyro.com> Cc: Mark H Weaver <mhw <at> netris.org>, 19973 <at> debbugs.gnu.org Subject: Re: bug#19973: Grafts break debug outputs Date: Thu, 23 Aug 2018 17:47:43 +0200
Hello Timothy, Timothy Sample <samplet <at> ngyro.com> skribis: > ludo <at> gnu.org (Ludovic Courtès) writes: [...] >> Next we can fix build IDs similarly (see <https://bugs.gnu.org/25752>), >> and maybe the Racket CRC issue that Timothy and Chris looked at >> recently, and maybe the Java manifest issue as well (is it still >> relevant?). > > The only concern I have is the level at which the hooks operate. In my > draft patch¹ I had the hooks running both on the client side and the > build side. This made it possible to get a bit more information about > the derivation being grafted. If everything happens at the build level > based on outputs, we will only be able to look at the structures and > names of the outputs. Yes, I agree that your proposal had the appeal of being possibly more extensible that what I posted here. However, as I wrote there, there are hooks that we’ll always want to run, independently of the input packages, such as the .gnu_debuglink and build-ID hooks; also, it costs nothing to have them unconditionally, we only pay for their functionality when candidate files exist. > That being said, this is probably okay. The Racket hook will just have > to check for “share/racket” to determine if it runs (and fail safely if > anything is amiss). Yes, that should work well enough. Thanks for your feedback! Ludo’.
ludo <at> gnu.org (Ludovic Courtès):Mark H Weaver <mhw <at> netris.org>:Message #38 received at 19973-done <at> debbugs.gnu.org (full text, mbox):
From: ludo <at> gnu.org (Ludovic Courtès) To: Mark H Weaver <mhw <at> netris.org> Cc: Timothy Sample <samplet <at> ngyro.com>, 19973-done <at> debbugs.gnu.org Subject: Re: bug#19973: Grafts break debug outputs Date: Fri, 24 Aug 2018 18:08:38 +0200
Hello, ludo <at> gnu.org (Ludovic Courtès) skribis: > The patches below address this by adding code to update the CRC in the > ‘.gnu_debuglink’ section upon grafting. It uses a simple hook > mechanism, a simplified version of what Timothy proposed. > > The ‘.gnu_debuglink’ hook does nothing for packages that lack a “debug” > output. For packages that have a “debug” output, only ELF files from > bin/, lib/, etc. are considered. Overall the run-time cost should be > low. I went ahead and pushed these two patches as 93c333895a4e2dc9baabec8ade60d9d2ac0a91e2 and e4297aa8b95cefa32e2595ce58886fc03b0561f7. I modified tests/debug-link.scm compared to what I posted such that it would skip tests that require %bootstrap-gcc when networking is lacking. Thanks for your feedback, Ludo’.
Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org.
(Sat, 22 Sep 2018 11:24:05 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.