GNU bug report logs - #78434
[PATCH 0/3] Add sqlelf

Previous Next

To reply to this bug, email your comments to 78434 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Karl Hallsby <karl <at> hallsby.com>
To: guix-patches <at> gnu.org
Cc: Raven Hallsby <karl <at> hallsby.com>
Subject: [PATCH 0/3] Add sqlelf
Date: Wed, 14 May 2025 22:39:55 -0500

From: Raven Hallsby <karl <at> hallsby.com>

Sqlelf[1,2] is a tool built on top of LIEF to inspect ELF the symbols and
strings of binaries and libraries (and their transitive dependencies) using
SQL. The results of parsing the ELF files is placed in an in-memory SQLite
database that can be queried entirely within memory and can be dumped to disk
for repeated lookups.

I initially placed this program in gnu/packages/elf.scm, but that module appears
to only have low-dependency core programs. Moving the sqlelf package can be
moved to another module easily.

[1] https://arxiv.org/abs/2405.03883
[2] https://github.com/fzakaria/sqlelf

Karl Hallsby (3):
  gnu: python-lief: Update to 0.14.1
  gnu: python-sh: Update to 2.0.6
  gnu: Init sqlelf

 gnu/packages/elf.scm        | 76 +++++++++++++++++++++++++++++++++++-
 gnu/packages/python-xyz.scm | 77 ++++++++++++++++++++++++++-----------
 2 files changed, 129 insertions(+), 24 deletions(-)


base-commit: 7b73f02c38d568147f1b6a7ff4467f73a212cd1e
--
2.49.0

Message #8 received at 78434 <at> debbugs.gnu.org (full text, mbox):

From: Karl Hallsby <karl <at> hallsby.com>
To: 78434 <at> debbugs.gnu.org
Cc: Raven Hallsby <karl <at> hallsby.com>
Subject: [PATCH 1/3] gnu: python-lief: Update to 0.14.1
Date: Wed, 14 May 2025 22:43:22 -0500

From: Raven Hallsby <karl <at> hallsby.com>

* gnu/packages/python-xyz.scm (python-lief): Update to 0.14.1.

Change-Id: I047d5250497148c3a78a61305a9bdc1c8a0b9b29
---
 gnu/packages/python-xyz.scm | 59 +++++++++++++++++++++++++++++--------
 1 file changed, 47 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 5d0ac818398..66786466ba0 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -39496,7 +39496,7 @@ (define-public python-misskey
 (define-public python-lief
   (package
     (name "python-lief")
-    (version "0.12.3")
+    (version "0.14.1")
     (source (origin
               (method git-fetch)
               (uri (git-reference
@@ -39505,19 +39505,54 @@ (define-public python-lief
               (file-name (git-file-name name version))
               (sha256
                (base32
-                "11i6hqmcjh56y554kqhl61698n9v66j2qk1c1g63mv2w07h2z661"))))
-    (build-system python-build-system)
-    (native-inputs (list cmake))
+                "0v8kqylm1f3cv750jjjla7qlpsw6jz1fc9qax0jqbpfjvym8xf3f"))))
+    (build-system cmake-build-system)
     (arguments
      (list
-      #:tests? #f                  ;needs network
-      #:phases #~(modify-phases %standard-phases
-                   (replace 'build
-                     (lambda _
-                       (invoke
-                        "python" "setup.py" "--sdk" "build"
-                        (string-append
-                         "-j" (number->string (parallel-job-count)))))))))
+      #:imported-modules `(,@%cmake-build-system-modules
+                           ,@%pyproject-build-system-modules)
+      #:modules '((guix build cmake-build-system)
+                  ((guix build pyproject-build-system) #:prefix py:)
+                  (guix build utils))
+      ;; All tests are written in Python and use Python's testing infra rather
+      ;; than a combination of CMake and Python. Must be done AFTER lief is
+      ;; installed to PYTHONPATH.
+      ;; Lief requires test inputs to be downloaded as a separate tarball from
+      ;; S3, which cannot have fixed-output derivations that track Lief's source.
+      #:tests? #f
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-before 'configure 'set-env-vars
+            (lambda _
+              (setenv "LIEF_VERSION_ENV" #$(package-version this-package))))
+          (add-after 'build 'build-python-module
+            ;; (assoc-ref py:%standard-phases 'build))
+            (lambda _
+              (with-directory-excursion "../source/api/python"
+                ((assoc-ref py:%standard-phases 'build)))))
+          ;; Remove cmake-build-system's check phase so we can replace it with
+          ;; one that works with Python later.
+          (delete 'check)
+          (add-after 'install 'install-python-module
+            (lambda* (#:key inputs outputs (configure-flags '()) use-setuptools?
+                      #:allow-other-keys)
+              (with-directory-excursion "../source/api/python"
+                (format #t "Python is at: ~a~%" (assoc-ref inputs "python"))
+                ((assoc-ref py:%standard-phases 'install)
+                 #:inputs inputs
+                 #:outputs outputs
+                 #:configure-flags configure-flags
+                 #:use-setuptools? use-setuptools?))))
+          (add-after 'install-python-module 'add-install-to-pythonpath
+            (assoc-ref py:%standard-phases 'add-install-to-pythonpath)))))
+    (native-inputs
+     (list ninja python-pytest))
+    (inputs
+     (list python-scikit-build-core
+           python-pathspec
+           python-pip
+           python-pydantic-2 python-pydantic-core
+           python-wrapper))
     (home-page "https://github.com/lief-project/LIEF")
     (synopsis "Library to instrument executable formats")
     (description
-- 
2.49.0

Message #11 received at 78434 <at> debbugs.gnu.org (full text, mbox):

From: Karl Hallsby <karl <at> hallsby.com>
To: 78434 <at> debbugs.gnu.org
Cc: Raven Hallsby <karl <at> hallsby.com>
Subject: [PATCH 2/3] gnu: python-sh: Update to 2.0.6
Date: Wed, 14 May 2025 22:43:24 -0500

From: Raven Hallsby <karl <at> hallsby.com>

gnu/packages/python-xyz.scm (python-sh): Update to 2.0.6.

Change-Id: I0407fe9bbfef5b98ff6f7198e7fdd41e09fa94e1
---
 gnu/packages/python-xyz.scm | 18 +++++++-----------
 1 file changed, 7 insertions(+), 11 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 66786466ba0..19b77c39d15 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -4099,25 +4099,21 @@ (define-public python-pylsp-mypy
 (define-public python-sh
   (package
     (name "python-sh")
-    (version "1.14.2")
+    (version "2.0.6")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "sh" version))
        (sha256
         (base32
-         "03gyss1rhj4in7pgysg4q0hxp3230whinlpy1532ljs99lrx0ywx"))))
-    (build-system python-build-system)
+         "18bvpw02lzhlpsaqaf0kxybp0jb72c5iy1n0w9vwf0gj2grrhacv"))))
+    (build-system pyproject-build-system)
     (arguments
-     '(#:phases
-       (modify-phases %standard-phases
-         (replace 'check
-           (lambda _
-             ;; XXX: A Python 2 test fails when HOME=/homeless-shelter.
-             (setenv "HOME" "/tmp")
-             (invoke "python" "sh.py" "test"))))))
+     ;; Tests rely on python-tox, which introduces a cyclic dependency problem
+     ;; here.
+     '(#:tests? #f))
     (native-inputs
-     (list python-coverage))
+     (list python-coverage python-poetry-core))
     (home-page "https://github.com/amoffat/sh")
     (synopsis "Python subprocess replacement")
     (description "This package provides a replacement for Python's
-- 
2.49.0

Message #14 received at 78434 <at> debbugs.gnu.org (full text, mbox):

From: Karl Hallsby <karl <at> hallsby.com>
To: 78434 <at> debbugs.gnu.org
Cc: Raven Hallsby <karl <at> hallsby.com>
Subject: [PATCH 3/3] gnu: Init sqlelf
Date: Wed, 14 May 2025 22:43:26 -0500

From: Raven Hallsby <karl <at> hallsby.com>

Change-Id: I45bfb48e4f4c11dc97ec642a5248420b9ec8d75d
---
 gnu/packages/elf.scm | 76 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 75 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/elf.scm b/gnu/packages/elf.scm
index 7b769770f83..2745f8cb124 100644
--- a/gnu/packages/elf.scm
+++ b/gnu/packages/elf.scm
@@ -13,6 +13,7 @@
 ;;; Copyright © 2022 Daniel Maksymow <daniel.maksymow <at> tuta.io>
 ;;; Copyright © 2023, 2024 Janneke Nieuwenhuizen <janneke <at> gnu.org>
 ;;; Copyright © 2024 Zheng Junjie <873216071 <at> qq.com>
+;;; Copyright © 2025, Raven Hallsby <karl <at> hallsby.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -36,16 +37,25 @@ (define-module (gnu packages elf)
   #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
-  #:use-module ((guix licenses) #:select (gpl2+ gpl3+ lgpl3+ lgpl2.0+ lgpl2.1 gpl2 bsd-2))
+  #:use-module (guix build-system pyproject)
+  #:use-module ((guix licenses) #:select (gpl2+ gpl3+ lgpl3+ lgpl2.0+ lgpl2.1 gpl2 bsd-2 expat))
   #:use-module (gnu packages)
   #:use-module (gnu packages autotools)
+  #:use-module (gnu packages bash)
   #:use-module (gnu packages compression)
+  #:use-module (gnu packages check)
+  #:use-module (gnu packages databases)
   #:use-module (gnu packages documentation)
+  #:use-module (gnu packages engineering)
   #:use-module (gnu packages gcc)
   #:use-module (gnu packages m4)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
+  #:use-module (gnu packages python-build)
+  #:use-module (gnu packages python-check)
+  #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages sphinx)
+  #:use-module (gnu packages sqlite)
   #:use-module (gnu packages texinfo)
   #:use-module (gnu packages xml)
   #:use-module (srfi srfi-1)
@@ -424,3 +434,67 @@ (define-public libdwarf
 debugging information format.")
     ;; See https://www.prevanders.net/dwarflicense.html:
     (license (list lgpl2.1 gpl2 bsd-2))))
+
+(define-public sqlelf
+  ;; Some fixes since last tagged release
+  (let ((commit "a87e97c17550a0415a961fde0164352f171e7f52")
+        (revision "0"))
+    (package
+      (name "sqlelf")
+      (version (git-version "0.5" revision commit))
+      (source
+       (origin
+         (method git-fetch)
+         (uri (git-reference
+               (url "https://github.com/fzakaria/sqlelf")
+               (commit commit)))
+         (file-name (git-file-name name version))
+         (sha256
+          (base32
+           "0f5d1dgnphqigy03lw6wjrgmij77y146whglxbixqm67wm7w8a43"))
+         (modules '((guix build utils)))
+         (snippet
+          #~(begin
+              ;; Remove vendored version of elftools
+              (delete-file-recursively "sqlelf/_vendor")
+              (substitute* "sqlelf/elf.py"
+                (("sqlelf\\._vendor\\.elftools") "elftools"))
+              ;; Fix slighly incorrect pyproject.toml so that setuptools
+              ;; installs a source submodule into the wheel.
+              (substitute* "pyproject.toml"
+                (("packages = \\[\\\"sqlelf\\\"\\]")
+                  "packages = [\"sqlelf\", \"sqlelf.tools\"]"))))))
+      (build-system pyproject-build-system)
+      (arguments
+       '(#:tests? #t
+         #:phases
+         (modify-phases %standard-phases
+           (add-before 'check 'set-test-bins
+             (lambda* (#:key inputs #:allow-other-keys)
+               (let ((ls (string-append (assoc-ref inputs "coreutils") "/bin/ls"))
+                     (sh (string-append (assoc-ref inputs "bash") "/bin/sh")))
+                 ;; Replace hard-coded /bin/sh with one from the Store.
+                 (format #t "Replace /bin/sh with ~a~%" sh)
+                 (substitute* "tests/test_cli.py"
+                   (("/bin/sh") sh))
+                 ;; Tell check phase where to find 'ls' in the store for tests.
+                 (format #t "Set $TEST_BINARY to ~a~%" ls)
+                 (setenv "TEST_BINARY" ls)))))))
+      (native-inputs
+       (list python-apsw
+             bash-minimal ; Needed for /bin/sh replacement in tests
+             capstone
+             python-capstone
+             python-lief ; MUST use python-lief 0.14.1
+             python-pyelftools
+             python-pytest
+             python-setuptools
+             python-sh
+             python-wheel))
+      (inputs
+       (list sqlite))
+      (home-page "https://github.com/fzakaria/sqlelf")
+      (synopsis "Explore ELF objects through the power of SQL")
+      (description
+       "A tool that utilizes SQLite's virtual table functionality to allow you to explore Linux ELF objects through SQL.")
+      (license expat))))
-- 
2.49.0

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.