GNU bug report logs - #33535
[PATCH] refresh: Account for overlapping updater coverage.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 33535 in the body.
You can then email your comments to 33535 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: ericbavier <at> centurylink.net
To: guix-patches <at> gnu.org
Cc: Eric Bavier <bavier <at> member.fsf.org>
Subject: [PATCH] refresh: Account for overlapping updater coverage.
Date: Tue, 27 Nov 2018 20:27:37 -0600

From: Eric Bavier <bavier <at> member.fsf.org>

* guix/scripts/refresh.scm (list-updaters-and-exit): Do not assume updater
predicates are disjoint.  Track covered packages directly.
---
Hello Guix,

Some of our packages are covered by more than one of our updaters:

scheme@(guile-user)> ,use(gnu packages)(guix packages)(guix upstream)
scheme@(guile-user)> ,use(srfi srfi-1)(srfi srfi-26)
scheme@(guile-user)> (define updaters (force (@@ (guix upstream) %updaters)))
scheme@(guile-user)> (define predicates (map upstream-updater-predicate updaters))
scheme@(guile-user)> (define doubles
  (fold-packages
    (lambda (pkg result)
      (if (> (count (cut <> pkg) predicates) 1)
         (cons pkg result)
       result))
    '()))
scheme@(guile-user)> (length doubles)
$1 = 469
scheme@(guile-user)> (map package-name doubles)
$2 = ("agda" "emacs-agda2-mode" "fribidi" "raincat" "ghc-tasty-quickcheck" ... "ghc-hxt")

It seams mostly packages covered by both the "hackage" and "stackage"
updaters.  And Fribidi is a GNU package but hosted on github.

Currently this leads to double-counting while computing total package
coverage and a too optimistic result (by about 5.4%).

The below patch fixes it by tracking the (un)covered packages directly.


 guix/scripts/refresh.scm | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index 58fc64db1..f7d2cffb7 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -179,24 +179,24 @@ specified with `--select'.\n"))
 
   (let* ((packages (fold-packages cons '()))
          (total    (length packages)))
-    (define covered
-      (fold (lambda (updater covered)
-              (let ((matches (count (upstream-updater-predicate updater)
-                                    packages)))
+    (define uncovered
+      (fold (lambda (updater uncovered)
+              (let ((matches (filter (upstream-updater-predicate updater)
+                                     packages)))
                 ;; TRANSLATORS: The parenthetical expression here is rendered
                 ;; like "(42% coverage)" and denotes the fraction of packages
                 ;; covered by the given updater.
                 (format #t (G_ "  - ~a: ~a (~2,1f% coverage)~%")
                         (upstream-updater-name updater)
                         (G_ (upstream-updater-description updater))
-                        (* 100. (/ matches total)))
-                (+ covered matches)))
-            0
+                        (* 100. (/ (length matches) total)))
+                (lset-difference eq? uncovered matches)))
+            packages
             (force %updaters)))
 
     (newline)
     (format #t (G_ "~2,1f% of the packages are covered by these updaters.~%")
-            (* 100. (/ covered total))))
+            (* 100. (/ (- total (length uncovered)) total))))
   (exit 0))
 
 (define (warn-no-updater package)
-- 
2.19.1

Message #8 received at 33535 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: ericbavier <at> centurylink.net
Cc: Eric Bavier <bavier <at> member.fsf.org>, 33535 <at> debbugs.gnu.org
Subject: Re: [bug#33535] [PATCH] refresh: Account for overlapping updater
 coverage.
Date: Thu, 29 Nov 2018 18:42:38 +0100

Hello!

ericbavier <at> centurylink.net skribis:

> From: Eric Bavier <bavier <at> member.fsf.org>
>
> * guix/scripts/refresh.scm (list-updaters-and-exit): Do not assume updater
> predicates are disjoint.  Track covered packages directly.
> ---
> Hello Guix,
>
> Some of our packages are covered by more than one of our updaters:
>
> scheme@(guile-user)> ,use(gnu packages)(guix packages)(guix upstream)
> scheme@(guile-user)> ,use(srfi srfi-1)(srfi srfi-26)
> scheme@(guile-user)> (define updaters (force (@@ (guix upstream) %updaters)))
> scheme@(guile-user)> (define predicates (map upstream-updater-predicate updaters))
> scheme@(guile-user)> (define doubles
>   (fold-packages
>     (lambda (pkg result)
>       (if (> (count (cut <> pkg) predicates) 1)
>          (cons pkg result)
>        result))
>     '()))
> scheme@(guile-user)> (length doubles)
> $1 = 469
> scheme@(guile-user)> (map package-name doubles)
> $2 = ("agda" "emacs-agda2-mode" "fribidi" "raincat" "ghc-tasty-quickcheck" ... "ghc-hxt")
>
> It seams mostly packages covered by both the "hackage" and "stackage"
> updaters.  And Fribidi is a GNU package but hosted on github.
>
> Currently this leads to double-counting while computing total package
> coverage and a too optimistic result (by about 5.4%).
>
> The below patch fixes it by tracking the (un)covered packages directly.

Oh, good catch.  LGTM!

That also means we’ll have to work on our updaters…  :-)

Thanks,
Ludo’.

Message #13 received at 33535-done <at> debbugs.gnu.org (full text, mbox):

From: Eric Bavier <ericbavier <at> centurylink.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 33535-done <at> debbugs.gnu.org
Subject: Re: [bug#33535] [PATCH] refresh: Account for overlapping updater
 coverage.
Date: Mon, 10 Dec 2018 22:59:41 -0600

[Message part 1 (text/plain, inline)]

On Thu, 29 Nov 2018 18:42:38 +0100
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Hello!
> 
> ericbavier <at> centurylink.net skribis:
> 
> > From: Eric Bavier <bavier <at> member.fsf.org>
> >
> > It seams mostly packages covered by both the "hackage" and "stackage"
> > updaters.  And Fribidi is a GNU package but hosted on github.
> >
> > Currently this leads to double-counting while computing total package
> > coverage and a too optimistic result (by about 5.4%).
> >
> > The below patch fixes it by tracking the (un)covered packages directly.  
> 
> Oh, good catch.  LGTM!

Applied in cba7ddcf603455c6692eb50c8bbf203a6bf17ab1

> 
> That also means we’ll have to work on our updaters…  :-)
> 

Yup.  

I have a prototype for an "arch" importer that parses the Archlinux
PKGBUILD files (32% coverage even without any fancy package name
mapping).

I also have a patch to the github updater so that it can update
packages that use git-fetch.  This brings the coverage up to 15.9% from
9.6%.  I think this will be useful if we keep moving packages away from
github's generated tarballs.

`~Eric

[Message part 2 (application/pgp-signature, inline)]

Message #16 received at 33535 <at> debbugs.gnu.org (full text, mbox):

From: swedebugia <at> riseup.net
To: 33535 <at> debbugs.gnu.org, ericbavier <at> centurylink.net
Cc: Guix-patches <guix-patches-bounces+swedebugia=riseup.net <at> gnu.org>,
 ludo <at> gnu.org, 33535-done <at> debbugs.gnu.org
Subject: Re: bug#33535: [PATCH] refresh: Account for overlapping updater
 coverage.
Date: Wed, 12 Dec 2018 03:11:44 -0800

On 2018-12-11 05:59, Eric Bavier wrote:
snip

> I have a prototype for an "arch" importer that parses the Archlinux
> PKGBUILD files (32% coverage even without any fancy package name
> mapping).

That sounds nice Eric. Can I see code somewhere? :)

-- 
Cheers 
Swedebugia

Message #22 received at 33535 <at> debbugs.gnu.org (full text, mbox):

From: Eric Bavier <ericbavier <at> centurylink.net>
To: swedebugia <at> riseup.net
Cc: 33535 <at> debbugs.gnu.org
Subject: Re: bug#33535: [PATCH] refresh: Account for overlapping updater
 coverage.
Date: Wed, 12 Dec 2018 08:55:04 -0600

[Message part 1 (text/plain, inline)]

On Wed, 12 Dec 2018 03:11:44 -0800
swedebugia <at> riseup.net wrote:

> On 2018-12-11 05:59, Eric Bavier wrote:
> snip
> 
> > I have a prototype for an "arch" importer that parses the Archlinux
> > PKGBUILD files (32% coverage even without any fancy package name
> > mapping).  
> 
> That sounds nice Eric. Can I see code somewhere? :)
> 

It's on my "feature/arch-updater" branch:

  https://notabug.org/bavier/guix/src/feature/arch-updater

The current commit:

  https://notabug.org/bavier/guix/commit/7d711d622db43bc2a9074d494f8cb11e6a9291c8

I'm open to suggestions :)

`~Eric

[Message part 2 (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.