GNU bug report logs -
#27468
[PATCH] gnu: Add linkchecker.
Previous Next
Reported by: Arun Isaac <arunisaac <at> systemreboot.net>
Date: Sat, 24 Jun 2017 00:19:02 UTC
Severity: normal
Tags: patch
Done: Arun Isaac <arunisaac <at> systemreboot.net>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 27468 in the body.
You can then email your comments to 27468 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Sat, 24 Jun 2017 00:19:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Arun Isaac <arunisaac <at> systemreboot.net>
:
New bug report received and forwarded. Copy sent to
guix-patches <at> gnu.org
.
(Sat, 24 Jun 2017 00:19:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
* gnu/packages/web.scm (linkchecker): New variable.
---
gnu/packages/web.scm | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 3625a1034..81bee1ec9 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -4674,3 +4674,43 @@ message stream (in a web server that is per connection).")
(description "@code{httpretty} is a helper for faking web requests,
inspired by Ruby's @code{fakeweb}.")
(license l:expat)))
+
+(define-public linkchecker
+ (package
+ (name "linkchecker")
+ (version "9.3")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "LinkChecker" version))
+ (sha256
+ (base32
+ "0v8pavf0bx33xnz1kwflv0r7lxxwj7vg3syxhy2wzza0wh6sc2pf"))))
+ (build-system python-build-system)
+ (inputs
+ `(("python2-dnspython" ,python2-dnspython)
+ ("python2-requests" ,python2-requests)))
+ (arguments
+ `(#:python ,python-2
+ #:phases
+ (modify-phases %standard-phases
+ ;; Remove faulty python-requests version check. This has been fixed
+ ;; upstream, and can be removed in version 9.4.
+ (add-after 'unpack 'remove-python-requests-version
+ (lambda _
+ (substitute* "linkcheck/__init__.py"
+ (("requests.__version__ <= '2.2.0'") "False")))))))
+ (home-page "https://linkcheck.github.io/linkchecker/")
+ (synopsis "Check websites for broken links")
+ (description "LinkChecker is a website validator. It checks for broken
+links in websites. It is recursive and multithreaded providing output in
+colored or normal text, HTML, SQL, CSV, XML or as a sitemap graph. It
+supports checking HTTP/1.1, HTTPS, FTP, mailto, news, nntp, telnet and local
+file links.")
+ (license (list l:gpl2+
+ l:bsd-2 ; linkcheck/better_exchook2.py
+ l:bsd-3 ; linkcheck/colorama.py
+ l:psfl ; linkcheck/gzip2.py
+ l:expat ; linkcheck/mem.py
+ l:isc ; third_party/dnspython
+ l:asl2.0)))) ; third_party/miniboa
--
2.12.2
Information forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Sat, 24 Jun 2017 00:28:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 27468 <at> debbugs.gnu.org (full text, mbox):
This package has bundled dependencies -- dnspython and miniboa -- under
"third_party/". They have been modified as described in
third_party/dnspython/changelog.txt and
third_party/miniboa-r42/CHANGES.txt respectively. Should I unbundle
these dependencies? If so, how do I handle the fact that the bundled
dependencies have been locally modified?
Information forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Tue, 22 Aug 2017 23:32:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 27468 <at> debbugs.gnu.org (full text, mbox):
* gnu/packages/web.scm (linkchecker): New variable.
---
gnu/packages/web.scm | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 652eb6792..8116b2fd5 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -4979,3 +4979,43 @@ websites lacking feeds. Supported websites include Facebook, Twitter,
Instagram and YouTube.")
(license (list l:public-domain
l:expat)))) ;; vendor/simplehtmldom/simple_html_dom.php
+
+(define-public linkchecker
+ (package
+ (name "linkchecker")
+ (version "9.3")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "LinkChecker" version))
+ (sha256
+ (base32
+ "0v8pavf0bx33xnz1kwflv0r7lxxwj7vg3syxhy2wzza0wh6sc2pf"))))
+ (build-system python-build-system)
+ (inputs
+ `(("python2-dnspython" ,python2-dnspython)
+ ("python2-requests" ,python2-requests)))
+ (arguments
+ `(#:python ,python-2
+ #:phases
+ (modify-phases %standard-phases
+ ;; Remove faulty python-requests version check. This has been fixed
+ ;; upstream, and can be removed in version 9.4.
+ (add-after 'unpack 'remove-python-requests-version
+ (lambda _
+ (substitute* "linkcheck/__init__.py"
+ (("requests.__version__ <= '2.2.0'") "False")))))))
+ (home-page "https://linkcheck.github.io/linkchecker/")
+ (synopsis "Check websites for broken links")
+ (description "LinkChecker is a website validator. It checks for broken
+links in websites. It is recursive and multithreaded providing output in
+colored or normal text, HTML, SQL, CSV, XML or as a sitemap graph. It
+supports checking HTTP/1.1, HTTPS, FTP, mailto, news, nntp, telnet and local
+file links.")
+ (license (list l:gpl2+
+ l:bsd-2 ; linkcheck/better_exchook2.py
+ l:bsd-3 ; linkcheck/colorama.py
+ l:psfl ; linkcheck/gzip2.py
+ l:expat ; linkcheck/mem.py
+ l:isc ; third_party/dnspython
+ l:asl2.0)))) ; third_party/miniboa
--
2.13.4
Information forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Tue, 22 Aug 2017 23:35:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 27468 <at> debbugs.gnu.org (full text, mbox):
This package seems to have fallen through the cracks. I just sent a new
patch made against the latest master.
> This package has bundled dependencies -- dnspython and miniboa -- under
> "third_party/". They have been modified as described in
> third_party/dnspython/changelog.txt and
> third_party/miniboa-r42/CHANGES.txt respectively. Should I unbundle
> these dependencies? If so, how do I handle the fact that the bundled
> dependencies have been locally modified?
I have raised the issue of unbundling with upstream. But, there has been
no response after 10 days.
https://github.com/wummel/linkchecker/issues/729
I recommend we don't unbundle anything on our own, and let upstream do
their homework. WDYT?
Information forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Wed, 23 Aug 2017 00:03:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 27468 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Arun Isaac <arunisaac <at> systemreboot.net> writes:
> * gnu/packages/web.scm (linkchecker): New variable.
[...]
> + (modify-phases %standard-phases
> + ;; Remove faulty python-requests version check. This has been fixed
> + ;; upstream, and can be removed in version 9.4.
> + (add-after 'unpack 'remove-python-requests-version
> + (lambda _
> + (substitute* "linkcheck/__init__.py"
> + (("requests.__version__ <= '2.2.0'") "False")))))))
Please end this phase on a #t since (substitute* ...) has an unspecified
return value.
> + (home-page "https://linkcheck.github.io/linkchecker/")
> + (synopsis "Check websites for broken links")
> + (description "LinkChecker is a website validator. It checks for broken
> +links in websites. It is recursive and multithreaded providing output in
> +colored or normal text, HTML, SQL, CSV, XML or as a sitemap graph. It
> +supports checking HTTP/1.1, HTTPS, FTP, mailto, news, nntp, telnet and local
> +file links.")
> + (license (list l:gpl2+
> + l:bsd-2 ; linkcheck/better_exchook2.py
> + l:bsd-3 ; linkcheck/colorama.py
> + l:psfl ; linkcheck/gzip2.py
> + l:expat ; linkcheck/mem.py
> + l:isc ; third_party/dnspython
> + l:asl2.0)))) ; third_party/miniboa
Is it possible to remove these third party libraries and use packaged
copies instead? At least 'dnspython' is already in Guix.
Also, since GPL2+ is stricter than the other licenses, the program as a
whole is effectively GPL2+, so we don't really need to mention those
one-off source files in the license field (but comments are great).
Not sure about the 3rd party bundles though, since they probably don't
depend on anything from the GPL2+ distribution. If they are installed,
I guess we should mention them with a FIXME.
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Wed, 23 Aug 2017 00:11:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 27468 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Arun Isaac <arunisaac <at> systemreboot.net> writes:
> This package seems to have fallen through the cracks. I just sent a new
> patch made against the latest master.
>
>> This package has bundled dependencies -- dnspython and miniboa -- under
>> "third_party/". They have been modified as described in
>> third_party/dnspython/changelog.txt and
>> third_party/miniboa-r42/CHANGES.txt respectively. Should I unbundle
>> these dependencies? If so, how do I handle the fact that the bundled
>> dependencies have been locally modified?
>
> I have raised the issue of unbundling with upstream. But, there has been
> no response after 10 days.
>
> https://github.com/wummel/linkchecker/issues/729
>
> I recommend we don't unbundle anything on our own, and let upstream do
> their homework. WDYT?
Sounds reasonable. Can you add this URL in a comment somewhere? LGTM
apart from the cosmetic issues raised in the other message.
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Wed, 23 Aug 2017 07:27:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 27468 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Marius Bakke writes:
>> + (modify-phases %standard-phases
>> + ;; Remove faulty python-requests version check. This has been fixed
>> + ;; upstream, and can be removed in version 9.4.
>> + (add-after 'unpack 'remove-python-requests-version
>> + (lambda _
>> + (substitute* "linkcheck/__init__.py"
>> + (("requests.__version__ <= '2.2.0'") "False")))))))
>
> Please end this phase on a #t since (substitute* ...) has an unspecified
> return value.
Done.
>> + (license (list l:gpl2+
>> + l:bsd-2 ; linkcheck/better_exchook2.py
>> + l:bsd-3 ; linkcheck/colorama.py
>> + l:psfl ; linkcheck/gzip2.py
>> + l:expat ; linkcheck/mem.py
>> + l:isc ; third_party/dnspython
>> + l:asl2.0)))) ; third_party/miniboa
>
> Is it possible to remove these third party libraries and use packaged
> copies instead? At least 'dnspython' is already in Guix.
linkchecker is using a modified version of dnspython and miniboa. So, I
don't think it will be easy. For a list of modifications, look at
https://github.com/wummel/linkchecker/blob/master/third_party/dnspython/changelog.txt
and https://github.com/wummel/linkchecker/blob/master/third_party/miniboa-r42/CHANGES.txt
> Also, since GPL2+ is stricter than the other licenses, the program as a
> whole is effectively GPL2+, so we don't really need to mention those
> one-off source files in the license field (but comments are great).
But, we generally don't hide this information from the user, right? Many
Guix packages have licenses of even single files mentioned in the
`license' field. Do you still think I should remove them?
> Not sure about the 3rd party bundles though, since they probably don't
> depend on anything from the GPL2+ distribution. If they are installed,
> I guess we should mention them with a FIXME.
I've added a FIXME.
I'm attaching a new patch with the discussed changes. Do let me know if
everything looks good.
[0001-gnu-Add-linkchecker.patch (text/x-patch, inline)]
From 63f4192ff8ed7341767797f4ca67ab87773e2338 Mon Sep 17 00:00:00 2001
From: Arun Isaac <arunisaac <at> systemreboot.net>
Date: Sat, 24 Jun 2017 05:37:45 +0530
Subject: [PATCH] gnu: Add linkchecker.
* gnu/packages/web.scm (linkchecker): New variable.
---
gnu/packages/web.scm | 44 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 44 insertions(+)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 652eb6792..68a9c2302 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -4979,3 +4979,47 @@ websites lacking feeds. Supported websites include Facebook, Twitter,
Instagram and YouTube.")
(license (list l:public-domain
l:expat)))) ;; vendor/simplehtmldom/simple_html_dom.php
+
+(define-public linkchecker
+ (package
+ (name "linkchecker")
+ (version "9.3")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "LinkChecker" version))
+ (sha256
+ (base32
+ "0v8pavf0bx33xnz1kwflv0r7lxxwj7vg3syxhy2wzza0wh6sc2pf"))))
+ (build-system python-build-system)
+ (inputs
+ `(("python2-dnspython" ,python2-dnspython)
+ ("python2-requests" ,python2-requests)))
+ (arguments
+ `(#:python ,python-2
+ #:phases
+ (modify-phases %standard-phases
+ ;; Remove faulty python-requests version check. This has been fixed
+ ;; upstream, and can be removed in version 9.4.
+ (add-after 'unpack 'remove-python-requests-version
+ (lambda _
+ (substitute* "linkcheck/__init__.py"
+ (("requests.__version__ <= '2.2.0'") "False"))
+ #t)))))
+ (home-page "https://linkcheck.github.io/linkchecker")
+ (synopsis "Check websites for broken links")
+ (description "LinkChecker is a website validator. It checks for broken
+links in websites. It is recursive and multithreaded providing output in
+colored or normal text, HTML, SQL, CSV, XML or as a sitemap graph. It
+supports checking HTTP/1.1, HTTPS, FTP, mailto, news, nntp, telnet and local
+file links.")
+ (license (list l:gpl2+
+ l:bsd-2 ; linkcheck/better_exchook2.py
+ l:bsd-3 ; linkcheck/colorama.py
+ l:psfl ; linkcheck/gzip2.py
+ l:expat ; linkcheck/mem.py
+ ;; FIXME: Unbundle dnspython and miniboa
+ ;; This issue has been raised upstream
+ ;; https://github.com/wummel/linkchecker/issues/729
+ l:isc ; third_party/dnspython
+ l:asl2.0)))) ; third_party/miniboa
--
2.13.4
Information forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Wed, 23 Aug 2017 07:35:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 27468 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Actually, if dnspython is currently bundled in linkchecker, there is no
need to have it in `inputs'. Attached is a patch with this change.
[0001-gnu-Add-linkchecker.patch (text/x-patch, inline)]
From bc092d955c8939bcf965cd3a92810e423c516fc5 Mon Sep 17 00:00:00 2001
From: Arun Isaac <arunisaac <at> systemreboot.net>
Date: Sat, 24 Jun 2017 05:37:45 +0530
Subject: [PATCH] gnu: Add linkchecker.
* gnu/packages/web.scm (linkchecker): New variable.
---
gnu/packages/web.scm | 43 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 652eb6792..7f4a514db 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -4979,3 +4979,46 @@ websites lacking feeds. Supported websites include Facebook, Twitter,
Instagram and YouTube.")
(license (list l:public-domain
l:expat)))) ;; vendor/simplehtmldom/simple_html_dom.php
+
+(define-public linkchecker
+ (package
+ (name "linkchecker")
+ (version "9.3")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "LinkChecker" version))
+ (sha256
+ (base32
+ "0v8pavf0bx33xnz1kwflv0r7lxxwj7vg3syxhy2wzza0wh6sc2pf"))))
+ (build-system python-build-system)
+ (inputs
+ `(("python2-requests" ,python2-requests)))
+ (arguments
+ `(#:python ,python-2
+ #:phases
+ (modify-phases %standard-phases
+ ;; Remove faulty python-requests version check. This has been fixed
+ ;; upstream, and can be removed in version 9.4.
+ (add-after 'unpack 'remove-python-requests-version
+ (lambda _
+ (substitute* "linkcheck/__init__.py"
+ (("requests.__version__ <= '2.2.0'") "False"))
+ #t)))))
+ (home-page "https://linkcheck.github.io/linkchecker")
+ (synopsis "Check websites for broken links")
+ (description "LinkChecker is a website validator. It checks for broken
+links in websites. It is recursive and multithreaded providing output in
+colored or normal text, HTML, SQL, CSV, XML or as a sitemap graph. It
+supports checking HTTP/1.1, HTTPS, FTP, mailto, news, nntp, telnet and local
+file links.")
+ (license (list l:gpl2+
+ l:bsd-2 ; linkcheck/better_exchook2.py
+ l:bsd-3 ; linkcheck/colorama.py
+ l:psfl ; linkcheck/gzip2.py
+ l:expat ; linkcheck/mem.py
+ ;; FIXME: Unbundle dnspython and miniboa
+ ;; This issue has been raised upstream
+ ;; https://github.com/wummel/linkchecker/issues/729
+ l:isc ; third_party/dnspython
+ l:asl2.0)))) ; third_party/miniboa
--
2.13.4
Information forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Thu, 24 Aug 2017 17:44:01 GMT)
Full text and
rfc822 format available.
Message #29 received at 27468 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Arun Isaac <arunisaac <at> systemreboot.net> writes:
>>> + (license (list l:gpl2+
>>> + l:bsd-2 ; linkcheck/better_exchook2.py
>>> + l:bsd-3 ; linkcheck/colorama.py
>>> + l:psfl ; linkcheck/gzip2.py
>>> + l:expat ; linkcheck/mem.py
>>> + l:isc ; third_party/dnspython
>>> + l:asl2.0)))) ; third_party/miniboa
>>
>> Is it possible to remove these third party libraries and use packaged
>> copies instead? At least 'dnspython' is already in Guix.
>
> linkchecker is using a modified version of dnspython and miniboa. So, I
> don't think it will be easy. For a list of modifications, look at
> https://github.com/wummel/linkchecker/blob/master/third_party/dnspython/changelog.txt
> and https://github.com/wummel/linkchecker/blob/master/third_party/miniboa-r42/CHANGES.txt
OK, thanks for checking.
>> Also, since GPL2+ is stricter than the other licenses, the program as a
>> whole is effectively GPL2+, so we don't really need to mention those
>> one-off source files in the license field (but comments are great).
>
> But, we generally don't hide this information from the user, right? Many
> Guix packages have licenses of even single files mentioned in the
> `license' field. Do you still think I should remove them?
I don't know :-) Licensing is a tricky topic. The GPL is rather
explicit in that it covers the "program as a whole". If we treat
individual parts differently, it may be misleading to the end user.
OTOH, the GPL permits using those source files individually under their
own terms. Perhaps the most practical solution would be to add a
"supplementary licenses" interface...
I am not a lawyer, so please use your own judgement. No strong opinion
either way. :-)
>> Not sure about the 3rd party bundles though, since they probably don't
>> depend on anything from the GPL2+ distribution. If they are installed,
>> I guess we should mention them with a FIXME.
>
> I've added a FIXME.
>
> I'm attaching a new patch with the discussed changes. Do let me know if
> everything looks good.
LGTM!
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
guix-patches <at> gnu.org
:
bug#27468
; Package
guix-patches
.
(Sat, 26 Aug 2017 07:44:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 27468 <at> debbugs.gnu.org (full text, mbox):
>>> Also, since GPL2+ is stricter than the other licenses, the program as a
>>> whole is effectively GPL2+, so we don't really need to mention those
>>> one-off source files in the license field (but comments are great).
>>
>> But, we generally don't hide this information from the user, right? Many
>> Guix packages have licenses of even single files mentioned in the
>> `license' field. Do you still think I should remove them?
>
> I don't know :-) Licensing is a tricky topic. The GPL is rather
> explicit in that it covers the "program as a whole". If we treat
> individual parts differently, it may be misleading to the end user.
>
> OTOH, the GPL permits using those source files individually under their
> own terms. Perhaps the most practical solution would be to add a
> "supplementary licenses" interface...
>
> I am not a lawyer, so please use your own judgement. No strong opinion
> either way. :-)
I'm confused too. So, I have escalated this question to
guix-devel. Let's see what others have to say.
Reply sent
to
Arun Isaac <arunisaac <at> systemreboot.net>
:
You have taken responsibility.
(Wed, 30 Aug 2017 07:31:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Arun Isaac <arunisaac <at> systemreboot.net>
:
bug acknowledged by developer.
(Wed, 30 Aug 2017 07:31:02 GMT)
Full text and
rfc822 format available.
Message #37 received at 27468-done <at> debbugs.gnu.org (full text, mbox):
Pushed! Thanks for the review!
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Wed, 27 Sep 2017 11:24:05 GMT)
Full text and
rfc822 format available.
This bug report was last modified 6 years and 184 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.