GNU bug report logs - #53388
New package: gallery-dl

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 53388 in the body.
You can then email your comments to 53388 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bird <birdsite <at> airmail.cc>
To: guix-patches <at> gnu.org
Subject: New package: gallery-dl
Date: Thu, 20 Jan 2022 11:48:25 +0000

[Message part 1 (text/plain, inline)]

Hello,

The attached patch adds a new package: gallery-dl in
gnu/packages/video.scm

Thanks!

[0001-gnu-gallery-dl-Add-a-new-package.patch (text/x-patch, attachment)]

Message #8 received at 53388 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Bird <birdsite <at> airmail.cc>, 53388 <at> debbugs.gnu.org
Subject: Re: [bug#53388] New package: gallery-dl
Date: Thu, 20 Jan 2022 17:10:02 +0000

[Message part 1 (text/plain, inline)]

Hi,

Bird schreef op do 20-01-2022 om 11:48 [+0000]:
> +    (propagated-inputs (list python-requests))

The README says it can use ‘youtube-dl’ or ‘yt-dlp’ for vide downloads
and ‘ffmpeg‘ for some conversion, so maybe these could be added?

Also, propagation can lead to profile collisions which can be
complicated to resolve, so can this be made a regular input instead of
a propagated input?  Some wrap-program might be necessary.

> +    (home-page "https://github.com/mikf/gallery-dl")
> +    (synopsis "Command-line program to download images from several
> sites")
> +    (description "Command-line program to download image galleries
> + and collections from several image hosting sites")

Does it only support images, or also videos?  From the ‘ffmpeg’ and
‘youtube-dl’ dependency, I would assume the latter.  Also, when I'm
reading this description, it reminds me a lot of youtube-dl -- perhaps
the description could explain it supports more video sites, or less
video sites, or not only video but also images, or something along
these lines.

> +    (license gpl2+)
> +    (arguments
> +     '(#:tests? #f))))

Why are tests disabled here?

Also, 'gallery-dl' (indirectly) uses openssl for the S in HTTPS,
so you may need to add SSL_CERT_DIR (or SSL_CERT_FILE, not sure) to
native-search-paths to make "guix shell --pure yt-dlp nss-certs" work.

Although there appears to be some disagreement in that area, see e.g.
some discussion in #53324, so maybe not.

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 53388 <at> debbugs.gnu.org (full text, mbox):

From: Bird <birdsite <at> airmail.cc>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 53388 <at> debbugs.gnu.org, Bird <birdsite <at> airmail.cc>
Subject: Re: [bug#53388] New package: gallery-dl
Date: Thu, 20 Jan 2022 17:42:17 +0000

Maxime Devos <maximedevos <at> telenet.be> writes:

> Hi,
>
> Bird schreef op do 20-01-2022 om 11:48 [+0000]:
>> +    (propagated-inputs (list python-requests))
>
> The README says it can use ‘youtube-dl’ or ‘yt-dlp’ for vide downloads
> and ‘ffmpeg‘ for some conversion, so maybe these could be added?
>
> Also, propagation can lead to profile collisions which can be
> complicated to resolve, so can this be made a regular input instead of
> a propagated input?  Some wrap-program might be necessary.

Can programs call to regular inputs that are not propagated?  Since
gallery-dl will need ffmpeg and python-requests (and possibly yt-dlp),
would it need to be in user's profile?

>
>> +    (home-page "https://github.com/mikf/gallery-dl")
>> +    (synopsis "Command-line program to download images from several
>> sites")
>> +    (description "Command-line program to download image galleries
>> + and collections from several image hosting sites")
>
> Does it only support images, or also videos?  From the ‘ffmpeg’ and
> ‘youtube-dl’ dependency, I would assume the latter.  Also, when I'm
> reading this description, it reminds me a lot of youtube-dl -- perhaps
> the description could explain it supports more video sites, or less
> video sites, or not only video but also images, or something along
> these lines.
>

It's really similar to youtube-dl but targets images primarily, i'll
make that clearer in the next patch

>
>> +    (license gpl2+)
>> +    (arguments
>> +     '(#:tests? #f))))
>
> Why are tests disabled here?
>

Sorry, I just assumed it will fail the tests due to network-lessness of
build environment, it actually passes the tests.

>
> Also, 'gallery-dl' (indirectly) uses openssl for the S in HTTPS,
> so you may need to add SSL_CERT_DIR (or SSL_CERT_FILE, not sure) to
> native-search-paths to make "guix shell --pure yt-dlp nss-certs" work.
>
> Although there appears to be some disagreement in that area, see e.g.
> some discussion in #53324, so maybe not.
>

I did a quick grep of SSL_CERT_FILE and SSL_CERT_DIR in gnu/packages/
but didn't find anything, could you please explain what i should do
here? 

>
> Greetings,
> Maxime.
>

Thanks,
Bird

Message #14 received at 53388 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Bird <birdsite <at> airmail.cc>
Cc: 53388 <at> debbugs.gnu.org
Subject: Re: [bug#53388] New package: gallery-dl
Date: Thu, 20 Jan 2022 13:24:17 -0500

On Thu, Jan 20, 2022 at 11:48:25AM +0000, Bird wrote:
> The attached patch adds a new package: gallery-dl in
> gnu/packages/video.scm

> From 799d589e6b5d76b4cac6272d22f87603729fb9e1 Mon Sep 17 00:00:00 2001
> From: Bird <birdsite <at> airmail.cc>
> Date: Thu, 20 Jan 2022 11:42:24 +0000
> Subject: [PATCH] gnu: gallery-dl: Add a new package. *
>  gnu/packages/video.scm(gallery-dl): Add package.

Hello, and thanks for the patch!

I applied it to a Git checkout of the Guix source code, and tried
rebuilding Guix, as described here in the Contributing chapter of the
manual:

https://guix.gnu.org/manual/devel/en/html_node/Building-from-Git.html

However, it crashes like this:

------
ice-9/eval.scm:293:34: error: gpl2+: unbound variable
hint: Did you forget `(use-modules (guix licenses))'?

make[2]: *** [Makefile:7380: make-packages-go] Error 1
make[2]: Leaving directory '/home/leo/work/guix'
make[1]: *** [Makefile:6453: all-recursive] Error 1
make[1]: Leaving directory '/home/leo/work/guix'
make: *** [Makefile:3982: all] Error 2
------

The reason for this error is that licenses in the gnu/packages/video.scm
module are "prefixed" to avoid a namespace collision between packages
and licenses with the same name, such as zlib and expat:

https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/video.scm?id=f223535e1c2f052f671e44a6c546d0ebde3591b1#n80

You can check other packages in that module for examples of how to
prefix the license.

> +    (license gpl2+)

I checked several of the .py files in the source code, and setup.py, and
they all seem to specify that the program is made available under the
terms of version 2 of the GPL, but no later versions. So, this package
should use "gpl2", not "gpl2+".

> +    (arguments
> +     '(#:tests? #f))))

We always run upstream test suites in Guix packages, because it helps
validate our packaging and finds bugs to assist upstream development.
If we do not run the tests, we add a code comment explaining why. In
Scheme, comments begin with the ; (semicolon) character.

So, can you send a revised patch that adjusts the license and either
runs the test suite or explains why not?

Thanks in advance!

Message #17 received at 53388 <at> debbugs.gnu.org (full text, mbox):

From: Bird <birdsite <at> airmail.cc>
To: Leo Famulari <leo <at> famulari.name>
Cc: 53388 <at> debbugs.gnu.org, Maxime Devos <maximedevos <at> telenet.be>,
 Bird <birdsite <at> airmail.cc>
Subject: Re: [bug#53388] New package: gallery-dl
Date: Fri, 21 Jan 2022 04:46:59 +0000

[Message part 1 (text/plain, inline)]

[Added Maxime because this patch should fix his worries too]

Leo Famulari <leo <at> famulari.name> writes:

> On Thu, Jan 20, 2022 at 11:48:25AM +0000, Bird wrote:

> I applied it to a Git checkout of the Guix source code, and tried
> rebuilding Guix, as described here in the Contributing chapter of the
> manual:
>
> https://guix.gnu.org/manual/devel/en/html_node/Building-from-Git.html
>
> However, it crashes like this:
[...]
> The reason for this error is that licenses in the gnu/packages/video.scm
> module are "prefixed" to avoid a namespace collision between packages
> and licenses with the same name, such as zlib and expat:
>
> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/video.scm?id=f223535e1c2f052f671e44a6c546d0ebde3591b1#n80
>
> You can check other packages in that module for examples of how to
> prefix the license.
>
>> +    (license gpl2+)

Thanks, fixed.

>
> I checked several of the .py files in the source code, and setup.py,
>and
> they all seem to specify that the program is made available under the
> terms of version 2 of the GPL, but no later versions. So, this package
> should use "gpl2", not "gpl2+".

I was confused due to the example on LICENSE containing phrase 'or (at
your option) any later version' but turns out that was an example.

>
>> +    (arguments
>> +     '(#:tests? #f))))
>
> We always run upstream test suites in Guix packages, because it helps
> validate our packaging and finds bugs to assist upstream development.
> If we do not run the tests, we add a code comment explaining why. In
> Scheme, comments begin with the ; (semicolon) character.
>
> So, can you send a revised patch that adjusts the license and either
> runs the test suite or explains why not?

It is attached in this mail.

>
> Thanks in advance!
>

Thanks!

[0001-gnu-gallery-dl-Add-a-new-package.patch (text/x-patch, attachment)]

Message #22 received at 53388-done <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Bird <birdsite <at> airmail.cc>
Cc: 53388-done <at> debbugs.gnu.org, Maxime Devos <maximedevos <at> telenet.be>
Subject: Re: [bug#53388] New package: gallery-dl
Date: Tue, 25 Jan 2022 13:31:36 -0500

On Fri, Jan 21, 2022 at 04:46:59AM +0000, Bird wrote:
> From 8a3ab5dba4f85dd1fb2ddaf683f9da7e5ec9854e Mon Sep 17 00:00:00 2001
> From: Bird <birdsite <at> airmail.cc>
> Date: Fri, 21 Jan 2022 04:32:39 +0000
> Subject: [PATCH] gnu: gallery-dl: Add a new package. *
>  gnu/packages/video.scm(gallery-dl): Add package.

Thanks! I made some minor cosmetic changes to indentation and
standardized the commit message and pushed as commit
cc74d19d663b78a9ec8207d7e3a41cabfbe1a100.

https://git.savannah.gnu.org/cgit/guix.git/commit/?id=cc74d19d663b78a9ec8207d7e3a41cabfbe1a100

I forgot to make sure your authorship was recorded in the copyright
header, so I did that in a followup commit.

Message #25 received at 53388-done <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Bird <birdsite <at> airmail.cc>
Cc: 53388-done <at> debbugs.gnu.org, Maxime Devos <maximedevos <at> telenet.be>
Subject: Re: [bug#53388] New package: gallery-dl
Date: Tue, 25 Jan 2022 13:37:17 -0500

On Thu, Jan 20, 2022 at 05:42:17PM +0000, Bird wrote:
> > The README says it can use ‘youtube-dl’ or ‘yt-dlp’ for vide downloads
> > and ‘ffmpeg‘ for some conversion, so maybe these could be added?
> >
> > Also, propagation can lead to profile collisions which can be
> > complicated to resolve, so can this be made a regular input instead of
> > a propagated input?  Some wrap-program might be necessary.
> 
> Can programs call to regular inputs that are not propagated?  Since
> gallery-dl will need ffmpeg and python-requests (and possibly yt-dlp),
> would it need to be in user's profile?

They can, but they need to learn how to find them. Usually that happens
by making the package record the "store path" of the input at
build-time.

On the other hand, propagated-inputs appear on $PATH, so if the calling
program does the normal thing it will find them.

But, the drawback of propagating, for example, FFmpeg, is that it makes
it impossible for users to `guix install` a custom FFmpeg package
alongside the propagated FFmpeg.

> >> +    (home-page "https://github.com/mikf/gallery-dl")

I looked at the source code, and this program respects some
configuration options to find the location of youtube-dl and ffmpeg:

https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#ugoiraffmpeg-location

It's a matter of opinion but, to me, both of those programs are things
that users may customize or choose particular versions of, so it's okay
to expect users to install them and configure gallery-dl properly.

If we want to improve this package in that regard... that's for a
followup patch!

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.