GNU bug report logs - #43418
ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 43418 in the body.
You can then email your comments to 43418 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nathan Dehnel <ncdehnel <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of
 youtube-dl so it will function correctly
Date: Tue, 15 Sep 2020 06:27:56 -0500

bash-5.0$ youtube-dl -x https://www.youtube.com/watch?v=7Ijd_iN9Blk
[youtube] 7Ijd_iN9Blk: Downloading webpage
[download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been downloaded
[download] 100% of 15.45MiB
ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please install one.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Nathan Dehnel <ncdehnel <at> gmail.com>
Cc: 43418 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as
 dependencies of youtube-dl so it will function correctly
Date: Tue, 15 Sep 2020 14:06:11 +0200

[Message part 1 (text/plain, inline)]

Hi Nathan,

Nathan Dehnel 写道：
> bash-5.0$ youtube-dl -x 
> https://www.youtube.com/watch?v=7Ijd_iN9Blk
> [youtube] 7Ijd_iN9Blk: Downloading webpage
> [download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been 
> downloaded
> [download] 100% of 15.45MiB
> ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please 
> install one.

It works fine without ‘-x’:

 λ youtube-dl https://www.youtube.com/watch?v=7Ijd_iN9Blk
 [youtube] 7Ijd_iN9Blk: Downloading webpage
 [download] Destination: Deadmau5 - Hit Save-7Ijd_iN9Blk.mp4
 [download] 100% of 37.09MiB in 00:02

This *is* functioning correctly in my view: that advanced options 
optionally require run-time dependencies is not a bug but a 
feature.  The programme clearly explains which package could be 
installed to continue, and the user can chose which one they 
prefer - including neither, which suffices for the majority of 
youtube-dling.

Matters would be different if the error message were less clear, 
or perhaps if ffmpeg weren't so insanely great:

 λ guix size youtube-dl | tail -n1
 total: 186.9 MiB
 λ guix size youtube-dl ffmpeg | tail -n1
 total: 811.2 MiB

(!)

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: raingloom <raingloom <at> riseup.net>
To: Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix <at> gnu.org>
Cc: Nathan Dehnel <ncdehnel <at> gmail.com>, 43418 <at> debbugs.gnu.org,
 Tobias Geerinckx-Rice <me <at> tobias.gr>
Subject: Re: bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as
 dependencies of youtube-dl so it will function correctly
Date: Tue, 15 Sep 2020 21:34:36 +0200

On Tue, 15 Sep 2020 14:06:11 +0200
Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix <at> gnu.org>
wrote:

> Hi Nathan,
> 
> Nathan Dehnel 写道：
> > bash-5.0$ youtube-dl -x 
> > https://www.youtube.com/watch?v=7Ijd_iN9Blk
> > [youtube] 7Ijd_iN9Blk: Downloading webpage
> > [download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been 
> > downloaded
> > [download] 100% of 15.45MiB
> > ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please 
> > install one.  
> 
> It works fine without ‘-x’:
> 
>   λ youtube-dl https://www.youtube.com/watch?v=7Ijd_iN9Blk
>   [youtube] 7Ijd_iN9Blk: Downloading webpage
>   [download] Destination: Deadmau5 - Hit Save-7Ijd_iN9Blk.mp4
>   [download] 100% of 37.09MiB in 00:02
> 
> This *is* functioning correctly in my view: that advanced options 
> optionally require run-time dependencies is not a bug but a 
> feature.  The programme clearly explains which package could be 
> installed to continue, and the user can chose which one they 
> prefer - including neither, which suffices for the majority of 
> youtube-dling.
> 
> Matters would be different if the error message were less clear, 
> or perhaps if ffmpeg weren't so insanely great:
> 
>   λ guix size youtube-dl | tail -n1
>   total: 186.9 MiB
>   λ guix size youtube-dl ffmpeg | tail -n1
>   total: 811.2 MiB
> 
> (!)
> 
> Kind regards,
> 
> T G-R

Maybe it's time we added an "optional dependencies" field?
There seems to be a bug report or help request like every week that
just boils down to "this package has an undocumented dependency".

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix <at> gnu.org>
Cc: Nathan Dehnel <ncdehnel <at> gmail.com>, 43418 <at> debbugs.gnu.org
Subject: Re: bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as
 dependencies of youtube-dl so it will function correctly
Date: Tue, 15 Sep 2020 19:17:06 -0400

On Tue, Sep 15, 2020 at 02:06:11PM +0200, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:
> Matters would be different if the error message were less clear, or perhaps
> if ffmpeg weren't so insanely great:
> 
>  λ guix size youtube-dl | tail -n1
>  total: 186.9 MiB
>  λ guix size youtube-dl ffmpeg | tail -n1
>  total: 811.2 MiB

I wonder, should we expect FFmpeg to already be referenced by somebody's
profile if they are using youtube-dl? VLC and mpv both depend on FFmpeg.
The use case of "download video and watch it on another machine (or
never watch it)" seems somewhat esoteric.

On the other hand, youtube-dl has no explicit dependencies — I guess
that everything it needs is from Python. That's beautiful, and I
understand the reluctance to add a dependency, especially on a "kitchen
sink" package like FFmpeg.

Message #26 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nathan Dehnel <ncdehnel <at> gmail.com>
To: raingloom <raingloom <at> riseup.net>
Cc: 43418 <at> debbugs.gnu.org, Tobias Geerinckx-Rice <me <at> tobias.gr>,
 Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix <at> gnu.org>
Subject: Re: bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as
 dependencies of youtube-dl so it will function correctly
Date: Tue, 15 Sep 2020 18:52:38 -0500

Ludovic wanted to add useflags to guix.
https://lists.gnu.org/archive/html/guix-devel/2019-05/msg00285.html
I don't know what became of it.

On Tue, Sep 15, 2020 at 2:44 PM raingloom <raingloom <at> riseup.net> wrote:
>
> On Tue, 15 Sep 2020 14:06:11 +0200
> Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix <at> gnu.org>
> wrote:
>
> > Hi Nathan,
> >
> > Nathan Dehnel 写道：
> > > bash-5.0$ youtube-dl -x
> > > https://www.youtube.com/watch?v=7Ijd_iN9Blk
> > > [youtube] 7Ijd_iN9Blk: Downloading webpage
> > > [download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been
> > > downloaded
> > > [download] 100% of 15.45MiB
> > > ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please
> > > install one.
> >
> > It works fine without ‘-x’:
> >
> >   λ youtube-dl https://www.youtube.com/watch?v=7Ijd_iN9Blk
> >   [youtube] 7Ijd_iN9Blk: Downloading webpage
> >   [download] Destination: Deadmau5 - Hit Save-7Ijd_iN9Blk.mp4
> >   [download] 100% of 37.09MiB in 00:02
> >
> > This *is* functioning correctly in my view: that advanced options
> > optionally require run-time dependencies is not a bug but a
> > feature.  The programme clearly explains which package could be
> > installed to continue, and the user can chose which one they
> > prefer - including neither, which suffices for the majority of
> > youtube-dling.
> >
> > Matters would be different if the error message were less clear,
> > or perhaps if ffmpeg weren't so insanely great:
> >
> >   λ guix size youtube-dl | tail -n1
> >   total: 186.9 MiB
> >   λ guix size youtube-dl ffmpeg | tail -n1
> >   total: 811.2 MiB
> >
> > (!)
> >
> > Kind regards,
> >
> > T G-R
>
> Maybe it's time we added an "optional dependencies" field?
> There seems to be a bug report or help request like every week that
> just boils down to "this package has an undocumented dependency".

Message #32 received at 43418 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Leo Famulari <leo <at> famulari.name>, 43418 <at> debbugs.gnu.org
Cc: ncdehnel <at> gmail.com
Subject: Re: bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as
 dependencies of youtube-dl so it will function correctly
Date: Wed, 16 Sep 2020 04:14:36 -0400

Leo Famulari <leo <at> famulari.name> writes:

> On Tue, Sep 15, 2020 at 02:06:11PM +0200, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:
>> Matters would be different if the error message were less clear, or perhaps
>> if ffmpeg weren't so insanely great:
>> 
>>  λ guix size youtube-dl | tail -n1
>>  total: 186.9 MiB
>>  λ guix size youtube-dl ffmpeg | tail -n1
>>  total: 811.2 MiB
>
> I wonder, should we expect FFmpeg to already be referenced by somebody's
> profile if they are using youtube-dl? VLC and mpv both depend on FFmpeg.
> The use case of "download video and watch it on another machine (or
> never watch it)" seems somewhat esoteric.

I was just about to disagree with you, on the grounds that GStreamer,
and in particular Totem (a.k.a. GNOME Videos) does not depend on FFmpeg.
It is, in fact, possible to watch videos downloaded using 'youtube-dl',
with only gst-plugins-{base,good} installed, provided that you request
freedom-respecting formats like VP9 and Opus.

How to request freedom-respecting formats?  Unfortunately, the obvious
"--prefer-free-formats" usually doesn't work in my experience, but you
can run "youtube-dl -F <URL>" to see a list of available formats, and
then something like "youtube-dl -f 247+251 <URL>" (for non-live youtube
videos of sufficient resolution) to download VP9 and Opus and combine
them into a file.

However, in my recent experience, youtube usually provides the
freedom-respecting formats only as separate audio and video streams
which must be recombined by youtube-dl, and in that case the 'ffmpeg'
command line tool seems to be required.

So, ultimately I tend to agree with Leo that 'youtube-dl' is unlikely to
be useful without 'ffmpeg', and therefore it might as well depend on it
somehow.

I see two possible approaches: ffmpeg could be added to youtube-dl's
propagated-inputs, or it could be instead be added to 'inputs', and we
could substitute hardcoded paths to 'ffmpeg' in the built 'youtube-dl'.
I generally tend to favor the latter approach, but there are valid
arguments for each approach and I don't feel strongly about it.

What do you think?

       Mark

Message #35 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Mark H Weaver <mhw <at> netris.org>
Cc: ncdehnel <at> gmail.com, 43418 <at> debbugs.gnu.org, bug-guix <at> gnu.org,
 Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as
 dependencies of youtube-dl so it will function correctly
Date: Wed, 16 Sep 2020 10:58:14 +0200

[Message part 1 (text/plain, inline)]

Leo, Mark,

Mark H Weaver 写道：
> Leo Famulari <leo <at> famulari.name> writes:
>
>> On Tue, Sep 15, 2020 at 02:06:11PM +0200, Tobias Geerinckx-Rice 
>> via Bug reports for GNU Guix wrote:
>>> Matters would be different if the error message were less 
>>> clear, or perhaps
>>> if ffmpeg weren't so insanely great:
>>> 
>>>  λ guix size youtube-dl | tail -n1
>>>  total: 186.9 MiB
>>>  λ guix size youtube-dl ffmpeg | tail -n1
>>>  total: 811.2 MiB
>>
>> I wonder, should we expect FFmpeg to already be referenced by 
>> somebody's
>> profile if they are using youtube-dl? VLC and mpv both depend 
>> on FFmpeg.
>> The use case of "download video and watch it on another machine 
>> (or
>> never watch it)" seems somewhat esoteric.

I don't see why my use case should be sidelined as ‘esoteric’. 
;-)

> However, in my recent experience, youtube usually provides the
> freedom-respecting formats only as separate audio and video 
> streams
> which must be recombined by youtube-dl, and in that case the 
> 'ffmpeg'
> command line tool seems to be required.

This is a good argument.  If including ffmpeg by default makes it 
easier to request & mux freeër formats, let's do that.

> I see two possible approaches: ffmpeg could be added to 
> youtube-dl's
> propagated-inputs, or it could be instead be added to 'inputs', 
> and we
> could substitute hardcoded paths to 'ffmpeg' in the built 
> 'youtube-dl'.
> I generally tend to favor the latter approach, but there are 
> valid
> arguments for each approach and I don't feel strongly about it.

Which advantages does the former have?  I wasn't aware of any.

Thanks!

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #41 received at 43418 <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: 43418 <at> debbugs.gnu.org
Subject: [PATCH] gnu: youtube-dl: Refer to ffmpeg.
Date: Wed, 16 Sep 2020 11:54:15 +0200

Addresses <https://issues.guix.gnu.org/43418>.

* gnu/packages/video.scm (youtube-dl)[inputs]: Add ffmpeg.
[arguments]: Fall back to input ffmpeg instead of searching $PATH.
---
 gnu/packages/video.scm | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/gnu/packages/video.scm b/gnu/packages/video.scm
index 8ebe7dbcdf..50f751f8da 100644
--- a/gnu/packages/video.scm
+++ b/gnu/packages/video.scm
@@ -2184,6 +2184,13 @@ To load this plugin, specify the following option when starting mpv:
      ;; 'youtube-dl.bash-completion'.
      `(#:tests? #f ; Many tests fail. The test suite can be run with pytest.
        #:phases (modify-phases %standard-phases
+                  (add-after 'unpack 'default-to-the-ffmpeg-input
+                    (lambda _
+                      ;; Continue respecting the --ffmpeg-location argument.
+                      (substitute* "youtube_dl/postprocessor/ffmpeg.py"
+                        (("\\.get\\('ffmpeg_location'\\)" match)
+                         (format #f "~a or '~a'" match (which "ffmpeg"))))
+                      #t))
                   (add-before 'install 'fix-the-data-directories
                     (lambda* (#:key outputs #:allow-other-keys)
                       (let ((prefix (assoc-ref outputs "out")))
@@ -2207,6 +2214,8 @@ To load this plugin, specify the following option when starting mpv:
                         (copy-file "youtube-dl.zsh"
                                    (string-append zsh "/_youtube-dl"))
                         #t))))))
+    (inputs
+     `(("ffmpeg" ,ffmpeg)))
     (synopsis "Download videos from YouTube.com and other sites")
     (description
      "Youtube-dl is a small command-line program to download videos from
-- 
2.28.0

Message #44 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: ncdehnel <at> gmail.com, 43418 <at> debbugs.gnu.org, bug-guix <at> gnu.org,
 Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as
 dependencies of youtube-dl so it will function correctly
Date: Wed, 16 Sep 2020 07:30:35 -0400

Hi Tobias,

Tobias Geerinckx-Rice <me <at> tobias.gr> writes:

> Mark H Weaver 写道：
>
>> I see two possible approaches: ffmpeg could be added to youtube-dl's
>> propagated-inputs, or it could be instead be added to 'inputs', and
>> we could substitute hardcoded paths to 'ffmpeg' in the built
>> 'youtube-dl'.  I generally tend to favor the latter approach, but
>> there are valid arguments for each approach and I don't feel strongly
>> about it.
>
> Which advantages does the former have?  I wasn't aware of any.

The former approach is arguably simpler, easier to maintain, and if the
user upgrades only 'ffmpeg' while leaving 'youtube-dl' unchanged, with
the latter approach 'youtube-dl' will continue using the old 'ffmpeg'
with its potential security holes, which users unfamiliar with Guix
might not expect.

I still generally prefer the latter approach for several reasons, most
notably because it means fewer dynamically-determined dependencies,
which are contrary to our goals for reproducible behavior, i.e. that if
the software in a given store item works today for your use case, it
will work the same way tomorrow.

However, it's worth noting that this property is less useful for a
program like 'youtube-dl', because a copy of 'youtube-dl' that works
today might well stop working tomorrow because of changes made on the
server side, regardless of our efforts.

      Mark

Message #50 received at 43418 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>, 43418 <at> debbugs.gnu.org
Subject: Re: bug#43418: [PATCH] gnu: youtube-dl: Refer to ffmpeg.
Date: Wed, 16 Sep 2020 07:31:03 -0400

Hi Tobias,

> Addresses <https://issues.guix.gnu.org/43418>.
>
> * gnu/packages/video.scm (youtube-dl)[inputs]: Add ffmpeg.
> [arguments]: Fall back to input ffmpeg instead of searching $PATH.

Looks good to me.  Thank you!

     Mark

Message #55 received at 43418-done <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Mark H Weaver <mhw <at> netris.org>, zimoun <zimon.toutoune <at> gmail.com>
Cc: 43418-done <at> debbugs.gnu.org
Subject: Re: bug#43418: [PATCH] gnu: youtube-dl: Refer to ffmpeg.
Date: Wed, 16 Sep 2020 13:54:42 +0200

[Message part 1 (text/plain, inline)]

Mark, Simon,

Mark H Weaver 写道：
>> Addresses <https://issues.guix.gnu.org/43418>.
>>
>> * gnu/packages/video.scm (youtube-dl)[inputs]: Add ffmpeg.
>> [arguments]: Fall back to input ffmpeg instead of searching 
>> $PATH.
>
> Looks good to me.  Thank you!

Thanks, pushed.

> The usage seems more Fixes than Addresses.

I'm still of the opinion that this was no bug but a feature 
request, and there was nothing to fix.  I promise to add it to the 
release notes if you think it's necessary, but would rather keep 
those limited to ‘real’ bugs as well.  It's not like we're short 
on those ;-)

Whatever it was, it's now closed, and we'll be spared of 
duplicates from now on.

Thanks all,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #58 received at 43418-done <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: Mark H Weaver <mhw <at> netris.org>, 43418-done <at> debbugs.gnu.org
Subject: Re: bug#43418: [PATCH] gnu: youtube-dl: Refer to ffmpeg.
Date: Wed, 16 Sep 2020 14:07:52 +0200

On Wed, 16 Sep 2020 at 13:54, Tobias Geerinckx-Rice <me <at> tobias.gr> wrote:

> > The usage seems more Fixes than Addresses.
>
> I'm still of the opinion that this was no bug but a feature
> request, and there was nothing to fix.  I promise to add it to the
> release notes if you think it's necessary, but would rather keep
> those limited to ‘real’ bugs as well.  It's not like we're short
> on those ;-)

I am fine with whatever the convention is.  Just one that we
collectively share. :-)
It helps to dig with the tandem "git log --grep" and "M-x debbugs", IMHO.
I mean it could make sense to have Fixes for "real" bugs and Addresses
for anything else tracked by debbugs (whishlist, typo, etc.).

Cheers,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.