GNU bug report logs - #47964
webkitgtk-2.32.0 build fails

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 47964 in the body.
You can then email your comments to 47964 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bone Baboon <bone.baboon <at> disroot.org>
To: bug-guix <at> gnu.org
Subject: webkitgtk-2.32.0 build fails
Date: Fri, 23 Apr 2021 00:46:02 -0400

On an x86_64 computer when I run `sudo guix system --no-substitutes
reconfigure config.scm` webkitgtk-2.32.0 fails to build.  When I run
`guix build --no-substitutes --cores=1 --max-jobs=1 webkitgtk` I get
this error message:

```
make: *** [Makefile:155: all] Error 2
command "make" "-j" "1" failed with status 2
builder for `/gnu/store/b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv' failed with exit code 1
build of /gnu/store/b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv failed
View build log at '/var/log/guix/drvs/b1/vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv.bz2'.
guix build: error: build of `/gnu/store/b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv' failed
```

`guix describe` outputs:

```
Generation 7    Apr 21 2021 15:56:57    (current)
  guix ae5128e
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: ae5128e21eb7afa66bd7cfd7fd1bc5764d00663e
```

End of the build log is below.  The entire build log was too large to
send by email.

```
[ 98%] Built target gtkdoc
make -f Source/WebKit/CMakeFiles/WebKit2WebExtension-4-gir.dir/build.make Source/WebKit/CMakeFiles/WebKit2WebExtension-4-gir.dir/depend
make[2]: Entering directory '/tmp/guix-build-webkitgtk-2.32.0.drv-0/build'
cd /tmp/guix-build-webkitgtk-2.32.0.drv-0/build && /gnu/store/3dsl2jalrcyldkrsqab1hc6sv8pyag9z-cmake-minimal-3.16.5/bin/cmake -E cmake_depends "Unix Makefiles" /tmp/guix-build-webkitgtk-2.32.0.drv-0/webkitgtk-2.32.0 /tmp/guix-build-webkitgtk-2.32.0.drv-0/webkitgtk-2.32.0/Source/WebKit /tmp/guix-build-webkitgtk-2.32.0.drv-0/build /tmp/guix-build-webkitgtk-2.32.0.drv-0/build/Source/WebKit /tmp/guix-build-webkitgtk-2.32.0.drv-0/build/Source/WebKit/CMakeFiles/WebKit2WebExtension-4-gir.dir/DependInfo.cmake --color=
Scanning dependencies of target WebKit2WebExtension-4-gir
make[2]: Leaving directory '/tmp/guix-build-webkitgtk-2.32.0.drv-0/build'
make -f Source/WebKit/CMakeFiles/WebKit2WebExtension-4-gir.dir/build.make Source/WebKit/CMakeFiles/WebKit2WebExtension-4-gir.dir/build
make[2]: Entering directory '/tmp/guix-build-webkitgtk-2.32.0.drv-0/build'
make[2]: *** No rule to make target 'JavaScriptCore-4.0.gir', needed by 'WebKit2-4.0.gir'.  Stop.
make[2]: Leaving directory '/tmp/guix-build-webkitgtk-2.32.0.drv-0/build'
make[1]: *** [CMakeFiles/Makefile2:1628: Source/WebKit/CMakeFiles/WebKit2WebExtension-4-gir.dir/all] Error 2
make[1]: Leaving directory '/tmp/guix-build-webkitgtk-2.32.0.drv-0/build'
make: *** [Makefile:155: all] Error 2
command "make" "-j" "1" failed with status 2
```

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Bone Baboon via Bug reports for GNU Guix <bug-guix <at> gnu.org>
Cc: 47964 <at> debbugs.gnu.org
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Fri, 23 Apr 2021 14:41:14 -0400

[Message part 1 (text/plain, inline)]

On Fri, Apr 23, 2021 at 12:46:02AM -0400, Bone Baboon via Bug reports for GNU Guix wrote:
> builder for `/gnu/store/b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv' failed with exit code 1

Our build farm (ci.guix.gnu.org) doesn't know about this derivation. I
logged in and tried to build it:

------
$ guix build /gnu/store/b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv
cannot build missing derivation ‘/gnu/store/b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv’
guix build: error: build of `/gnu/store/b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv' failed
------

> Generation 7    Apr 21 2021 15:56:57    (current)
>   guix ae5128e
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     branch: master
>     commit: ae5128e21eb7afa66bd7cfd7fd1bc5764d00663e

I tried to produce that derivation with the time machine:

------
$ guix time-machine --commit=ae5128e21eb7afa66bd7cfd7fd1bc5764d00663e -- build webkitgtk --derivations
/gnu/store/z39pp5idq8ikhbf0a80li4ilb121kxfz-webkitgtk-2.32.0.drv
$ guix time-machine --commit=ae5128e21eb7afa66bd7cfd7fd1bc5764d00663e -- build webkitgtk --derivations --no-grafts
/gnu/store/0r5f5k5qckqzq21kfb1cd2m2niw6wsvy-webkitgtk-2.32.0.drv
------

We do have substitutes for those derivations.

This means that something is changing your "view" of Guix. You aren't
building the same things that the rest of us are building.

Are you using any channels? Or maybe custom packages via
GUIX_PACKAGE_PATH? Or anything else we should know about?

Can you send the file
'/gnu/store/b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv'
as an attachment?

In the meantime I am testing the build with a single core.

[signature.asc (application/pgp-signature, inline)]

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bone Baboon <bone.baboon <at> disroot.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 47964 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Fri, 23 Apr 2021 16:11:02 -0400

[Message part 1 (text/plain, inline)]

Since I submitted this bug I have done a `guix pull --no-substitutes`
and tried do reconfigure the system and the build of webkitgtk fails.

Leo Famulari writes:
> Are you using any channels?

There is no `~/.config/guix/channels.scm` file.

There are no extra channel showing in the output of `guix pull
--list-generations`. The output for the current generation is:

```
Generation 8    Apr 22 2021 18:20:30    (current)
  guix c22204c
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: c22204cf7bf12056c43e5da2fd99c95866143ddc
  5 new packages: cl-pathname-utils, daemonize,
    ecl-pathname-utils, kraken2, sbcl-pathname-utils
  25 packages upgraded: cpupower <at> 5.11.16,
    exfatprogs <at> 1.1.1, freefall <at> 5.11.16,
    guix-build-coordinator <at> 0-25.c49b1c8, icedove-wayland <at> 78.10.0,
    icedove <at> 78.10.0, libretro-lowresnx <at> 1.2, libupnp <at> 1.14.6,
    libxlsxwriter <at> 1.0.3, linux-libre-bpf <at> 5.11.16,
    linux-libre-headers <at> 5.11.16, linux-libre <at> 5.11.16,
    openconnect-sso <at> 0.7.3, openvpn <at> 2.5.2, perf <at> 5.11.16, retroarch <at> 1.9.1,
    tmon <at> 5.11.16, turbostat <at> 5.11.16, waybar <at> 0.9.7,
    wine-staging-patchset-data <at> 6.6, wine-staging <at> 6.6,
    wine64-staging <at> 6.6, wireshark <at> 3.4.5, wxmaxima <at> 21.04.0,
    x86-energy-perf-policy <at> 5.11.16
```

> Or maybe custom packages via GUIX_PACKAGE_PATH?

`echo $GUIX_PACKAGE_PATH` outputs an empty line.

> Or anything else we should know about?

What else should I look into?

> Can you send the file
> '/gnu/store/b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv'
> as an attachment?

The derivation is attached.

[b1vs7n6dvmsrk9510q8hqjnq6m9mvrm3-webkitgtk-2.32.0.drv (text/plain, attachment)]

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Bone Baboon <bone.baboon <at> disroot.org>
Cc: 47964 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Fri, 23 Apr 2021 16:25:07 -0400

On Fri, Apr 23, 2021 at 04:11:02PM -0400, Bone Baboon wrote:
> The derivation is attached.

Thanks.

I'm comparing your derivation to the canonical derivation for webkitgtk
from Guix Git commit ae5128e21eb7 on x86_64-linux, /gnu/store/0r5f5k5qckqzq21kfb1cd2m2niw6wsvy-webkitgtk-2.32.0.drv

I don't have very good tools for this, but vbindiff is better than
nothing:

$ vbindiff /gnu/store/0r5f5k5qckqzq21kfb1cd2m2niw6wsvy-webkitgtk-2.32.0.drv bone-baboon-webkitgtk.drv

And I see that many dependency's derivations are also different, and
things are also listed in a different order.

To be continued...

Message #26 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bone Baboon <bone.baboon <at> disroot.org>
To: Bone Baboon <bone.baboon <at> disroot.org>
Cc: 47964 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Fri, 23 Apr 2021 17:37:31 -0400

[Message part 1 (text/plain, inline)]

Bone Baboon via Bug reports for GNU Guix writes:
> End of the build log is below.  The entire build log was too large to
> send by email.

Here is the entire build log compressed and attached.

[webkitgtk-full-build-log.txt.lz (application/octet-stream, attachment)]

Message #32 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Bone Baboon <bone.baboon <at> disroot.org>
Cc: 47964 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Fri, 23 Apr 2021 18:04:51 -0400

On Fri, Apr 23, 2021 at 04:25:07PM -0400, Leo Famulari wrote:
> To be continued...

Well, I think that all that was a red herring.

I googled your error message:

"No rule to make target 'JavaScriptCore-4.0.gir', needed by 'WebKit2-4.0.gir'.  Stop."

And there are very many results, from many distros, and the answer in
all cases seems to be "Don't build this package with -j1".

I have to agree with that :) It's weird that a build fails with -j1, but
this build takes a long time even with many cores.

The upstream bug report:

https://bugs.webkit.org/show_bug.cgi?id=195251

Message #38 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bone Baboon <bone.baboon <at> disroot.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 47964 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Mon, 26 Apr 2021 09:58:36 -0400

Leo Famulari writes:
> I googled your error message:
>
> "No rule to make target 'JavaScriptCore-4.0.gir', needed by 'WebKit2-4.0.gir'.  Stop."
>
> And there are very many results, from many distros, and the answer in
> all cases seems to be "Don't build this package with -j1".
>
> I have to agree with that :) It's weird that a build fails with -j1, but
> this build takes a long time even with many cores.
>
> The upstream bug report:
>
> https://bugs.webkit.org/show_bug.cgi?id=195251

Thank you for sharing that.

`guix build --no-substitutes --cores=1 --max-jobs=1 webkitgtk`
`sudo guix system --no-substitutes --cores=1 --max-jobs=1 reconfigure configuration.scm`
These two commands fail to build webkitgtk with the error message
reported previously.  The problem being what you have identified (using
one core).

`sudo guix system --no-substitutes reconfigure configuration.scm`
Fails to build webkitgtk because of what looks like an out of memory
issue.  With "oom_reaper" showing up in /var/log/messages.

`sudo guix system --no-substitutes --cores=2 --max-jobs=1 reconfigure configuration.scm`
This command was successfully able to build webkitgtk.

Message #46 received at 47964-done <at> debbugs.gnu.org (full text, mbox):

From: Bone Baboon <bone.baboon <at> disroot.org>
To: 47964-done <at> debbugs.gnu.org
Subject: Closing bug#47964
Date: Mon, 26 Apr 2021 10:52:27 -0400

Closing bug#47964 because webkitgtk can successfully be built using the
command `sudo guix system --no-substitutes --cores=2 --max-jobs=1
reconfigure configuration.scm`.  The build of webkitgtk is sensitive to
the number of cores used.

Message #49 received at 47964 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Bone Baboon <bone.baboon <at> disroot.org>
Cc: 47964 <at> debbugs.gnu.org
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Mon, 26 Apr 2021 12:08:12 -0400

On Mon, Apr 26, 2021 at 09:58:36AM -0400, Bone Baboon wrote:
> `guix build --no-substitutes --cores=1 --max-jobs=1 webkitgtk`
> `sudo guix system --no-substitutes --cores=1 --max-jobs=1 reconfigure configuration.scm`
> These two commands fail to build webkitgtk with the error message
> reported previously.  The problem being what you have identified (using
> one core).
> 
> `sudo guix system --no-substitutes reconfigure configuration.scm`
> Fails to build webkitgtk because of what looks like an out of memory
> issue.  With "oom_reaper" showing up in /var/log/messages.
> 
> `sudo guix system --no-substitutes --cores=2 --max-jobs=1 reconfigure configuration.scm`
> This command was successfully able to build webkitgtk.

Thanks for taking the time to perform all these builds; I'm sure they
took a long time.

I'm guessing that "--cores=2 --max-jobs=1" used fewer threads than the
guix-daemon would normally choose by default on your computer? Like,
maybe you have 4 cores / hyperthreads or something like that?

It's unusual that a program requires multithreaded compilation, but I
wonder if we can add a property to the package definition to require at
least 2 threads when building.

Message #52 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Leo Famulari <leo <at> famulari.name>
Cc: 47964 <at> debbugs.gnu.org, bug-guix <at> gnu.org,
 Bone Baboon <bone.baboon <at> disroot.org>
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Mon, 26 Apr 2021 18:44:10 +0200

[Message part 1 (text/plain, inline)]

Leo,

Leo Famulari 写道：
> It's unusual that a program requires multithreaded compilation, 
> but I
> wonder if we can add a property to the package definition to 
> require at
> least 2 threads when building.

That's a great idea, although I'm not keen on .*using properties 
for it.

How about this?

Kind regards,

T G-R

[0001-gnu-webkitgtk-Always-use-at-least-2-make-jobs.patch (text/x-patch, inline)]

From a9ef0c78c6aa60ab49f6fcf4fc6b43afdb6f0183 Mon Sep 17 00:00:00 2001
From: Tobias Geerinckx-Rice <me <at> tobias.gr>
Date: Mon, 26 Apr 2021 18:39:58 +0200
Subject: [PATCH] gnu: webkitgtk: Always use at least 2 make jobs.

Fixes <http://issues.guix.gnu.org/47964>.

* gnu/packages/webkit.scm (webkitgtk)[arguments]: Use at least -j2.
---
 gnu/packages/webkit.scm | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/gnu/packages/webkit.scm b/gnu/packages/webkit.scm
index 083c8ccb08..74ffe92b84 100644
--- a/gnu/packages/webkit.scm
+++ b/gnu/packages/webkit.scm
@@ -260,6 +260,9 @@ acceleration in mind, leveraging common 3D graphics APIs for best performance.")
                           ;; now, we explicitly disable it to prevent an error
                           ;; at configuration time.
                           "-DUSE_GSTREAMER_GL=OFF")
+       #:make-flags
+       ;; Never build with unsupported -j1: https://issues.guix.gnu.org/47964#5
+       (list "-j" (number->string (max 2 (parallel-job-count))))
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'configure-bubblewrap-store-directory
-- 
2.31.1

[signature.asc (application/pgp-signature, inline)]

Message #58 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: 47964 <at> debbugs.gnu.org, bug-guix <at> gnu.org,
 Bone Baboon <bone.baboon <at> disroot.org>
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Mon, 26 Apr 2021 13:19:37 -0400

[Message part 1 (text/plain, inline)]

On Mon, Apr 26, 2021 at 06:44:10PM +0200, Tobias Geerinckx-Rice wrote:
> How about this?

> From a9ef0c78c6aa60ab49f6fcf4fc6b43afdb6f0183 Mon Sep 17 00:00:00 2001
> From: Tobias Geerinckx-Rice <me <at> tobias.gr>
> Date: Mon, 26 Apr 2021 18:39:58 +0200
> Subject: [PATCH] gnu: webkitgtk: Always use at least 2 make jobs.
> 
> Fixes <http://issues.guix.gnu.org/47964>.
> 
> * gnu/packages/webkit.scm (webkitgtk)[arguments]: Use at least -j2.
> ---
>  gnu/packages/webkit.scm | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/gnu/packages/webkit.scm b/gnu/packages/webkit.scm
> index 083c8ccb08..74ffe92b84 100644
> --- a/gnu/packages/webkit.scm
> +++ b/gnu/packages/webkit.scm
> @@ -260,6 +260,9 @@ acceleration in mind, leveraging common 3D graphics APIs for best performance.")
>                            ;; now, we explicitly disable it to prevent an error
>                            ;; at configuration time.
>                            "-DUSE_GSTREAMER_GL=OFF")
> +       #:make-flags
> +       ;; Never build with unsupported -j1: https://issues.guix.gnu.org/47964#5
> +       (list "-j" (number->string (max 2 (parallel-job-count))))

LGTM

[signature.asc (application/pgp-signature, inline)]

Message #64 received at 47964 <at> debbugs.gnu.org (full text, mbox):

From: Bone Baboon <bone.baboon <at> disroot.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 47964 <at> debbugs.gnu.org
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Mon, 26 Apr 2021 14:25:28 -0400

Leo Famulari writes:
> Thanks for taking the time to perform all these builds; I'm sure they
> took a long time.

Thank you for helping me figure out how to build this package.

> I'm guessing that "--cores=2 --max-jobs=1" used fewer threads than the
> guix-daemon would normally choose by default on your computer? Like,
> maybe you have 4 cores / hyperthreads or something like that?

Yes that is correct. On the computer that was having trouble building
webkitgtk there are 4 cores and 8 threads and 8GiB of installed memory.

Message #67 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Leo Famulari <leo <at> famulari.name>
Cc: 47964 <at> debbugs.gnu.org, bug-guix <at> gnu.org,
 Bone Baboon <bone.baboon <at> disroot.org>
Subject: Re: bug#47964: webkitgtk-2.32.0 build fails
Date: Tue, 27 Apr 2021 14:48:30 +0200

[Message part 1 (text/plain, inline)]

Leo Famulari 写道：
> LGTM

Thanks Leo!  Pushed as e5c65fb7372e10a2d6531d25f3b7d7c31679a21e.

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.