GNU bug report logs - #50066
core-update: Raw origin tarballs are not handled correctly.

Previous Next

Package: guix;

Reported by: Mathieu Othacehe <othacehe <at> gnu.org>

Date: Sun, 15 Aug 2021 08:43:02 UTC

Severity: important

Tags: patch

Done: Mathieu Othacehe <othacehe <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 50066 in the body.
You can then email your comments to 50066 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#50066; Package guix. (Sun, 15 Aug 2021 08:43:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mathieu Othacehe <othacehe <at> gnu.org>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Sun, 15 Aug 2021 08:43:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: bug-guix <at> gnu.org
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: core-update: Raw origin tarballs are not handled correctly.
Date: Sun, 15 Aug 2021 10:42:13 +0200
Hello,

When building "emacs-org-contrib" on core-updates-frozen, the following
message is printed:

--8<---------------cut here---------------start------------->8---
tar: Exiting with failure status due to previous errors
error: in phase 'unpack': uncaught exception:
%exception #<&invoke-error program: "tar" arguments: ("xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `unpack' failed after 0.0 seconds
command "tar" "xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar" failed with status 2
note: keeping build directory `/tmp/guix-build-emacs-org-contrib-20210809.drv-6'
builder for `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed with exit code 1
build of /gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv failed
View build log at '/var/log/guix/drvs/52/jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv.bz2'.
guix build: error: build of `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed
--8<---------------cut here---------------end--------------->8---

That's because
"/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar"
is empty. I believe this is a consequence of:
cfcead2e515c0dae02127e5a76496463898be6b6, and in particular this
section:

--8<---------------cut here---------------start------------->8---
              ;; If SOURCE is a directory (such as a checkout), return a
              ;; directory.  Otherwise create a tarball.
              (cond
               ((file-is-directory? #+source)
                (copy-recursively directory #$output
                                  #:log (%make-void-port "w")))
               ((not #+comp)
                (copy-file file #$output)) <---------- We should call repack instead here for raw tarballs?
               (else
                (repack directory #$output)))))))
--8<---------------cut here---------------end--------------->8---

Maxim, could you please have a look?

Thanks,

Mathieu




Information forwarded to bug-guix <at> gnu.org:
bug#50066; Package guix. (Fri, 27 Aug 2021 04:20:02 GMT) Full text and rfc822 format available.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: bug-guix <at> gnu.org
Subject: Re: core-update: Raw origin tarballs are not handled correctly.
Date: Fri, 27 Aug 2021 00:19:33 -0400
Hello Mathieu,

Mathieu Othacehe <othacehe <at> gnu.org> writes:

> Hello,
>
> When building "emacs-org-contrib" on core-updates-frozen, the following
> message is printed:
>
> tar: Exiting with failure status due to previous errors
> error: in phase 'unpack': uncaught exception:
> %exception #<&invoke-error program: "tar" arguments: ("xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar") exit-status: 2 term-signal: #f stop-signal: #f> 
> phase `unpack' failed after 0.0 seconds
> command "tar" "xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar" failed with status 2
> note: keeping build directory `/tmp/guix-build-emacs-org-contrib-20210809.drv-6'
> builder for `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed with exit code 1
> build of /gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv failed
> View build log at '/var/log/guix/drvs/52/jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv.bz2'.
> guix build: error: build of `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed
>
>
> That's because
> "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar"
> is empty. I believe this is a consequence of:
> cfcead2e515c0dae02127e5a76496463898be6b6, and in particular this
> section:
>
>               ;; If SOURCE is a directory (such as a checkout), return a
>               ;; directory.  Otherwise create a tarball.
>               (cond
>                ((file-is-directory? #+source)
>                 (copy-recursively directory #$output
>                                   #:log (%make-void-port "w")))
>                ((not #+comp)
>                 (copy-file file #$output)) <---------- We should call repack instead here for raw tarballs?
>                (else
>                 (repack directory #$output)))))))
>
> Maxim, could you please have a look?

Thanks for the report; it looks like a bug indeed.  I had probably
overlooked that case (where there is no compression but we are still
dealing with an archive format).  The annoying thing is that touching
this code will trigger a world rebuild; so unless we find other big
problems and batch the fixes, it may not be worth it.

The site.py problem you reported may be one, if I find a solution.

Other things on my mind:

1. rustc bootstrap from 1.39 (the itch to make it faster comes back
everytime I have to build it ;-)).

2. fontconfig update that should allow per-profile fonts management via
  XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
  having to build rust and getting distracted by 1. :-))

Not a very useful reply in itself, but I felt that communicating what
I've been busy with was better than a longer silence :-).

Thank you,

Maxim




Severity set to 'important' from 'normal' Request was from Maxim Cournoyer <maxim.cournoyer <at> gmail.com> to control <at> debbugs.gnu.org. (Fri, 27 Aug 2021 17:04:01 GMT) Full text and rfc822 format available.

Information forwarded to bug-guix <at> gnu.org:
bug#50066; Package guix. (Tue, 31 Aug 2021 03:51:02 GMT) Full text and rfc822 format available.

Message #13 received at 50066 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50066 <at> debbugs.gnu.org
Subject: Re: bug#50066: core-update: Raw origin tarballs are not handled
 correctly.
Date: Mon, 30 Aug 2021 23:50:20 -0400
Hello Mathieu,

[...]

> Thanks for the report; it looks like a bug indeed.  I had probably
> overlooked that case (where there is no compression but we are still
> dealing with an archive format).  The annoying thing is that touching
> this code will trigger a world rebuild; so unless we find other big
> problems and batch the fixes, it may not be worth it.
>
> The site.py problem you reported may be one, if I find a solution.
>
> Other things on my mind:
>
> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back
> everytime I have to build it ;-)).

I'm nearing completion on this front.

> 2. fontconfig update that should allow per-profile fonts management via
>   XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
>   having to build rust and getting distracted by 1. :-))

I'll propose a few patches to include in the 'frozen' core-updates
branch since we've found a world-rebuilding change worthy of addressing
anyway (the patch below).

1 file changed, 4 insertions(+), 4 deletions(-)
guix/packages.scm | 8 ++++----

modified   guix/packages.scm
@@ -836,10 +836,10 @@ specifies modules in scope when evaluating SNIPPET."
                ((file-is-directory? #+source)
                 (copy-recursively directory #$output
                                   #:log (%make-void-port "w")))
-               ((not #+comp)
-                (copy-file file #$output))
-               (else
-                (repack directory #$output)))))))
+               ((or #+comp (tarball? #+source))
+                (repack directory #$output))
+               (else                    ;single uncompressed file
+                (copy-file file #$output)))))))
 
     (let ((name (if (or (checkout? original-file-name)
                         (not (compressor original-file-name)))

Sadly the plain .tar file was/is not covered by tests (and not easily
so), but if his was my only omission, it should be correct (TM).
Attentive review needed :-).

Thank you!

Maxim




Added tag(s) patch. Request was from Maxim Cournoyer <maxim.cournoyer <at> gmail.com> to control <at> debbugs.gnu.org. (Tue, 31 Aug 2021 13:48:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-guix <at> gnu.org:
bug#50066; Package guix. (Wed, 01 Sep 2021 07:10:02 GMT) Full text and rfc822 format available.

Message #18 received at 50066 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 50066 <at> debbugs.gnu.org
Subject: Re: bug#50066: core-update: Raw origin tarballs are not handled
 correctly.
Date: Wed, 01 Sep 2021 09:09:22 +0200
Hello Maxim,

>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back
>> everytime I have to build it ;-)).
>
> I'm nearing completion on this front.

That will be a welcomed improvement!

>> 2. fontconfig update that should allow per-profile fonts management via
>>   XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
>>   having to build rust and getting distracted by 1. :-))
>
> I'll propose a few patches to include in the 'frozen' core-updates
> branch since we've found a world-rebuilding change worthy of addressing
> anyway (the patch below).

Great! As we are going for a full rebuild, maybe we should also consider
some of the Thiago fixes targeting core-updates-frozen such as
https://issues.guix.gnu.org/50239.

> Sadly the plain .tar file was/is not covered by tests (and not easily
> so), but if his was my only omission, it should be correct (TM).
> Attentive review needed :-).

This seems reasonable and I'm currently rebuilding the world to test
this fix and the Python fix. I should maybe apply your Rust bootstrap
patch if you have it around to gain some time.

Thanks for all those fixes!

Mathieu




Information forwarded to bug-guix <at> gnu.org:
bug#50066; Package guix. (Wed, 01 Sep 2021 12:58:01 GMT) Full text and rfc822 format available.

Message #21 received at 50066 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50066 <at> debbugs.gnu.org
Subject: Re: bug#50066: core-update: Raw origin tarballs are not handled
 correctly.
Date: Wed, 01 Sep 2021 08:57:24 -0400
Hi Mathieu,

Mathieu Othacehe <othacehe <at> gnu.org> writes:

> Hello Maxim,
>
>>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back
>>> everytime I have to build it ;-)).
>>
>> I'm nearing completion on this front.
>
> That will be a welcomed improvement!
>
>>> 2. fontconfig update that should allow per-profile fonts management via
>>>   XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
>>>   having to build rust and getting distracted by 1. :-))
>>
>> I'll propose a few patches to include in the 'frozen' core-updates
>> branch since we've found a world-rebuilding change worthy of addressing
>> anyway (the patch below).
>
> Great! As we are going for a full rebuild, maybe we should also consider
> some of the Thiago fixes targeting core-updates-frozen such as
> https://issues.guix.gnu.org/50239.

Yes, I think we should cherry-pick anything valuable before the big
rebuild of the frozen branch (perhaps we should simply sync with
core-updates?)

>> Sadly the plain .tar file was/is not covered by tests (and not easily
>> so), but if his was my only omission, it should be correct (TM).
>> Attentive review needed :-).
>
> This seems reasonable and I'm currently rebuilding the world to test
> this fix and the Python fix. I should maybe apply your Rust bootstrap
> patch if you have it around to gain some time.

It's currently building Rust 1.47 (it takes about 15 minutes per Rust).
I'll build it all until 1.54, then recheck a few ones for
reproducibility (at least the first and last ones).  If all is good I'll
send the patches I have.

> Thanks for all those fixes!

Thank you for the testing/reports!

Maxim




Information forwarded to bug-guix <at> gnu.org:
bug#50066; Package guix. (Fri, 03 Sep 2021 13:31:01 GMT) Full text and rfc822 format available.

Message #24 received at 50066 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50066 <at> debbugs.gnu.org
Subject: Re: bug#50066: core-update: Raw origin tarballs are not handled
 correctly.
Date: Fri, 03 Sep 2021 09:30:29 -0400
Hi Mathieu,

Mathieu Othacehe <othacehe <at> gnu.org> writes:

> Hello Maxim,
>
>>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back
>>> everytime I have to build it ;-)).
>>
>> I'm nearing completion on this front.
>
> That will be a welcomed improvement!
>
>>> 2. fontconfig update that should allow per-profile fonts management via
>>>   XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
>>>   having to build rust and getting distracted by 1. :-))
>>
>> I'll propose a few patches to include in the 'frozen' core-updates
>> branch since we've found a world-rebuilding change worthy of addressing
>> anyway (the patch below).
>
> Great! As we are going for a full rebuild, maybe we should also consider
> some of the Thiago fixes targeting core-updates-frozen such as
> https://issues.guix.gnu.org/50239.
>
>> Sadly the plain .tar file was/is not covered by tests (and not easily
>> so), but if his was my only omission, it should be correct (TM).
>> Attentive review needed :-).
>
> This seems reasonable and I'm currently rebuilding the world to test
> this fix and the Python fix. I should maybe apply your Rust bootstrap
> patch if you have it around to gain some time.

successfully built /gnu/store/vag2a53byhgrkpchviqz78lilall6412-emacs-org-contrib-20210519.drv
/gnu/store/40hl0dli745i2ggs1aqhn2drh0fkmrbx-emacs-org-contrib-20210519

:-).

I'm trying to fix one last thing with my local patch set (mozjs not
building with rust <at> 1.54), but if takes too much time, I'll send what I
have and we can figure it out together.

Thanks,

Maxim




Reply sent to Mathieu Othacehe <othacehe <at> gnu.org>:
You have taken responsibility. (Sun, 19 Dec 2021 16:42:02 GMT) Full text and rfc822 format available.

Notification sent to Mathieu Othacehe <othacehe <at> gnu.org>:
bug acknowledged by developer. (Sun, 19 Dec 2021 16:42:02 GMT) Full text and rfc822 format available.

Message #29 received at 50066-done <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 50066-done <at> debbugs.gnu.org
Subject: Re: bug#50066: core-update: Raw origin tarballs are not handled
 correctly.
Date: Sun, 19 Dec 2021 17:40:58 +0100
Hey Maxim,

> successfully built /gnu/store/vag2a53byhgrkpchviqz78lilall6412-emacs-org-contrib-20210519.drv
> /gnu/store/40hl0dli745i2ggs1aqhn2drh0fkmrbx-emacs-org-contrib-20210519

Thanks for fixing this one,

Closing,

Mathieu




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 17 Jan 2022 12:24:10 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 71 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.