GNU bug report logs -
#25752
go incremental builds broken
Previous Next
To reply to this bug, email your comments to 25752 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Thu, 16 Feb 2017 15:06:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Hank Donnay <hdonnay <at> gmail.com>
:
New bug report received and forwarded. Copy sent to
bug-guix <at> gnu.org
.
(Thu, 16 Feb 2017 15:06:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Hello,
It seems the guix's go package is broken when the go tool is used for
incremental builds. Any attempt to use 'install' or 'build -i' results
in an attempt to write to the store. A one-liner:
guix environment --ad-hoc go -- bash -c 'export t=$(mktemp -d); cd
$t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go install cmd/go; cd
&& rm -rf $t'
Another command reports that (seemingly) the entire stdlib is marked as stale:
guix environment --ad-hoc go -- bash -c 'export t=$(mktemp -d); cd
$t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go list -f '\''{{join
.Deps "\n"}}'\'' cmd/go | xargs -n1 go list -f '\''{{if
.Stale}}{{.ImportPath}}: {{.StaleReason}}{{end}}'\''; cd && rm -rf $t'
The function for determining staleness is here (after the giant
comment explaining the reasoning):
https://golang.org/src/cmd/go/pkg.go#L1111
I don't see anything wrong with the package definition, but could be
missing something. My only hunch at this point is that something might
be modifying src/runtime/internal/sys/zversion.go, as that entire file
is included in the build ID computation.
Thanks,
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Fri, 24 Feb 2017 19:51:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 25752 <at> debbugs.gnu.org (full text, mbox):
After talking on IRC, it seems like grafts are the problem. Adding
`--no-grafts` to the above commands has them succeed.
The GOROOT (store location, in this case) gets encoded in the
generated source by the 'dist' tool, so changing that is going to
cause issues.
Is there a way to disable grafts on a per-package basis, in the definition?
On Thu, Feb 16, 2017 at 10:06 AM, GNU bug Tracking System
<help-debbugs <at> gnu.org> wrote:
> Thank you for filing a new bug report with debbugs.gnu.org.
>
> This is an automatically generated reply to let you know your message
> has been received.
>
> Your message is being forwarded to the package maintainers and other
> interested parties for their attention; they will reply in due course.
>
> Your message has been sent to the package maintainer(s):
> bug-guix <at> gnu.org
>
> If you wish to submit further information on this problem, please
> send it to 25752 <at> debbugs.gnu.org.
>
> Please do not send mail to help-debbugs <at> gnu.org unless you wish
> to report a problem with the Bug-tracking system.
>
> --
> 25752: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=25752
> GNU Bug Tracking System
> Contact help-debbugs <at> gnu.org with problems
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Sat, 25 Feb 2017 15:59:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 25752 <at> debbugs.gnu.org (full text, mbox):
On Fri, Feb 24, 2017 at 02:50:10PM -0500, Hank Donnay wrote:
> After talking on IRC, it seems like grafts are the problem. Adding
> `--no-grafts` to the above commands has them succeed.
>
> The GOROOT (store location, in this case) gets encoded in the
> generated source by the 'dist' tool, so changing that is going to
> cause issues.
>
> Is there a way to disable grafts on a per-package basis, in the definition?
You can disable them on the command-line for any tool that builds
things, using the '--no-grafts' option, which is a "Common build option"
[0]. For example, `guix environment --ad-hoc go --no-grafts`.
If you do this, you should understand that the ungrafted Go package will
contain some serious security vulnerabilities, since we only use grafts
to fix vulnerabilities in core packages. [1]
We should figure out why this issue manifests when go is grafted, and
fix the root cause.
[0]
https://www.gnu.org/software/guix/manual/html_node/Common-Build-Options.html
[1]
https://www.gnu.org/software/guix/manual/html_node/Security-Updates.html
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Tue, 07 Mar 2017 21:51:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 25752 <at> debbugs.gnu.org (full text, mbox):
Hello,
Hank Donnay <hdonnay <at> gmail.com> skribis:
> The function for determining staleness is here (after the giant
> comment explaining the reasoning):
> https://golang.org/src/cmd/go/pkg.go#L1111
This method relies on the build ID to, which is defined like this (info
"(ld) Options"):
`--build-id'
`--build-id=STYLE'
Request the creation of a `.note.gnu.build-id' ELF note section or
a `.buildid' COFF section. The contents of the note are unique
bits identifying this linked file. STYLE can be `uuid' to use 128
random bits, `sha1' to use a 160-bit SHA1 hash on the normative
parts of the output contents, `md5' to use a 128-bit MD5 hash on
the normative parts of the output contents, or `0xHEXSTRING' to
use a chosen bit string specified as an even number of hexadecimal
digits (`-' and `:' characters between digit pairs are ignored).
If STYLE is omitted, `sha1' is used.
The `md5' and `sha1' styles produces an identifier that is always
the same in an identical output file, but will be unique among all
nonidentical output files. It is not intended to be compared as a
checksum for the file's contents. A linked file may be changed
later by other tools, but the build ID bit string identifying the
original linked file does not change.
Passing `none' for STYLE disables the setting from any
`--build-id' options earlier on the command line.
I suppose Go uses one of md5 or sha1, which is a good thing since it
allows for reproducible builds.
However, grafting breaks this, similarly to <https://bugs.gnu.org/19973>
since they change file contents without recomputing the build ID.
Having Go use --build-id=uuid would work around the problem, but it
would also prevent bit-reproducible builds.
Perhaps our grafting code will have to handle .note.gnu.build-id
specially.
Thoughts?
Thanks for reporting the issue,
Ludo’.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Fri, 18 Dec 2020 20:04:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 25752 <at> debbugs.gnu.org (full text, mbox):
Hi,
On Sat, 25 Feb 2017 at 10:58, Leo Famulari <leo <at> famulari.name> wrote:
> On Fri, Feb 24, 2017 at 02:50:10PM -0500, Hank Donnay wrote:
>> After talking on IRC, it seems like grafts are the problem. Adding
>> `--no-grafts` to the above commands has them succeed.
>>
>> The GOROOT (store location, in this case) gets encoded in the
>> generated source by the 'dist' tool, so changing that is going to
>> cause issues.
>>
>> Is there a way to disable grafts on a per-package basis, in the definition?
>
> You can disable them on the command-line for any tool that builds
> things, using the '--no-grafts' option, which is a "Common build option"
> [0]. For example, `guix environment --ad-hoc go --no-grafts`.
>
> If you do this, you should understand that the ungrafted Go package will
> contain some serious security vulnerabilities, since we only use grafts
> to fix vulnerabilities in core packages. [1]
>
> We should figure out why this issue manifests when go is grafted, and
> fix the root cause.
I suspect closing this bug had been forgotten. If yes, feel free to
close it. Otherwise, I do not understand what is the next actionable
step.
<http://issues.guix.gnu.org/issue/25752>
All the best,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Fri, 18 Dec 2020 21:32:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 25752 <at> debbugs.gnu.org (full text, mbox):
On Fri, Dec 18, 2020 at 09:00:14PM +0100, zimoun wrote:
> I suspect closing this bug had been forgotten. If yes, feel free to
> close it. Otherwise, I do not understand what is the next actionable
> step.
I don't think this bug has been fixed, although I have not checked.
Currently, Go incremental builds and library re-use is not functional in
Guix, unless I missed a new development.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Mon, 11 Jan 2021 12:57:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 25752 <at> debbugs.gnu.org (full text, mbox):
Hi Leo,
On Fri, 18 Dec 2020 at 16:31, Leo Famulari <leo <at> famulari.name> wrote:
> I don't think this bug has been fixed, although I have not checked.
> Currently, Go incremental builds and library re-use is not functional in
> Guix, unless I missed a new development.
Thanks. Could you provide a Go example for reproducing? Well, a
reproducer to track what could be wrong on the grafting mechanism for
Go. If I understand correctly.
All the best,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Mon, 11 Jan 2021 20:03:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 25752 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Mon, Jan 11, 2021 at 01:46:48PM +0100, zimoun wrote:
> Hi Leo,
>
> On Fri, 18 Dec 2020 at 16:31, Leo Famulari <leo <at> famulari.name> wrote:
>
> > I don't think this bug has been fixed, although I have not checked.
> > Currently, Go incremental builds and library re-use is not functional in
> > Guix, unless I missed a new development.
>
> Thanks. Could you provide a Go example for reproducing? Well, a
> reproducer to track what could be wrong on the grafting mechanism for
> Go. If I understand correctly.
>
I actually spent a bit of time on this today/yesterday. I modified the
go-build-system to copy the pkg and src folders from build to build
(instead of just the src folders) and the pkg folders were ignored.
Upstream documentation isn't clear about when the pkg folder is used and
when it is ignored.
One thing I did try was to set the pkgdir to (string-append (getenv
"GOROOT") "/pkg"), but then it didn't find the pkg folder from go:out. I
didn't try copying go:out/pkg to GOROOT/pkg to see if that would fix it;
I assumed that even if it did work we wouldn't want to propagate all of
go with the go packages.
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Wed, 09 Jun 2021 21:44:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 25752 <at> debbugs.gnu.org (full text, mbox):
Hi,
On Mon, 11 Jan 2021 at 22:01, Efraim Flashner <efraim <at> flashner.co.il> wrote:
> On Mon, Jan 11, 2021 at 01:46:48PM +0100, zimoun wrote:
>> On Fri, 18 Dec 2020 at 16:31, Leo Famulari <leo <at> famulari.name> wrote:
>>
>> > I don't think this bug has been fixed, although I have not checked.
>> > Currently, Go incremental builds and library re-use is not functional in
>> > Guix, unless I missed a new development.
>>
>> Thanks. Could you provide a Go example for reproducing? Well, a
>> reproducer to track what could be wrong on the grafting mechanism for
>> Go. If I understand correctly.
>>
>
> I actually spent a bit of time on this today/yesterday. I modified the
> go-build-system to copy the pkg and src folders from build to build
> (instead of just the src folders) and the pkg folders were ignored.
> Upstream documentation isn't clear about when the pkg folder is used and
> when it is ignored.
>
> One thing I did try was to set the pkgdir to (string-append (getenv
> "GOROOT") "/pkg"), but then it didn't find the pkg folder from go:out. I
> didn't try copying go:out/pkg to GOROOT/pkg to see if that would fix it;
> I assumed that even if it did work we wouldn't want to propagate all of
> go with the go packages.
What is the status of this bug? Since I am not using Go at all, it is
hard for me to have a reproducer. Is it possible to provide a
reproducer? Thanks.
Cheers,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Tue, 14 Sep 2021 12:47:05 GMT)
Full text and
rfc822 format available.
Message #32 received at 25752 <at> debbugs.gnu.org (full text, mbox):
Hi,
On Fri, 18 Dec 2020 at 16:31, Leo Famulari <leo <at> famulari.name> wrote:
> On Fri, Dec 18, 2020 at 09:00:14PM +0100, zimoun wrote:
>> I suspect closing this bug had been forgotten. If yes, feel free to
>> close it. Otherwise, I do not understand what is the next actionable
>> step.
>
> I don't think this bug has been fixed, although I have not checked.
> Currently, Go incremental builds and library re-use is not functional in
> Guix, unless I missed a new development.
What is the status of this bug? Since I am not using Go at all, it is
hard for me to have a reproducer. Is it possible to provide a
reproducer? Thanks.
Cheers,
simon
PS: I asked the same thing 13 weeks ago. Therefore, without any answer
and giving this bug is really old not so much commented, I propose to
close it.
Added tag(s) moreinfo.
Request was from
zimoun <zimon.toutoune <at> gmail.com>
to
control <at> debbugs.gnu.org
.
(Tue, 14 Sep 2021 12:47:06 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Fri, 26 Nov 2021 01:22:02 GMT)
Full text and
rfc822 format available.
Message #37 received at 25752 <at> debbugs.gnu.org (full text, mbox):
Hi,
On Mon, 11 Jan 2021 at 22:01, Efraim Flashner <efraim <at> flashner.co.il> wrote:
> On Mon, Jan 11, 2021 at 01:46:48PM +0100, zimoun wrote:
>> On Fri, 18 Dec 2020 at 16:31, Leo Famulari <leo <at> famulari.name> wrote:
>>
>> > I don't think this bug has been fixed, although I have not checked.
>> > Currently, Go incremental builds and library re-use is not functional in
>> > Guix, unless I missed a new development.
>>
>> Thanks. Could you provide a Go example for reproducing? Well, a
>> reproducer to track what could be wrong on the grafting mechanism for
>> Go. If I understand correctly.
>>
>
> I actually spent a bit of time on this today/yesterday. I modified the
> go-build-system to copy the pkg and src folders from build to build
> (instead of just the src folders) and the pkg folders were ignored.
> Upstream documentation isn't clear about when the pkg folder is used and
> when it is ignored.
>
> One thing I did try was to set the pkgdir to (string-append (getenv
> "GOROOT") "/pkg"), but then it didn't find the pkg folder from go:out. I
> didn't try copying go:out/pkg to GOROOT/pkg to see if that would fix it;
> I assumed that even if it did work we wouldn't want to propagate all of
> go with the go packages.
What is the status of this old bug [1]? It is marked ’moreinfo’ because
it misses a reproducer to be able to act on. Therefore, I am going to
close it.
Cheers,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Wed, 26 Oct 2022 22:54:02 GMT)
Full text and
rfc822 format available.
Message #40 received at 25752 <at> debbugs.gnu.org (full text, mbox):
Hi,
On Tue, 14 Sep 2021 at 12:43, zimoun <zimon.toutoune <at> gmail.com> wrote:
> On Fri, 18 Dec 2020 at 16:31, Leo Famulari <leo <at> famulari.name> wrote:
>> On Fri, Dec 18, 2020 at 09:00:14PM +0100, zimoun wrote:
>>> I suspect closing this bug had been forgotten. If yes, feel free to
>>> close it. Otherwise, I do not understand what is the next actionable
>>> step.
>>
>> I don't think this bug has been fixed, although I have not checked.
>> Currently, Go incremental builds and library re-use is not functional in
>> Guix, unless I missed a new development.
>
> What is the status of this bug? Since I am not using Go at all, it is
> hard for me to have a reproducer. Is it possible to provide a
> reproducer? Thanks.
> PS: I asked the same thing 13 weeks ago. Therefore, without any answer
> and giving this bug is really old not so much commented, I propose to
> close it.
Date: Wed, 09 Jun 2021 23:38:50 +0200 (1 year, 19 weeks, 5 days ago)
Date: Fri, 26 Nov 2021 02:12:36 +0100 (47 weeks, 5 days, 19 hours ago)
Go incremental is still not functional but is it worth to keep this
report open?
Giving the none activity of this bug and no reply about the status after
waiting a long time, I will close it if no objection.
Cheers,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Thu, 27 Oct 2022 10:22:01 GMT)
Full text and
rfc822 format available.
Message #43 received at 25752 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 26-10-2022 22:42, zimoun wrote:
> Hi,
>
> On Tue, 14 Sep 2021 at 12:43, zimoun <zimon.toutoune <at> gmail.com> wrote:
>> On Fri, 18 Dec 2020 at 16:31, Leo Famulari <leo <at> famulari.name> wrote:
>>> On Fri, Dec 18, 2020 at 09:00:14PM +0100, zimoun wrote:
>>>> I suspect closing this bug had been forgotten. If yes, feel free to
>>>> close it. Otherwise, I do not understand what is the next actionable
>>>> step.
>>>
>>> I don't think this bug has been fixed, although I have not checked.
>>> Currently, Go incremental builds and library re-use is not functional in
>>> Guix, unless I missed a new development.
>>
>> What is the status of this bug? Since I am not using Go at all, it is
>> hard for me to have a reproducer. Is it possible to provide a
>> reproducer? Thanks. [...]
A reproducer was given in
<https://issues.guix.gnu.org/25752#0>:
guix environment --ad-hoc go -- bash -c 'export t=$(mktemp -d); cd
$t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go install cmd/go; cd
&& rm -rf $t'
guix environment --ad-hoc go -- bash -c 'export t=$(mktemp -d); cd
$t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go list -f '\''{{join
.Deps "\n"}}'\'' cmd/go | xargs -n1 go list -f '\''{{if
.Stale}}{{.ImportPath}}: {{.StaleReason}}{{end}}'\''; cd && rm -rf $t'
I don't know if this reproducer still works/fails.
Greetings,
Maxime.
[OpenPGP_0x49E3EE22191725EE.asc (application/pgp-keys, attachment)]
[OpenPGP_signature (application/pgp-signature, attachment)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Fri, 28 Oct 2022 08:48:02 GMT)
Full text and
rfc822 format available.
Message #46 received at 25752 <at> debbugs.gnu.org (full text, mbox):
Hi,
On Thu, 27 Oct 2022 at 12:21, Maxime Devos <maximedevos <at> telenet.be> wrote:
> A reproducer was given in
> <https://issues.guix.gnu.org/25752#0>:
>
> guix environment --ad-hoc go -- bash -c 'export t=$(mktemp -d); cd
> $t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go install cmd/go; cd
> && rm -rf $t'
--8<---------------cut here---------------start------------->8---
$ guix shell -C go bash coreutils -- bash -c 'export t=$(mktemp -d); cd $t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go install cmd/go; cd && echo $t'
[...]
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH
/tmp/tmp.srzPbcMLCc
$ guix shell -C go bash coreutils gcc-toolchain -- bash -c 'export t=$(mktemp -d); cd $t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go install cmd/go; cd && echo $t'
[...]
go install cmd/go: copying /tmp/go-build1675975230/b001/exe/a.out: open /gnu/store/82adn4rx1h5fc7zms5g0ymwr8dk060nh-go-1.19.1/lib/go/bin/go: read-only file system
/tmp/tmp.qOfxzeZDxH
--8<---------------cut here---------------end--------------->8---
Well, as I am not a Go user, what is the expected result of the command?
Efraim commented:
I actually spent a bit of time on this today/yesterday. I
modified the go-build-system to copy the pkg and src folders
from build to build (instead of just the src folders) and the
pkg folders were ignored. Upstream documentation isn't clear
about when the pkg folder is used and when it is ignored.
<https://issues.guix.gnu.org/25752#7>
Cheers,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#25752
; Package
guix
.
(Tue, 17 Oct 2023 07:47:01 GMT)
Full text and
rfc822 format available.
Message #49 received at 25752 <at> debbugs.gnu.org (full text, mbox):
Hi Go team,
I was reviewing the status of old bugs and I find bug#25752
https://issues.guix.gnu.org/issue/25752
Since I am not a Go user, is it possible that Go team says if this bug
is still worth keeping open?
On Fri, 28 Oct 2022 at 10:22, zimoun <zimon.toutoune <at> gmail.com> wrote:
> On Thu, 27 Oct 2022 at 12:21, Maxime Devos <maximedevos <at> telenet.be> wrote:
>
>> A reproducer was given in
>> <https://issues.guix.gnu.org/25752#0>:
>>
>> guix environment --ad-hoc go -- bash -c 'export t=$(mktemp -d); cd
>> $t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go install cmd/go; cd
>> && rm -rf $t'
>
> $ guix shell -C go bash coreutils -- bash -c 'export t=$(mktemp -d); cd $t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go install cmd/go; cd && echo $t'
> [...]
>
> # runtime/cgo
> cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH
> /tmp/tmp.srzPbcMLCc
>
> $ guix shell -C go bash coreutils gcc-toolchain -- bash -c 'export t=$(mktemp -d); cd $t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go install cmd/go; cd && echo $t'
> [...]
>
> go install cmd/go: copying /tmp/go-build1675975230/b001/exe/a.out: open /gnu/store/82adn4rx1h5fc7zms5g0ymwr8dk060nh-go-1.19.1/lib/go/bin/go: read-only file system
> /tmp/tmp.qOfxzeZDxH
>
> Well, as I am not a Go user, what is the expected result of the command?
>
>
> Efraim commented:
>
> I actually spent a bit of time on this today/yesterday. I
> modified the go-build-system to copy the pkg and src folders
> from build to build (instead of just the src folders) and the
> pkg folders were ignored. Upstream documentation isn't clear
> about when the pkg folder is used and when it is ignored.
>
> <https://issues.guix.gnu.org/25752#7>
Well, as I previously mentioned, I do not know what is the expected
result of the reproduced. :-)
Cheers,
simon
This bug report was last modified 1 year and 30 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.