GNU bug report logs - #16364
auto-compile noise can't be avoided by script

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guile; Reported by: Zefram <zefram@HIDDEN>; dated Sun, 5 Jan 2014 23:45:16 UTC; Maintainer for guile is bug-guile@HIDDEN.

Message received at 16364 <at> debbugs.gnu.org:


Received: (at 16364) by debbugs.gnu.org; 17 Jan 2014 21:56:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 17 16:56:17 2014
Received: from localhost ([127.0.0.1]:55599 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1W4HOi-0004xm-Rm
	for submit <at> debbugs.gnu.org; Fri, 17 Jan 2014 16:56:17 -0500
Received: from river.fysh.org ([5.135.154.127]:34418)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <zefram@HIDDEN>) id 1W4HOf-0004xc-C7
 for 16364 <at> debbugs.gnu.org; Fri, 17 Jan 2014 16:56:14 -0500
Received: from zefram by river.fysh.org with local (Exim 4.80 #2 (Debian))
 id 1W4HOb-0008RN-Ju; Fri, 17 Jan 2014 21:56:09 +0000
Date: Fri, 17 Jan 2014 21:56:09 +0000
From: Zefram <zefram@HIDDEN>
To: Ludovic Courtes <ludo@HIDDEN>
Subject: Re: bug#16364: auto-compile noise can't be avoided by script
Message-ID: <20140117215609.GH21945@HIDDEN>
References: <20140105234106.GI30283@HIDDEN>
 <87fvomuwfw.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87fvomuwfw.fsf@HIDDEN>
X-Spam-Score: -0.3 (/)
X-Debbugs-Envelope-To: 16364
Cc: 16364 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.3 (/)

Ludovic Courtes wrote:
>However, you can set the environment variable GUILE_AUTO_COMPILE=0.
>
>Do you think that would solve the problem?

It does not solve the problem.  Firstly, it can't be done from the
#! line at all, so the script can't do it early enough.  It only works
if it's already been set by the user, which is no good for what should
be an internal detail of the program.  Secondly, it suffers the second
problem that I noted with --no-auto-compile: if there's already a cached
compilation then that'll be looked at, and if it's out of date then a
"newer than" banner is emitted.  With the environment variable set the
cached version will never be updated, nor will it be deleted, so the
banner then appears on every execution.

-zefram




Information forwarded to bug-guile@HIDDEN:
bug#16364; Package guile. Full text available.

Message received at 16364 <at> debbugs.gnu.org:


Received: (at 16364) by debbugs.gnu.org; 17 Jan 2014 21:31:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 17 16:31:53 2014
Received: from localhost ([127.0.0.1]:55589 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1W4H16-0004E9-I7
	for submit <at> debbugs.gnu.org; Fri, 17 Jan 2014 16:31:52 -0500
Received: from hera.aquilenet.fr ([141.255.128.1]:34994)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <ludo@HIDDEN>) id 1W4H14-0004Ds-4r
 for 16364 <at> debbugs.gnu.org; Fri, 17 Jan 2014 16:31:51 -0500
Received: from localhost (localhost [127.0.0.1])
 by hera.aquilenet.fr (Postfix) with ESMTP id 39A6D1B58;
 Fri, 17 Jan 2014 22:31:48 +0100 (CET)
Received: from hera.aquilenet.fr ([127.0.0.1])
 by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id Kfv+S7lMK1bD; Fri, 17 Jan 2014 22:31:48 +0100 (CET)
Received: from pluto (reverse-83.fdn.fr [80.67.176.83])
 by hera.aquilenet.fr (Postfix) with ESMTPSA id D9E90A49;
 Fri, 17 Jan 2014 22:31:47 +0100 (CET)
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
To: Zefram <zefram@HIDDEN>
Subject: Re: bug#16364: auto-compile noise can't be avoided by script
References: <20140105234106.GI30283@HIDDEN>
Date: Fri, 17 Jan 2014 22:31:47 +0100
In-Reply-To: <20140105234106.GI30283@HIDDEN> (zefram@HIDDEN's message of
 "Sun, 5 Jan 2014 23:41:06 +0000")
Message-ID: <87fvomuwfw.fsf@HIDDEN>
User-Agent: Gnus/5.130007 (Ma Gnus v0.7) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 16364
Cc: 16364 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

Zefram <zefram@HIDDEN> skribis:

> I can turn off the auto-compilation from within the script by using the
> --no-auto-compile option, but that breaks compatibility to 1.8:

However, you can set the environment variable GUILE_AUTO_COMPILE=3D0.

Do you think that would solve the problem?

Thanks,
Ludo=E2=80=99.




Information forwarded to bug-guile@HIDDEN:
bug#16364; Package guile. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 5 Jan 2014 23:44:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jan 05 18:44:27 2014
Received: from localhost ([127.0.0.1]:37213 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1VzxMo-0004zf-An
	for submit <at> debbugs.gnu.org; Sun, 05 Jan 2014 18:44:27 -0500
Received: from eggs.gnu.org ([208.118.235.92]:49957)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <zefram@HIDDEN>) id 1VzxJm-0004tX-OM
 for submit <at> debbugs.gnu.org; Sun, 05 Jan 2014 18:41:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <zefram@HIDDEN>) id 1VzxJl-0001Uz-5B
 for submit <at> debbugs.gnu.org; Sun, 05 Jan 2014 18:41:18 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
 version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:34448)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <zefram@HIDDEN>) id 1VzxJl-0001Uv-24
 for submit <at> debbugs.gnu.org; Sun, 05 Jan 2014 18:41:17 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:51679)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <zefram@HIDDEN>) id 1VzxJj-0004mh-GF
 for bug-guile@HIDDEN; Sun, 05 Jan 2014 18:41:16 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <zefram@HIDDEN>) id 1VzxJi-0001Uh-5W
 for bug-guile@HIDDEN; Sun, 05 Jan 2014 18:41:15 -0500
Received: from river.fysh.org ([2001:41d0:8:d47f::2]:52450)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <zefram@HIDDEN>) id 1VzxJh-0001Ud-SE
 for bug-guile@HIDDEN; Sun, 05 Jan 2014 18:41:14 -0500
Received: from zefram by river.fysh.org with local (Exim 4.80 #2 (Debian))
 id 1VzxJa-0001ZM-T3; Sun, 05 Jan 2014 23:41:06 +0000
Date: Sun, 5 Jan 2014 23:41:06 +0000
From: Zefram <zefram@HIDDEN>
To: bug-guile@HIDDEN
Subject: auto-compile noise can't be avoided by script
Message-ID: <20140105234106.GI30283@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
 (bad octet value).
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
 (bad octet value).
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.0 (----)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Sun, 05 Jan 2014 18:44:19 -0500
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.0 (----)

Guile 2.0.9 has a facility to automatically cache a compiled version
of any Scheme source file that it loads, and it wants the world to
know about it!  If auto-compilation is enabled, which it is by default,
then when guile loads a file (that was not already compiled) it emits a
banner describing the auto-compilation.  This interferes with the proper
functionality of any program written as a guile script, by producing
output that the program did not intend.  Working around this is tricky
(discussed below).  There's no straightforward way for a script to avoid
the noise while being portable between guile versions 1.8 and 2.0.
There's also no way to avoid the noise while actually getting the
auto-compilation behaviour.

In my particular case, my script makes interesting use of the
read-eval (#.) feature, which means that the compilation process
actually can't work.  This means that *every* time the script is run,
not just the first time, guile emits the banner about auto-compilation,
followed by a rather misleading warning/error about compilation failure.
It's misleading because it then goes on to execute the script just fine.
I can demonstrate this with a minimal test case (using read-eval in an
uninteresting way, just making the compiler barf by not having applied
eval-when to enable it):

$ cat t0
#!/usr/bin/guile -s
!#
(fluid-set! read-eval? #t)
(display #."hello world")
(newline)
$ guile-1.8 -s t0
hello world
$ guile-2.0 -s t0
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/zefram/usr/guile/t0
;;; WARNING: compilation of /home/zefram/usr/guile/t0 failed:
;;; ERROR: #. read expansion found and read-eval? is #f.
hello world
$

I can turn off the auto-compilation from within the script by using the
--no-auto-compile option, but that breaks compatibility to 1.8:

$ cat t1
#!/usr/bin/guile \
--no-auto-compile -s
!#
(fluid-set! read-eval? #t)
(display #."hello world")
(newline)
$ guile-2.0 '\' t1
hello world
$ guile-1.8 '\' t1
guile-1.8: Unrecognized switch `--no-auto-compile'
Usage: guile-1.8 OPTION ...
Evaluate Scheme code, interactively or from a script.
...

Aside from the portability concern, turning off auto-compilation doesn't
actually fix the problem.  If a compiled version has previously been
cached for the filename of a script being run, guile will consider
using the cached version even if --no-auto-compile was supplied: the
switch only controls the attempt to compile for the cache.  If the
cached compilation is up to date then it is used silently, which is OK.
But if it's out of date, because the cache was for a different script
that previously existed under the same name, then guile emits a banner
saying that it's out of date (implying that the cached compilation is
therefore not being used).  So the script's visible behaviour is defiled
even if it applies the option.

Observe what happens to the second script in this sequence:

$ echo '(display "hello world\n")' >t10
$ guile-2.0 t10
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/zefram/usr/guile/t10
;;; compiled /home/zefram/.cache/guile/ccache/2.0-LE-8-2.0/home/zefram/usr/guile/t10.go
hello world
$ echo '(display "goodbye world\n")' >t10
$ guile-2.0 --no-auto-compile t10        
;;; note: source file /home/zefram/usr/guile/t10
;;;       newer than compiled /home/zefram/.cache/guile/ccache/2.0-LE-8-2.0/home/zefram/usr/guile/t10.go
goodbye world

I have, however, come up with a truly ugly workaround.  The meta option
system can be used to introduce a -c option that explicitly loads the
script file via primitive-eval, which does not attempt compilation.
(Nor does it look at the compilation cache, so this even avoids the
problem that --no-auto-compile runs into.)  Running the script this way
yields a different command line (visible through (program-arguments))
from that which arrives when the script is run via -s, so if the script
is to process its command line, for robustness it must pay attention to
which way it was invoked.  All together, this looks like:

$ cat t11
#!/usr/bin/guile \
-c (begin\
\ \ \ (define\ arg-hack\ #t)\
\ \ \ (primitive-load\ (cadr\ (program-arguments))))
!#
(define argv
  (if (false-if-exception arg-hack)
    (cdr (program-arguments))
    (program-arguments)))
(write argv)
(newline)
$ guile-1.6 '\' t11 a b c
("t11" "a" "b" "c")
$ guile-1.6 -s t11 a b c 
("t11" "a" "b" "c")
$ guile-1.8 '\' t11 a b c
("t11" "a" "b" "c")
$ guile-1.8 -s t11 a b c 
("t11" "a" "b" "c")
$ guile-2.0 '\' t11 a b c
("t11" "a" "b" "c")
$ guile-2.0 -s t11 a b c 
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/zefram/usr/guile/t11
;;; /home/zefram/usr/guile/t11:7:6: warning: possibly unbound variable `arg-hack'
;;; compiled /home/zefram/.cache/guile/ccache/2.0-LE-8-2.0/home/zefram/usr/guile/t11.go
("t11" "a" "b" "c")
$ guile-2.0 -s t11 a b c
("t11" "a" "b" "c")

I'm not comfortable with this as a workaround.  It smells fragile.
Also note that though this does avoid the banner appearing for
#!-based executions, it's not muffling the banner per se but actually
preventing compilation.  While for some programs it's desirable to
prevent compilation per se (because of the compiler's limitations),
there are plenty of programs that would like to be compiled and only
want to muffle the banner.  Losing the efficiency of compilation is
potentially a high price to pay for clean output.

Guile should not be emitting this banner by default.  It's really not
acceptable to damage the visible behaviour of a program that worked
fine on older guile versions.  It also, for this auto-compilation to
serve as the invisible optimising cache as which it's intended, ought
to keep quiet about compilation failure: the fallback to interpreting
the script should be silent.

Debian incarnation of this bug report:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=734009

-zefram




Acknowledgement sent to Zefram <zefram@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-guile@HIDDEN. Full text available.
Report forwarded to bug-guile@HIDDEN:
bug#16364; Package guile. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Fri, 31 Oct 2014 17:00:04 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.