GNU bug report logs - #11419
Get rid of ylwrap, and simplify yacc/lex rules (was: Re: FYI: maint: fix the generation of the synclines for bison's parser)

Previous Next

Package: automake;

Reported by: Stefano Lattarini <stefano.lattarini <at> gmail.com>

Date: Sun, 6 May 2012 10:41:02 UTC

Severity: wishlist

Tags: wontfix

Done: Stefano Lattarini <stefano.lattarini <at> gmail.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 11419 in the body.
You can then email your comments to 11419 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Sun, 06 May 2012 10:41:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Stefano Lattarini <stefano.lattarini <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-automake <at> gnu.org. (Sun, 06 May 2012 10:41:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Akim Demaille <akim.demaille <at> gmail.com>
Cc: bug-automake <at> gnu.org, Bison Patches <bison-patches <at> gnu.org>
Subject: Get rid of ylwrap, and simplify yacc/lex rules (was: Re: FYI: maint:
	fix the generation of the synclines for bison's parser)
Date: Sun, 06 May 2012 12:38:09 +0200
Severity: wishlist

[Adding bug-automake in CC:]

Hi Akim.

On 05/06/2012 11:20 AM, Akim Demaille wrote:
> This is taken from master.  I work with a VPATH build, and the
> generated synclines are very wrong.  Probably something that
> ylwrap should handle better btw.
>
Or maybe we could get rid of ylwrap and assume yacc/lex support the
'-o' option.  After all:

  - the yacc- and lex-generated files are automatically distributed,
    so only the maintainers should be expected to regenerate them;

  - the sytem yacc on OpenBSD and NetBSD (not surre yet about FreeBSD,
    will check) support the '-o' option as well; and their lex too,
    since its flex

  - supporting proprietary yacc and lex should be low-priority IMHO.

Might this be something to consider for Automake 1.13?

Regards,
  Stefano




Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 08 May 2012 10:52:02 GMT) Full text and rfc822 format available.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Akim Demaille <akim.demaille <at> gmail.com>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: bug-automake <at> gnu.org, Bison Patches <bison-patches <at> gnu.org>
Subject: Re: Get rid of ylwrap,
	and simplify yacc/lex rules (was: Re: FYI: maint: fix the generation
	of the synclines for bison's parser)
Date: Tue, 8 May 2012 12:48:24 +0200
Le 6 mai 2012 à 12:38, Stefano Lattarini a écrit :

> Severity: wishlist
> 
> [Adding bug-automake in CC:]
> 
> Hi Akim.
> 
> On 05/06/2012 11:20 AM, Akim Demaille wrote:
>> This is taken from master.  I work with a VPATH build, and the
>> generated synclines are very wrong.  Probably something that
>> ylwrap should handle better btw.

On second thought, this is unfair to ylwrap.  The issue is
rather that bison issues synclines which are relative the
cwd instead of relative to the path of the output file.

/tmp $ echo "%% exp: 'a' { action(); };" > in/in.y
/tmp $ bison in/in.y -o out/out.c
/tmp $ grep '^#line' out/out.c
#line 70 "out/out.c"
#line 105 "out/out.c"
#line 1 "in/in.y"
#line 1310 "out/out.c"
#line 1315 "out/out.c"

Both "out/out.c" and "in/in.y" are wrong.  They should be
"out.c" and "../in/in.y".

This is why bison gets (in my VPATH-build where builddir is two
levels down inside srcdir):

#line 277 "parse-gram.y"
#line 2271 "../../../src/parse-gram.c"

with a properly computed relative path, it should be
"parse-gram.c".  This is what the patch was fixing.

This is painful and regularly results in Emacs not being able
to find the file where the error occurred.

This should be fixed before 2.5.1 imho.  I'll see if gnulib
offers something to compute relative paths.

> Or maybe we could get rid of ylwrap and assume yacc/lex support the
> '-o' option.  After all:
> 
>  - the yacc- and lex-generated files are automatically distributed,
>    so only the maintainers should be expected to regenerate them;
> 
>  - the sytem yacc on OpenBSD and NetBSD (not surre yet about FreeBSD,
>    will check) support the '-o' option as well; and their lex too,
>    since its flex
> 
>  - supporting proprietary yacc and lex should be low-priority IMHO.
> 
> Might this be something to consider for Automake 1.13?

I have no strong opinion about this.  But I like simplifications :)



Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 08 May 2012 11:10:02 GMT) Full text and rfc822 format available.

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Akim Demaille <akim.demaille <at> gmail.com>
Cc: bug-automake <at> gnu.org, Bison Patches <bison-patches <at> gnu.org>
Subject: Re: Get rid of ylwrap, and simplify yacc/lex rules
Date: Tue, 08 May 2012 13:07:19 +0200
On 05/08/2012 12:48 PM, Akim Demaille wrote:
> 
> On second thought, this is unfair to ylwrap.  The issue is
> rather that bison issues synclines which are relative the
> cwd instead of relative to the path of the output file.
> 
> [SNIP]
>
Well, good for us then :-)

>> Or maybe we could get rid of ylwrap and assume yacc/lex support the
>> '-o' option.  After all:
>>
>>  - the yacc- and lex-generated files are automatically distributed,
>>    so only the maintainers should be expected to regenerate them;
>>
>>  - the sytem yacc on OpenBSD and NetBSD (not surre
>>
s/surre/sure/ here, of course.

>> yet about FreeBSD, will check)
>>
It seems to be the case with FreeBSD (8.2) as well :-)

>> support the '-o' option as well; and their lex too, since its flex
>>
And here, s/its/it's/;  what a sloppy typing I did :-(

>>  - supporting proprietary yacc and lex should be low-priority IMHO.
>>
>> Might this be something to consider for Automake 1.13?
> 
> I have no strong opinion about this.  But I like simplifications :)
>
Me too (at least when the maintenance of the simplified code base is
up to me ;-)

Still, there's a feature of ylwrap I'm going to miss, i.e., the fact
that the timestamp of the header file generated with 'yacc -d' is not
updated if its content is not changed.  This could probably be saved
by inlining the relevant part of the ylwrap logic in the new Makefile
rules though.  Or would be worth to add such a logic to bison proper?

Regards,
  Stefano




Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 08 May 2012 14:24:02 GMT) Full text and rfc822 format available.

Message #14 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Akim Demaille <akim.demaille <at> gmail.com>
Cc: bug-gnulib <bug-gnulib <at> gnu.org>, 11419 <at> debbugs.gnu.org,
	Stefano Lattarini <stefano.lattarini <at> gmail.com>,
	bison-patches <at> gnu.org, Coreutils <coreutils <at> gnu.org>
Subject: Re: bug#11419: Get rid of ylwrap, and simplify yacc/lex rules
Date: Tue, 08 May 2012 08:21:05 -0600
[Message part 1 (text/plain, inline)]
[adding coreutils, bug-gnulib]

On 05/08/2012 04:48 AM, Akim Demaille wrote:

> This is why bison gets (in my VPATH-build where builddir is two
> levels down inside srcdir):
> 
> #line 277 "parse-gram.y"
> #line 2271 "../../../src/parse-gram.c"
> 
> with a properly computed relative path, it should be
> "parse-gram.c".  This is what the patch was fixing.
> 
> This is painful and regularly results in Emacs not being able
> to find the file where the error occurred.
> 
> This should be fixed before 2.5.1 imho.  I'll see if gnulib
> offers something to compute relative paths.

Coreutils has code to compute relative paths; used by both 'realpath'
and 'ln'; maybe it's time to graduate that to a full-blown gnulib module
now that we want to use it elsewhere.

-- 
Eric Blake   eblake <at> redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Mon, 25 Jun 2012 09:35:01 GMT) Full text and rfc822 format available.

Message #17 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Akim Demaille <akim <at> lrde.epita.fr>
Cc: Paul Eggert <eggert <at> CS.UCLA.EDU>, Bison Patches <bison-patches <at> gnu.org>,
	"automake-patches <at> gnu.org" <automake-patches <at> gnu.org>,
	Bison Bugs <bug-bison <at> gnu.org>, 11419 <at> debbugs.gnu.org,
	Bruce Korb <bkorb <at> gnu.org>, Akim Demaille <akim.demaille <at> gmail.com>
Subject: Re: ylwrap does not rename y.tab.h in y.tab.c
Date: Mon, 25 Jun 2012 11:30:18 +0200
Reference:
<http://lists.gnu.org/archive/html/bug-bison/2012-06/msg00032.html>

On 06/25/2012 10:25 AM, Akim Demaille wrote:
> 
> Le 20 juin 2012 à 07:26, Akim Demaille a écrit :
> 
>>
>> Le 15 juin 2012 à 13:34, Akim Demaille a écrit :
>>
>>> I installed it, and some more tests, in maint, as follows.
>>
>>>
>>> From 0f11eec272453d5646e5aeda135650b4fb8ac33d Mon Sep 17 00:00:00 2001
>>> From: Akim Demaille <akim <at> lrde.epita.fr>
>>> Date: Tue, 12 Jun 2012 16:15:14 +0200
>>> Subject: [PATCH 1/2] yacc.c: instead of duplicating y.tab.h inside y.tac.c, include it.
>>
>> We have a problem with this change (which can be seen if
>> you make check in master): as far as I can tell, ylwrap,
>> which goes via y.tab.c and y.tab.h, does _not_ change the
>> occurrences of y.tab.h in y.tab.c.  So in the end, the
>> parser implementation file foo.c includes y.tab.h instead of
>> foo.h.
>>
>> This is Automake 1.12.1, ylwrap 2011-08-25.18.
>>
>> Because of this, it is unclear to me whether we can leave that
>> change, users would face the same issue as bison-master :(
>> And it does not seem acceptable that Bison 2.6 mandates Automake
>> 1.12.2 or whatever.
>>
>> Any suggestion?  In particular a workaround (for Makefile.am)
>> which would work for both older and newer Automakes.
> 
> Well, I guess I must step back.  I installed what follows
> in maint.
>
Sigh, advancement on Bison kept back by the fact that Automake used to
bend over backwards to support inferior yacc implementation that today
hardly anybody is using to build an autoconfiscated package.  It's now
definitely time to implement a 'no-ylwrap' option that prevents the use
of 'ylwrap'.  IMHO this option should go in Automake 1.12.2, become the
default in Automake 1.13, and the only supported mode of operation in
Automake 1.14 (unless real users object).

Patches will follow in the next days.

> What are the versions of Automake to fix?  Should 1.11.*
> be addressed too?
>
I don't think it's worth.  People that will want their generated Makefile.in
working correctly with new Bison will upgrade to Automake 1.12.2.

> I have no idea how wide spread 1.12 is.
>
Well, if it's not much widespread, this new feature will help it become so ;-)

Thanks,
  Stefano




Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Mon, 25 Jun 2012 14:06:01 GMT) Full text and rfc822 format available.

Message #20 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: automake-patches <at> gnu.org
Cc: paul Eggert <eggert <at> CS.UCLA.EDU>, Akim Demaille <akim <at> lrde.epita.fr>,
	bug-bison <at> gnu.org, bison-patches <at> gnu.org, 11419 <at> debbugs.gnu.org,
	Bruce Korb <bkorb <at> gnu.org>, Akime Demaille <akim.demaille <at> gmail.com>
Subject: [PATCH] yacc,
	lex: new 'no-ylwrap' option to prevent use of the 'ylwrap' script
Date: Mon, 25 Jun 2012 16:01:13 +0200
When used with good yacc and lex implementations, like Flex and GNU Bison,
the 'ylwarp' script (meant to work around the deficiencies of older or
inferior yacc and lex implementations) creates far more problems and
annoyances than it solves.  For more details, just see the huge amount of
related bug reports and complaints:

  <http://debbugs.gnu.org/11419>
  <http://debbugs.gnu.org/10825>
  <http://debbugs.gnu.org/9933>
  <http://debbugs.gnu.org/7648>
  <http://lists.gnu.org/archive/html/automake/2010-10/msg00073.html>
  <http://lists.gnu.org/archive/html/automake/2009-03/msg00062.html>
  <http://lists.gnu.org/archive/html/automake/2009-03/msg00061.html>

So we now offer an option ('no-ylwrap') to disable the use of ylwrap in
the Automake-generated rules.  We also offer the inverse option 'ylwrap',
to force the use of the 'ylwrap' script; it is a no-op right now, but its
use will allow us to switch the default in Automake 1.13 (i.e., we will
start *not* using ylwrap by default) while still supporting users who
 want to continue using ylwrap.

* lib/Automake/Options.pm (_is_valid_easy_option, _process_option_list):
Recognize the options 'no-ylwrap' and 'ylwrap'.
* automake.in (handle_languages): Pass the new transform '?USE-YLWRAP?'
to the processed $rule_file; this transform expands to false if the
'no-ylwrap' option is used, and to true otherwise.
(yacc_lex_finish_helper): Don't require nor copy the 'ylwrap' script if
the 'no-ylwrap' option is active.
* lib/am/lex.am, lib/am/yacc.am: Simplify rules not to use the 'ylwrap'
script if the '?USE-YLWRAP?' transform is true.  Related refactorings.
* t/lex-multiple.sh: New test, checks that if we get ylwrap out of the
way, we can build a program using several lexers at once.
* t/list-of-tests.mk (handwritten_TESTS): Add it.
* t/yacc-bison-skeleton.sh: Use the 'no-ylwrap' option.  This allow the
test to pass, at last!
* t/yacc-bison-skeleton-cxx.sh: Likewise.
* t/list-of-tests.mk (XFAIL_TESTS): Remove those two tests.

Signed-off-by: Stefano Lattarini <stefano.lattarini <at> gmail.com>
---

 Here's the first step in our revamping of yacc/lex rules.  It's quite
 rough and  has probably suboptimal semantics, but it will offer a simple
 and quick way out for all those users which were fighting against ylwrap.
 So, OK for maint?  I will follow up with a documentation patch (and a
 NEWS update) if this change is accepted.

 Regards,
   Stefano

 automake.in                  |    6 ++-
 lib/Automake/Options.pm      |    6 +++
 lib/am/lex.am                |   14 ++++--
 lib/am/yacc.am               |   14 ++++--
 t/lex-multiple.sh            |  106 ++++++++++++++++++++++++++++++++++++++++++
 t/list-of-tests.mk           |    3 +-
 t/yacc-bison-skeleton-cxx.sh |    5 +-
 t/yacc-bison-skeleton.sh     |    1 +
 8 files changed, 143 insertions(+), 12 deletions(-)
 create mode 100755 t/lex-multiple.sh

diff --git a/automake.in b/automake.in
index cfc48bf..939c303 100644
--- a/automake.in
+++ b/automake.in
@@ -1472,7 +1472,9 @@ sub handle_languages
 			     COMPILE   => '$(' . $lang->compiler . ')',
 			     LTCOMPILE => '$(LT' . $lang->compiler . ')',
 			     -o        => $output_flag,
-			     SUBDIROBJ => !! option 'subdir-objects');
+			     SUBDIROBJ => !! option 'subdir-objects',
+                             'USE-YLWRAP' => ! option 'no-ylwrap',
+                             );
 	}
 
 	# Now include code for each specially handled object with this
@@ -1549,6 +1551,7 @@ sub handle_languages
 			     COMPILE   => $obj_compile,
 			     LTCOMPILE => $obj_ltcompile,
 			     -o        => $output_flag,
+                             'USE-YLWRAP' => ! option 'no-ylwrap',
 			     %file_transform);
 	}
 
@@ -6084,6 +6087,7 @@ sub lang_lex_target_hook
 # This is a helper for both lex and yacc.
 sub yacc_lex_finish_helper
 {
+  return if option 'no-ylwrap';
   return if defined $language_scratch{'lex-yacc-done'};
   $language_scratch{'lex-yacc-done'} = 1;
 
diff --git a/lib/Automake/Options.pm b/lib/Automake/Options.pm
index 2f977bd..920a62f 100644
--- a/lib/Automake/Options.pm
+++ b/lib/Automake/Options.pm
@@ -285,6 +285,7 @@ sub _is_valid_easy_option ($)
     no-installinfo
     no-installman
     no-texinfo.tex
+    no-ylwrap
     nostdinc
     readme-alpha
     std-options
@@ -333,6 +334,11 @@ sub _process_option_list (\%@)
           # This is a little of an hack, but good enough for the moment.
           delete $options->{'parallel-tests'};
         }
+      elsif ($_ eq 'ylwrap')
+        {
+          # This is a little of an hack, but good enough for the moment.
+          delete $options->{'no-ylwrap'};
+        }
       elsif (/^filename-length-max=(\d+)$/)
         {
           delete $options->{$_};
diff --git a/lib/am/lex.am b/lib/am/lex.am
index 7467f4f..19555b0 100644
--- a/lib/am/lex.am
+++ b/lib/am/lex.am
@@ -23,8 +23,14 @@ endif %?MAINTAINER-MODE%
 
 ?GENERIC?%EXT%%DERIVED-EXT%:
 ?!GENERIC?%OBJ%: %SOURCE%
-?GENERIC?	%VERBOSE%$(am__skiplex) $(SHELL) $(YLWRAP) %SOURCE% $(LEX_OUTPUT_ROOT).c %OBJ% -- %COMPILE%
-?!GENERIC?	%VERBOSE% \
-?!GENERIC??DIST_SOURCE?	$(am__skiplex) \
+if %?GENERIC%
+	%VERBOSE%$(am__skiplex) \
+?USE-YLWRAP?	$(SHELL) $(YLWRAP) %SOURCE% $(LEX_OUTPUT_ROOT).c %OBJ% -- %COMPILE%
+?!USE-YLWRAP?	%COMPILE% -o $@ %SOURCE%
+else !%?GENERIC%
+	%VERBOSE% \
+?DIST_SOURCE?	$(am__skiplex) \
 ## For non-suffix rules, we must emulate a VPATH search on %SOURCE%.
-?!GENERIC?	$(SHELL) $(YLWRAP) `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE% $(LEX_OUTPUT_ROOT).c %OBJ% -- %COMPILE%
+?USE-YLWRAP?	$(SHELL) $(YLWRAP)  `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE% $(LEX_OUTPUT_ROOT).c %OBJ% -- %COMPILE%
+?!USE-YLWRAP?	%COMPILE% -o $@  `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE%
+endif  !%?GENERIC%
diff --git a/lib/am/yacc.am b/lib/am/yacc.am
index e74259f..77af963 100644
--- a/lib/am/yacc.am
+++ b/lib/am/yacc.am
@@ -43,8 +43,14 @@ endif %?FIRST%
 
 ?GENERIC?%EXT%%DERIVED-EXT%:
 ?!GENERIC?%OBJ%: %SOURCE%
-?GENERIC?	%VERBOSE%$(am__skipyacc) $(SHELL) $(YLWRAP) %SOURCE% y.tab.c %OBJ% y.tab.h `echo %OBJ% | $(am__yacc_c2h)` y.output %BASE%.output -- %COMPILE%
-?!GENERIC?	%VERBOSE% \
-?!GENERIC??DIST_SOURCE?	$(am__skipyacc) \
+if %?GENERIC%
+	%VERBOSE%$(am__skipyacc) \
+?USE-YLWRAP?	$(SHELL) $(YLWRAP) %SOURCE% y.tab.c %OBJ% y.tab.h `echo %OBJ% | $(am__yacc_c2h)` y.output %BASE%.output -- %COMPILE%
+?!USE-YLWRAP?	%COMPILE% -o $@ %SOURCE%
+else !%?GENERIC%
+	%VERBOSE% \
+?DIST_SOURCE?	$(am__skipyacc) \
 ## For non-suffix rules, we must emulate a VPATH search on %SOURCE%.
-?!GENERIC?	$(SHELL) $(YLWRAP) `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE% y.tab.c %OBJ% y.tab.h `echo %OBJ% | $(am__yacc_c2h)` y.output %BASE%.output -- %COMPILE%
+?USE-YLWRAP?	$(SHELL) $(YLWRAP)  `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE% y.tab.c %OBJ% y.tab.h `echo %OBJ% | $(am__yacc_c2h)` y.output %BASE%.output -- %COMPILE%
+?!USE-YLWRAP?	%COMPILE% -o $@ % `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE%
+endif  !%?GENERIC%
diff --git a/t/lex-multiple.sh b/t/lex-multiple.sh
new file mode 100755
index 0000000..f15b3f3
--- /dev/null
+++ b/t/lex-multiple.sh
@@ -0,0 +1,106 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# If we get ylwrap out of the way, we can build a program using
+# several lexers at once.
+
+required='cc flex'
+. ./defs || Exit 1
+
+cp "$am_scriptdir"/ar-lib . || fatal_ "fetching auxiliary script 'ar-lib'"
+
+cat >> configure.ac << 'END'
+AC_PROG_CC
+AC_PROG_LEX
+AM_PROG_AR
+AC_PROG_RANLIB
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = no-ylwrap
+
+bin_PROGRAMS = zardoz
+
+zardoz_SOURCES = main.c
+# Convenience libraries.
+noinst_LIBRARIES = liblex.a liblex-foo.a liblex-bar.a
+zardoz_LDADD = $(noinst_LIBRARIES)
+
+liblex_a_SOURCES = 0.l
+
+liblex_foo_a_LFLAGS = -Pfoo
+liblex_foo_a_SOURCES = a.l
+
+liblex_bar_a_LFLAGS = -Pbar_
+liblex_bar_a_SOURCES = b.l
+END
+
+cat > main.c << 'END'
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+int main (int argc, char *argv[])
+{
+  if (argc != 2)
+    abort ();
+  else if (!strcmp(argv[1], "--vanilla"))
+    return (yylex () != 121);
+  else if (!strcmp(argv[1], "--foo"))
+    return (foolex () != 121);
+  else if (!strcmp(argv[1], "--bar"))
+    return (bar_lex () != 121);
+  else
+    abort ();
+}
+END
+
+cat > 0.l << 'END'
+%{
+#define YY_NO_UNISTD_H 1
+%}
+%%
+"VANILLA" { printf (":%s:\n", yytext); return 121; }
+. { printf (":%s:\n", yytext); return 1; }
+%%
+/* Avoid possible link errors. */
+int yywrap (void) { return 1; }
+END
+
+sed 's/VANILLA/FOO/' 0.l > a.l
+sed 's/VANILLA/BAR/' 0.l > b.l
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+test ! -f ylwrap
+
+./configure
+$MAKE
+
+if ! cross_compiling; then
+  echo VANILLA | ./zardoz --vanilla
+  echo FOO | ./zardoz --foo
+  echo BAR | ./zardoz --bar
+  #./zardoz --vanilla </dev/null && exit 1
+  echo BAR | ./zardoz --foo && Exit 1
+  : For shells with buster 'set -e'.
+fi
+
+$MAKE distcheck
+
+:
diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk
index e16dd1d..a5962d9 100644
--- a/t/list-of-tests.mk
+++ b/t/list-of-tests.mk
@@ -30,8 +30,6 @@ t/pm/Version3.pl
 
 XFAIL_TESTS = \
 t/all.sh \
-t/yacc-bison-skeleton-cxx.sh \
-t/yacc-bison-skeleton.sh \
 t/cond17.sh \
 t/gcj6.sh \
 t/override-conditional-2.sh \
@@ -581,6 +579,7 @@ t/lex-depend-cxx.sh \
 t/lex-depend-grep.sh \
 t/lex-depend.sh \
 t/lex-line.sh \
+t/lex-multiple.sh \
 t/lex-nodist.sh \
 t/lex-pr204.sh \
 t/lflags.sh \
diff --git a/t/yacc-bison-skeleton-cxx.sh b/t/yacc-bison-skeleton-cxx.sh
index f5d8545..1ab3667 100755
--- a/t/yacc-bison-skeleton-cxx.sh
+++ b/t/yacc-bison-skeleton-cxx.sh
@@ -20,7 +20,10 @@
 required='c++ bison'
 . ./defs || Exit 1
 
-cat >> configure.ac << 'END'
+cat > configure.ac << 'END'
+AC_INIT([C++ skeleton], [0.1])
+AM_INIT_AUTOMAKE([no-ylwrap])
+AC_CONFIG_FILES([Makefile])
 AC_PROG_CXX
 AC_PROG_YACC
 AC_OUTPUT
diff --git a/t/yacc-bison-skeleton.sh b/t/yacc-bison-skeleton.sh
index 8cf957b..6cf7972 100755
--- a/t/yacc-bison-skeleton.sh
+++ b/t/yacc-bison-skeleton.sh
@@ -27,6 +27,7 @@ AC_OUTPUT
 END
 
 cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = no-ylwrap
 bin_PROGRAMS = zardoz
 zardoz_SOURCES = zardoz.y foo.c
 AM_YFLAGS = -d --skeleton glr.c
-- 
1.7.9.5





Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 26 Jun 2012 14:32:02 GMT) Full text and rfc822 format available.

Message #23 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Akim Demaille <akim <at> lrde.epita.fr>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: Paul Eggert <eggert <at> CS.UCLA.EDU>, Bison Bugs <bug-bison <at> gnu.org>,
	"automake-patches <at> gnu.org" <automake-patches <at> gnu.org>,
	Bison Patches <bison-patches <at> gnu.org>, 11419 <at> debbugs.gnu.org,
	Bruce Korb <bkorb <at> gnu.org>
Subject: Re: ylwrap does not rename y.tab.h in y.tab.c
Date: Tue, 26 Jun 2012 16:27:34 +0200
Hi all,

Le 25 juin 2012 à 11:30, Stefano Lattarini a écrit :

>> Well, I guess I must step back.  I installed what follows
>> in maint.
>> 
> Sigh, advancement on Bison kept back by the fact that Automake used to
> bend over backwards to support inferior yacc implementation that today
> hardly anybody is using to build an autoconfiscated package.  It's now
> definitely time to implement a 'no-ylwrap' option that prevents the use
> of 'ylwrap'.  IMHO this option should go in Automake 1.12.2, become the
> default in Automake 1.13, and the only supported mode of operation in
> Automake 1.14 (unless real users object).

But in any case, ylwrap must also be fixed, if people still
want to stick to it.  And, BTW, glr parsers, C++ parsers
etc. are broken by ylwrap currently as they all use a
header and depend upon it.

A future version of bison will use it too for plain parsers.
People stuck with an old Automake can make a "y.tab.h" file
that just #includes "the-right-header.h" to work around the
issue.





Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 26 Jun 2012 14:39:02 GMT) Full text and rfc822 format available.

Message #26 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Akim Demaille <akim <at> lrde.epita.fr>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: paul Eggert <eggert <at> CS.UCLA.EDU>, bug-bison <at> gnu.org,
	automake-patches <at> gnu.org, bison-patches <at> gnu.org,
	11419 <at> debbugs.gnu.org, Bruce Korb <bkorb <at> gnu.org>
Subject: Re: [PATCH] yacc,
	lex: new 'no-ylwrap' option to prevent use of the 'ylwrap' script
Date: Tue, 26 Jun 2012 16:34:02 +0200
Hi Stefano,

Thanks for this!

Le 25 juin 2012 à 16:01, Stefano Lattarini a écrit :

> When used with good yacc and lex implementations, like Flex and GNU Bison,
> the 'ylwarp'

ylwrap

> script (meant to work around the deficiencies of older or
> inferior yacc and lex implementations) creates far more problems and
> annoyances than it solves.  For more details, just see the huge amount of
> related bug reports and complaints:
>  

> use will allow us to switch the default in Automake 1.13 (i.e., we will
> start *not* using ylwrap by default) while still supporting users who
> want to continue using ylwrap.

Shouldn't some Autoconf machinery just check if -o is supported?
I don't see the need for the user to have to decide whether
to use ylwrap or not.  But I agree this is certainly harder to do.


> 
> * lib/Automake/Options.pm (_is_valid_easy_option, _process_option_list):
> Recognize the options 'no-ylwrap' and 'ylwrap'.
> * automake.in (handle_languages): Pass the new transform '?USE-YLWRAP?'
> to the processed $rule_file; this transform expands to false if the
> 'no-ylwrap' option is used, and to true otherwise.
> (yacc_lex_finish_helper): Don't require nor copy the 'ylwrap' script if
> the 'no-ylwrap' option is active.
> * lib/am/lex.am, lib/am/yacc.am: Simplify rules not to use the 'ylwrap'
> script if the '?USE-YLWRAP?' transform is true.  Related refactorings.
> * t/lex-multiple.sh: New test, checks that if we get ylwrap out of the
> way, we can build a program using several lexers at once.
> * t/list-of-tests.mk (handwritten_TESTS): Add it.
> * t/yacc-bison-skeleton.sh: Use the 'no-ylwrap' option.  This allow the

allows

> test to pass, at last!
> * t/yacc-bison-skeleton-cxx.sh: Likewise.
> * t/list-of-tests.mk (XFAIL_TESTS): Remove those two tests.
> 

> @@ -333,6 +334,11 @@ sub _process_option_list (\%@)
>           # This is a little of an hack, but good enough for the moment.
>           delete $options->{'parallel-tests'};
>         }
> +      elsif ($_ eq 'ylwrap')
> +        {
> +          # This is a little of an hack, but good enough for the moment.

a hack (several times apparently :).

> +          delete $options->{'no-ylwrap'};
> +        }
>       elsif (/^filename-length-max=(\d+)$/)
>         {
>           delete $options->{$_};





Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 26 Jun 2012 15:40:02 GMT) Full text and rfc822 format available.

Message #29 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Akim Demaille <akim <at> lrde.epita.fr>
Cc: paul Eggert <eggert <at> CS.UCLA.EDU>, bug-bison <at> gnu.org,
	automake-patches <at> gnu.org, bison-patches <at> gnu.org,
	11419 <at> debbugs.gnu.org, Bruce Korb <bkorb <at> gnu.org>
Subject: Re: [PATCH] yacc, lex: new 'no-ylwrap' option to prevent use of the
	'ylwrap' script
Date: Tue, 26 Jun 2012 17:35:11 +0200
On 06/26/2012 04:34 PM, Akim Demaille wrote:
> Hi Stefano,
> 
> Thanks for this!
> 
> Le 25 juin 2012 à 16:01, Stefano Lattarini a écrit :
> 
>> When used with good yacc and lex implementations, like Flex and GNU Bison,
>> the 'ylwarp'
> 
> ylwrap
> 
>> script (meant to work around the deficiencies of older or
>> inferior yacc and lex implementations) creates far more problems and
>> annoyances than it solves.  For more details, just see the huge amount of
>> related bug reports and complaints:
>>  
> 
>> use will allow us to switch the default in Automake 1.13 (i.e., we will
>> start *not* using ylwrap by default) while still supporting users who
>> want to continue using ylwrap.
> 
> Shouldn't some Autoconf machinery just check if -o is supported?
>
This is probably a better idea, yes.  This could probably be done by
enhancing AM_PROG_LEX and defining a similar new AM_PROG_YACC macro.
Or better again, it could be done directly in AC_PROG_LEX and
AC_PROG_YACC, so that we could just rely on them in the future
(in the meantime, we could copy their new definitions in our
'm4/' directories, so that the new improved definitions will still
be accessible to users with older Autoconfs).

> I don't see the need for the user to have to decide whether
> to use ylwrap or not.  But I agree this is certainly harder to do.
>
Not much harder actually; and it's certainly much nicer to DTRT
instead of throwing another knob to tweak to the poor developer.
So I'll certainly give your proposal a try.

Thanks,
  Stefano




Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 26 Jun 2012 16:03:01 GMT) Full text and rfc822 format available.

Message #32 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Akim Demaille <akim <at> lrde.epita.fr>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: paul Eggert <eggert <at> CS.UCLA.EDU>, bug-bison <at> gnu.org,
	automake-patches <at> gnu.org, bison-patches <at> gnu.org,
	11419 <at> debbugs.gnu.org, Bruce Korb <bkorb <at> gnu.org>
Subject: Re: [PATCH] yacc,
	lex: new 'no-ylwrap' option to prevent use of the 'ylwrap' script
Date: Tue, 26 Jun 2012 17:58:48 +0200
Le 26 juin 2012 à 17:35, Stefano Lattarini a écrit :

> This is probably a better idea, yes.  This could probably be done by
> enhancing AM_PROG_LEX and defining a similar new AM_PROG_YACC macro.
> Or better again, it could be done directly in AC_PROG_LEX and
> AC_PROG_YACC, so that we could just rely on them in the future
> (in the meantime, we could copy their new definitions in our
> 'm4/' directories, so that the new improved definitions will still
> be accessible to users with older Autoconfs).

I had sent a message about such macros to Autoconf, but
to no avail.

http://lists.gnu.org/archive/html/autoconf-patches/2012-04/msg00001.html

This initial work leaves in Bison as m4/flex.m4.  I know there is
work to do (test, doc etc.), but at least I'd like some feedback.
Then I'll proceed to Yacc/Bison.





Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 26 Jun 2012 16:24:02 GMT) Full text and rfc822 format available.

Message #35 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Akim Demaille <akim <at> lrde.epita.fr>
Cc: paul Eggert <eggert <at> CS.UCLA.EDU>, Autoconf <Autoconf <at> gnu.org>,
	bug-bison <at> gnu.org, automake-patches <at> gnu.org,
	bison-patches <at> gnu.org, 11419 <at> debbugs.gnu.org,
	Stefano Lattarini <stefano.lattarini <at> gmail.com>, Bruce Korb <bkorb <at> gnu.org>
Subject: Re: [PATCH] yacc, lex: new 'no-ylwrap' option to prevent use of the
	'ylwrap' script
Date: Tue, 26 Jun 2012 10:18:54 -0600
[Message part 1 (text/plain, inline)]
[adding autoconf]

On 06/26/2012 09:58 AM, Akim Demaille wrote:
> 
> Le 26 juin 2012 à 17:35, Stefano Lattarini a écrit :
> 
>> This is probably a better idea, yes.  This could probably be done by
>> enhancing AM_PROG_LEX and defining a similar new AM_PROG_YACC macro.
>> Or better again, it could be done directly in AC_PROG_LEX and
>> AC_PROG_YACC, so that we could just rely on them in the future
>> (in the meantime, we could copy their new definitions in our
>> 'm4/' directories, so that the new improved definitions will still
>> be accessible to users with older Autoconfs).
> 
> I had sent a message about such macros to Autoconf, but
> to no avail.
> 
> http://lists.gnu.org/archive/html/autoconf-patches/2012-04/msg00001.html

Eek - that just shows that I'm really behind on reading my email.

Just from reading this summary, the idea of improving AC_PROG_LEX and
AC_PROG_YACC to be more useful makes sense, especially if it would make
automake easier to maintain.  What sort of improvements are needed?

> 
> This initial work leaves in Bison as m4/flex.m4.  I know there is
> work to do (test, doc etc.), but at least I'd like some feedback.
> Then I'll proceed to Yacc/Bison.

-- 
Eric Blake   eblake <at> redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org



[signature.asc (application/pgp-signature, attachment)]

Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 26 Jun 2012 16:32:01 GMT) Full text and rfc822 format available.

Message #38 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Akim Demaille <akim <at> lrde.epita.fr>
To: Eric Blake <eblake <at> redhat.com>
Cc: paul Eggert <eggert <at> CS.UCLA.EDU>, Autoconf <Autoconf <at> gnu.org>,
	bug-bison <at> gnu.org, automake-patches <at> gnu.org,
	bison-patches <at> gnu.org, 11419 <at> debbugs.gnu.org,
	Stefano Lattarini <stefano.lattarini <at> gmail.com>, Bruce Korb <bkorb <at> gnu.org>
Subject: Re: [PATCH] yacc,
	lex: new 'no-ylwrap' option to prevent use of the 'ylwrap' script
Date: Tue, 26 Jun 2012 18:27:27 +0200
(wow, _that_ is quite a list of CCs.  Hi mum!)

Hi Eric,

Le 26 juin 2012 à 18:18, Eric Blake a écrit :

> Eek - that just shows that I'm really behind on reading my email.

Thou shalt be punished.  Beware of my wrath.

> Just from reading this summary, the idea of improving AC_PROG_LEX and
> AC_PROG_YACC to be more useful makes sense, especially if it would make
> automake easier to maintain.  What sort of improvements are needed?

In the present case, I believe the only thing Automake needs
to know is whether -o is supported by lex/yacc.



Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Tue, 26 Jun 2012 17:02:02 GMT) Full text and rfc822 format available.

Message #41 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Akim Demaille <akim <at> lrde.epita.fr>
Cc: paul Eggert <eggert <at> CS.UCLA.EDU>, Autoconf <Autoconf <at> gnu.org>,
	bug-bison <at> gnu.org, automake-patches <at> gnu.org,
	bison-patches <at> gnu.org, 11419 <at> debbugs.gnu.org,
	Bruce Korb <bkorb <at> gnu.org>, Eric Blake <eblake <at> redhat.com>
Subject: Re: [PATCH] yacc, lex: new 'no-ylwrap' option to prevent use of the
	'ylwrap' script
Date: Tue, 26 Jun 2012 18:57:49 +0200
Hi Eric.

On 06/26/2012 06:27 PM, Akim Demaille wrote:
>
> Le 26 juin 2012 à 18:18, Eric Blake a écrit :
> 
>> Just from reading this summary, the idea of improving AC_PROG_LEX and
>> AC_PROG_YACC to be more useful makes sense, especially if it would make
>> automake easier to maintain.  What sort of improvements are needed?
> 
> In the present case, I believe the only thing Automake needs
> to know is whether -o is supported by lex/yacc.
>
Yes.  And to ensure proper compatibility with C++ and with the use of the
'-d' option, we should consider the '-o' support good enough only provided
that, when "yacc -d -o foo.C-EXT" is used, the name of the created header
is "foo.H-EXT", where H-EXT is C-EXT with all 'c' characters substituted
by 'h' ones (so that foo.c -> foo.h, foo.c++ -> foo.h++, foo.cc -> foo.hh,
etc).  This is what Bison does.

Once we know that lex supports '-o', the determination of the value of
$LEX_OUTPUT_ROOT will mostly become moot, and could be dispensed with
(at least for what concerns Automake).

Also, it would be nice if AC_PROG_YACC and AC_PROG_LEX would allow us to
change the list of the searched programs (this will allow the Automake
configure script to prefer non-bison 'yacc' and non-flex 'lex' programs
for use in the testsuite, to ensure better coverage).

Finally, while we are at it, we might also think about making default
values for $YACC and $LEX configurable (rather than hard-coding them
to resp. 'yacc' and ':').

Oh, and setting a couple of (self-explanatory) variables $LEX_IS_FLEX
and $YACC_IS_BISON wouldn't hurt (albeit it would not be needed by
Automake at this point).

Regards,
  Stefano




Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Sat, 12 Jan 2013 19:19:02 GMT) Full text and rfc822 format available.

Message #44 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: 11419 <at> debbugs.gnu.org
Cc: Akim Demaille <akim.demaille <at> gmail.com>
Subject: Get rid of ylwrap,
	and simplify yacc/lex rules: withdrawn this wishlist entry
Date: Sat, 12 Jan 2013 20:18:00 +0100
tags 11419 wontfix
close 11419
thanks

Reference:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11419>

The improvements to the ylwrap script contributed by Akim in the 1.12.x
and 1.13 releases have fixed most of the issues or bugs this proposal
intended to address; even better, they did so in a completely transparent
and backward-compatible way.

While this proposal could theoretically still provide some enhancement
in corner cases, those would be very marginal gains, and since I don't
like the gratuitous proliferation of options and dubious knobs, I'm
retiring this wishlist entry for now.

Thanks,
  Stefano




Added tag(s) wontfix. Request was from Stefano Lattarini <stefano.lattarini <at> gmail.com> to control <at> debbugs.gnu.org. (Sat, 12 Jan 2013 19:19:02 GMT) Full text and rfc822 format available.

bug closed, send any further explanations to 11419 <at> debbugs.gnu.org and Stefano Lattarini <stefano.lattarini <at> gmail.com> Request was from Stefano Lattarini <stefano.lattarini <at> gmail.com> to control <at> debbugs.gnu.org. (Sat, 12 Jan 2013 19:19:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-automake <at> gnu.org:
bug#11419; Package automake. (Fri, 01 Feb 2013 17:18:01 GMT) Full text and rfc822 format available.

Message #51 received at 11419 <at> debbugs.gnu.org (full text, mbox):

From: Akim Demaille <akim.demaille <at> gmail.com>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 11419 <at> debbugs.gnu.org
Subject: Re: Get rid of ylwrap, and simplify yacc/lex rules: withdrawn this
	wishlist entry
Date: Fri, 1 Feb 2013 18:16:37 +0100
On Sat, Jan 12, 2013 at 8:18 PM, Stefano Lattarini
<stefano.lattarini <at> gmail.com> wrote:
> tags 11419 wontfix
> close 11419
> thanks
>
> Reference:
> <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11419>
>
> The improvements to the ylwrap script contributed by Akim in the 1.12.x
> and 1.13 releases have fixed most of the issues or bugs this proposal
> intended to address; even better, they did so in a completely transparent
> and backward-compatible way.
>
> While this proposal could theoretically still provide some enhancement
> in corner cases, those would be very marginal gains, and since I don't
> like the gratuitous proliferation of options and dubious knobs, I'm
> retiring this wishlist entry for now.

Not quite actually.  Currently Automake invoke Bison as yacc, i.e., it
passes the flag -y, --yacc.  This flag is expected to trigger all sorts of
warnings if you use features that are not supported by POSIX Yacc.

Because Bison is frequently invoked with -y but people do use Bison
features in their grammar, I cannot have --yacc properly do its job:
non-Yaccisms are not diagnosed to the user.

I would really like to see -y stop being used so that differences with
Yacc itself can be reported.

Cheers!




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sat, 02 Mar 2013 12:24:02 GMT) Full text and rfc822 format available.

This bug report was last modified 11 years and 28 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.