GNU bug report logs - #7884
Yacc and FreeBSD make in VPATH builds

Previous Next

To reply to this bug, email your comments to 7884 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: bug-automake <at> gnu.org
Subject: Yacc and FreeBSD make in VPATH builds
Date: Fri, 21 Jan 2011 22:28:24 +0100

[Message part 1 (text/plain, inline)]

Hello automakers.

What I'm going to illustrate here is a long-standing bug in automake;
but since I don't see it described in the manual, nor reported in the
GNATS database, making it explicitly registered here in Debbugs won't
hurt IMHO.

-*-*-

Here is a description of the bug.

 1) The package developer declares a Yacc-generated program in the
    Makefile.am:
      bin_PROGRAMS = calc
      cacl_SOURCES = calc.y

 2) From that input, automake can generate rules to create *and distribute*
    the Yacc-derived C file `calc.c'.  This C file is distributed so that
    the user won't need to have a yacc program installed in order to build
    the package.

 3) The user has GNU bison installed on his developement machine, and also
    wants to take advantage of bison-specific features.  So he decides to
    just require GNU bison, knowing that such a decision shouldn't affect
    the end users --  the `calc.c' file will be shipped in the distribution
    tarball, so that those users won't need bison in order to rebuild it.

 4) The developer completes the program, and uses "make dist" to build
    the distribution tarball; a proper, up-to-date copy of the generated
    `calc.c' file will be bundled there.

 4) A FreeBSD user, which doesn't have bison installed, but which has the
    standard BSD yacc available in his PATH, fetch the tarball, and
    attempts a VPATH build of the package:
      $ tar xzvf calc-1.0.tar.gz
      $ cd calc-1.0
      $ mkdir build && cd build
      $ ../configure
      ...
      $ make  # this is FreeBSD make
   
 5) For some reason, FreeBSD make does not see, or decides not to use,
    the `calc.c' file in VPATH (here, `..'), and tries to rebuild it
    in thecurrent directory, obviously using FreeBSD yacc.  Oops.
   
I used to think that this bug was due to the fact that FreeBSD make
never performs a VPATH search for a dependency that has an explicit
rule, as reported also in the autoconf manual, see:
 <http://www.gnu.org/software/autoconf/manual/html_node/Make-Target-Lookup.html>

But digging some more, I realized that the bug is present even when
only suffix rues are involved :-(  (see the minmal testcase attached).

-*-*-

The already-committed (to master) testcase `yacc-dist-nobuild.test'
should provide coverage for the bug.

-*-*-

Proposals for a fix (Note: none of these solutions has been really
attempted by me in practice, and none of them seems particularly
appealing, either):

 - Always create yacc-derived and distributed .c and .h files in
   $(srcdir), and explicitly instruct make to alwsys look fr them
   there at first -- similarly to what is done for info files
   (yuck)

 - Tell the developer to add proper AC_CONFIG_LINKS to configure in
   order to support VPATH builds with FreeBSD make (yuck^2, since
   that means keeping track of renamed files when object-specific
   flags are involved).

 - Make the `parse.c' and `parse.h' rules look for same named files
   in $(srcdir) during a VPATH build, and copy them in $(builddir)
   if they're neweer than `parse.y'.  But then how can distclean be
   made to work in evey situation?

 - Tell that FreeBSD make is not supported in VPATH builds (not
   really a solution, I admit).

-*-*-

Regards,
   Stefano

[foo.test (application/x-shellscript, inline)]

Message #8 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 7884 <at> debbugs.gnu.org
Subject: Re: bug#7884: Yacc and FreeBSD make in VPATH builds
Date: Sat, 22 Jan 2011 10:11:15 +0100

* Stefano Lattarini wrote on Fri, Jan 21, 2011 at 10:28:24PM CET:
>  - Always create yacc-derived and distributed .c and .h files in
>    $(srcdir), and explicitly instruct make to alwsys look fr them
>    there at first -- similarly to what is done for info files
>    (yuck)

Yes, this is ugly, but it is a way that can be made to work.
One complication is that, for portable semantics, if a file is
ever mentioned as $(srcdir)/file as a prerequisite, then it also
needs to be listed that way as a target.  (I think; it might've
been the other way round.)

Downside of this is that now builds fail to work as expected when
there are newer versions of the files in the build tree.  GNU make
users would likely complain.

Still, this could be viable as optional behavior.

>  - Tell the developer to add proper AC_CONFIG_LINKS to configure in
>    order to support VPATH builds with FreeBSD make (yuck^2, since
>    that means keeping track of renamed files when object-specific
>    flags are involved).
> 
>  - Make the `parse.c' and `parse.h' rules look for same named files
>    in $(srcdir) during a VPATH build, and copy them in $(builddir)
>    if they're neweer than `parse.y'.  But then how can distclean be
>    made to work in evey situation?

I doubt that these two options can be made to work sanely.  A
  file: $(srcdir)/file

doesn't work with GNU make, for example: it will rightfully complain
about a circular dependency.

>  - Tell that FreeBSD make is not supported in VPATH builds (not
>    really a solution, I admit).

Why not?  VPATH is not specified by Posix, and there is a reason for
that: behavior is just too variable and different between makes.  This
means that only some setups work well, others don't.

I'm concerned mostly with users downloading stuff: they should be able
to build source packages.  If VPATH doesn't work, that is not a big
problem because they can still use an in-tree build.  It doesn't make
the package unusable for them.

The actual complication, as far as I can see, comes from the fact that
I want VPATH builds to work for the packages I'm interested in.  ;-)

Cheers,
Ralf

Message #11 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
Cc: 7884 <at> debbugs.gnu.org
Subject: Re: bug#7884: Yacc and FreeBSD make in VPATH builds
Date: Tue, 25 Jan 2011 20:52:56 +0100

Hello automakers.

A small update on this ...

On Saturday 22 January 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Fri, Jan 21, 2011 at 10:28:24PM CET:
> >  - Always create yacc-derived and distributed .c and .h files in
> >    $(srcdir), and explicitly instruct make to alwsys look fr them
> >    there at first -- similarly to what is done for info files
> >    (yuck)
> 
> Yes, this is ugly, but it is a way that can be made to work.
> One complication is that, for portable semantics, if a file is
> ever mentioned as $(srcdir)/file as a prerequisite, then it also
> needs to be listed that way as a target.  (I think; it might've
> been the other way round.)
> 
> Downside of this is that now builds fail to work as expected when
> there are newer versions of the files in the build tree.  GNU make
> users would likely complain.
>
Yes... and they would be right IMO.  To be honest, if I were just an
automake user, I'd complain myself about such a behaviour.  So IMVHO
this is not a viable solution, after all.

> Still, this could be viable as optional behavior.
> 
> >  - Tell the developer to add proper AC_CONFIG_LINKS to configure in
> >    order to support VPATH builds with FreeBSD make (yuck^2, since
> >    that means keeping track of renamed files when object-specific
> >    flags are involved).
> >
Hmm... maybe a variation of this might be a good solution after all, at
least for FreeBSD make: the special `.BEGIN' target[1][2] might be used
to copy Yacc-derived *.c and *.h files from $(srcdir) to $(builddir),
unless they are already there.

 [1] See <http://www.freebsd.org/cgi/man.cgi?query=make> for more info.
 [2] This variable doesn't seem to cause problems with GNU make, Solaris
     XPG4/CCS make, Solaris dmake, not Heirloom make.

Note that the 'distcleancheck' target would still fail with this change,
but that's more of an annoyance rather than a serious bug (like the one
under discussion is IMHO).  And we could try to fix that later.

> >  - Make the `parse.c' and `parse.h' rules look for same named files
> >    in $(srcdir) during a VPATH build, and copy them in $(builddir)
> >    if they're neweer than `parse.y'.  But then how can distclean be
> >    made to work in evey situation?
> 
> I doubt that these two options can be made to work sanely.  A
>   file: $(srcdir)/file
> 
> doesn't work with GNU make, for example: it will rightfully complain
> about a circular dependency.
> 
> >  - Tell that FreeBSD make is not supported in VPATH builds (not
> >    really a solution, I admit).
> 
> Why not?  VPATH is not specified by Posix, and there is a reason for
> that: behavior is just too variable and different between makes.  This
> means that only some setups work well, others don't.
> 
> I'm concerned mostly with users downloading stuff: they should be able
> to build source packages.  If VPATH doesn't work, that is not a big
> problem because they can still use an in-tree build.  It doesn't make
> the package unusable for them.
> 
Anyway, I think this bug woul be worth fixing, if that can be done
without too much fuss.

> The actual complication, as far as I can see, comes from the fact that
> I want VPATH builds to work for the packages I'm interested in.  ;-)
> 
:-)

Regards,
  Stefano

Message #14 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 7884 <at> debbugs.gnu.org
Subject: Re: bug#7884: Yacc and FreeBSD make in VPATH builds
Date: Tue, 25 Jan 2011 21:05:17 +0100

* Stefano Lattarini wrote on Tue, Jan 25, 2011 at 08:52:56PM CET:
> On Saturday 22 January 2011, Ralf Wildenhues wrote:
> > 
> > >  - Tell the developer to add proper AC_CONFIG_LINKS to configure in
> > >    order to support VPATH builds with FreeBSD make (yuck^2, since
> > >    that means keeping track of renamed files when object-specific
> > >    flags are involved).
> > >
> Hmm... maybe a variation of this might be a good solution after all, at
> least for FreeBSD make: the special `.BEGIN' target[1][2] might be used
> to copy Yacc-derived *.c and *.h files from $(srcdir) to $(builddir),
> unless they are already there.

How do you ensure that you don't overwrite newer such files in the build
tree?  Please note that 'test' and 'ls' might not have the same time
stamp reading granularity that 'make' might have.  Well, at least in
general, this might be less of a problem on FreeBSD specifically.

Anyway, the above sounds a lot like a hack with possible traps, I'm not
sure I like going that way.

> Note that the 'distcleancheck' target would still fail with this change,
> but that's more of an annoyance rather than a serious bug (like the one
> under discussion is IMHO).  And we could try to fix that later.

Well, regressions are generally worse than bugs.

Generally, whatever solution you would like to propose, I would suggest
that you could first try to create in a simple Makefile the rules as you
think automake should produce them; then we have something to digest
(and possibly throw away) without having to invest a lot of work in a
proper automake implementation.  It is often possible to rule out
strategies quickly at such an early stage.

Thanks,
Ralf

Message #17 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
Cc: 7884 <at> debbugs.gnu.org
Subject: Re: bug#7884: Yacc and FreeBSD make in VPATH builds
Date: Thu, 27 Jan 2011 11:54:50 +0100

[Message part 1 (text/plain, inline)]

On Tuesday 25 January 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Tue, Jan 25, 2011 at 08:52:56PM CET:
> > On Saturday 22 January 2011, Ralf Wildenhues wrote:
> > > 
> > > >  - Tell the developer to add proper AC_CONFIG_LINKS to configure in
> > > >    order to support VPATH builds with FreeBSD make (yuck^2, since
> > > >    that means keeping track of renamed files when object-specific
> > > >    flags are involved).
> > > >
> > Hmm... maybe a variation of this might be a good solution after all, at
> > least for FreeBSD make: the special `.BEGIN' target[1][2] might be used
> > to copy Yacc-derived *.c and *.h files from $(srcdir) to $(builddir),
> > unless they are already there.
> 
> How do you ensure that you don't overwrite newer such files in the build
> tree?
>
Because we won't copy a file `foo.c' from srcdir to builddir if a `foo.c'
already exists in builddir -- even if this `foo.c' in builddir is older
than the one in the srcdir.  This should be good enough for the intended
usages (VPATH builds with FreeBSD make), no?

> Please note that 'test' and 'ls' might not have the same time
> stamp reading granularity that 'make' might have.  Well, at least in
> general, this might be less of a problem on FreeBSD specifically.
>
I don't plan to do any by-hand timestamp checking.

> Anyway, the above sounds a lot like a hack with possible traps, I'm not
> sure I like going that way.
>
But the good thing about this hack is that it's (at least, should be)
triggered only with BSD make, for which VPATH builds involving Yacc
files are already broken.

> > Note that the 'distcleancheck' target would still fail with this change,
> > but that's more of an annoyance rather than a serious bug (like the one
> > under discussion is IMHO).  And we could try to fix that later.
> 
> Well, regressions are generally worse than bugs.
>
Yes, but this wouldn't be a regression, since currently the 'distcleancheck'
target is already broken in VPATH builds with FreeBSD make.

> Generally, whatever solution you would like to propose, I would suggest
> that you could first try to create in a simple Makefile the rules as you
> think automake should produce them;
>
Good advice; and in fact, I've played this way with the testcase
`yacc-dist-nobuild.test' before advancing my proposal, to avoid losing
my time and yours.  See attachement.  It works with GNU make, Heirloom
make, FreeBSD make and NetBSD make.

> then we have something to digest
> (and possibly throw away) without having to invest a lot of work in a
> proper automake implementation.  It is often possible to rule out
> strategies quickly at such an early stage.
>
I heartily agree.

> Thanks,
> Ralf
> 

Thanks,
  Stefano

[yacc-freebsd-make-vpath-experiment.test (application/x-shellscript, inline)]

Message #20 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 7884 <at> debbugs.gnu.org
Subject: Re: bug#7884: Yacc and FreeBSD make in VPATH builds
Date: Thu, 27 Jan 2011 19:34:14 +0100

* Stefano Lattarini wrote on Thu, Jan 27, 2011 at 11:54:50AM CET:
> On Tuesday 25 January 2011, Ralf Wildenhues wrote:
> > * Stefano Lattarini wrote on Tue, Jan 25, 2011 at 08:52:56PM CET:
> > > Hmm... maybe a variation of this might be a good solution after all, at
> > > least for FreeBSD make: the special `.BEGIN' target[1][2] might be used
> > > to copy Yacc-derived *.c and *.h files from $(srcdir) to $(builddir),
> > > unless they are already there.
> > 
> > How do you ensure that you don't overwrite newer such files in the build
> > tree?
> >
> Because we won't copy a file `foo.c' from srcdir to builddir if a `foo.c'
> already exists in builddir -- even if this `foo.c' in builddir is older
> than the one in the srcdir.  This should be good enough for the intended
> usages (VPATH builds with FreeBSD make), no?

Hmm.

I must admit that I still don't like this a lot.  BSD is free software,
we shouldn't have to put ugly workarounds in place for it, and this
workaround does have some ugliness to it.  Here's a radically different
idea: get them to fix/improve their make.

What do you think about this?
If you like it, would you be willing to write a PR and/or patch?
(I could otherwise try, but I'm not sure I have time for a patch.)

Thanks,
Ralf

Message #23 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: bug-automake <at> gnu.org
Cc: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>, 7884 <at> debbugs.gnu.org
Subject: Re: bug#7884: Yacc and FreeBSD make in VPATH builds
Date: Thu, 27 Jan 2011 20:26:42 +0100

On Thursday 27 January 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Thu, Jan 27, 2011 at 11:54:50AM CET:
> > On Tuesday 25 January 2011, Ralf Wildenhues wrote:
> > > * Stefano Lattarini wrote on Tue, Jan 25, 2011 at 08:52:56PM CET:
> > > > Hmm... maybe a variation of this might be a good solution after all, at
> > > > least for FreeBSD make: the special `.BEGIN' target[1][2] might be used
> > > > to copy Yacc-derived *.c and *.h files from $(srcdir) to $(builddir),
> > > > unless they are already there.
> > > 
> > > How do you ensure that you don't overwrite newer such files in the build
> > > tree?
> > >
> > Because we won't copy a file `foo.c' from srcdir to builddir if a `foo.c'
> > already exists in builddir -- even if this `foo.c' in builddir is older
> > than the one in the srcdir.  This should be good enough for the intended
> > usages (VPATH builds with FreeBSD make), no?
> 
> Hmm.
> 
> I must admit that I still don't like this a lot.  BSD is free software,
> we shouldn't have to put ugly workarounds in place for it, and this
> workaround does have some ugliness to it.
>
But IMHO its ugliness pales when confronted with that of other
VPATH-related workarounds that are currently in automake:
   `test -f '$<' || echo $(srcdir)`/$<
And moreover, while this last ugliness is imposed to every make
implementation, even the well-behaved ones, the ugliness introduced
by my patch only affects the "offending" make implementations
(here, BSD make).

> Here's a radically different idea: get them to fix/improve their make.
> 
> What do you think about this?
>
That's surely good in the long run and for the future, but it won't help
with older versions of FreeBSD.

Also, I'm halfway through an automake patch on the lines I've sketched,
and I can say that it seems remarkably easy at this point ...

> If you like it, would you be willing to write a PR and/or patch?
>
I might try to write a PR if you don't have time, but I don't feel
qualified to write a patch (nor very motivated either, TBH).  Sorry.

> (I could otherwise try, but I'm not sure I have time for a patch.)
> 
> Thanks,
> Ralf
> 
> 

Regards,
   Stefano

Message #29 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: automake-patches <at> gnu.org
Cc: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>, 7884 <at> debbugs.gnu.org
Subject: [PATCH] yacc: fix VPATH builds with FreeBSD make (was: Re: bug#7884:
	Yacc and FreeBSD make in VPATH builds)
Date: Thu, 27 Jan 2011 21:15:36 +0100

[Message part 1 (text/plain, inline)]

On Thursday 27 January 2011, Stefano Lattarini wrote:
> Also, I'm halfway through an automake patch on the lines I've sketched,
> and I can say that it seems remarkably easy at this point ...
>
In fact it has been even easier than I thought, after the discovery that
the same 'Automake::Variable::define' call can be used to initialize the
value of a variable as well as to append to it.

The attached patch should fix automake bug#7884 (and now the test
`yacc-dist-nobuild.test' passes also with FreeBSD make).

OK for the 'yacc-work' branch?

Regards,
  Stefano

[0001-yacc-fix-VPATH-builds-with-FreeBSD-make.patch (text/x-patch, inline)]

From 95e09f4b2b52d0e4ede51e238c2f8fce684fd085 Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Date: Thu, 27 Jan 2011 20:30:56 +0100
Subject: [PATCH] yacc: fix VPATH builds with FreeBSD make

Fixes automake bug#7884.

* lib/am/yacc.am (.BEGIN): New target, hack to support VPATH
builds with FreeBSD make.
* automake.in (lang_yacc_target_hook): Now update the private
make variable `am__yacc_c_outputs', used by the target above.
* tests/yacc-dist-nobuild.test: Update.
* tests/yacc-subdir-objects-vpath.test: New test.
* tests/yacc-cond.test: Likewise.
* tests/yacc-cond-minimal.test: New test, still failing with
BSD make (but it's not a big deal).
* tests/Makefile.am (TESTS): Update.
---
 ChangeLog                            |   15 ++++
 automake.in                          |    7 ++
 lib/am/yacc.am                       |   20 +++++
 tests/Makefile.am                    |    3 +
 tests/Makefile.in                    |    3 +
 tests/yacc-cond-minimal.test         |   81 ++++++++++++++++++++
 tests/yacc-cond.test                 |  122 ++++++++++++++++++++++++++++++
 tests/yacc-dist-nobuild.test         |    7 +-
 tests/yacc-subdir-objects-vpath.test |  137 ++++++++++++++++++++++++++++++++++
 9 files changed, 392 insertions(+), 3 deletions(-)
 create mode 100755 tests/yacc-cond-minimal.test
 create mode 100755 tests/yacc-cond.test
 create mode 100755 tests/yacc-subdir-objects-vpath.test

diff --git a/ChangeLog b/ChangeLog
index 4470c28..93194b3 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,18 @@
+2011-01-27   Stefano Lattarini  <stefano.lattarini <at> gmail.com>
+
+	yacc: fix VPATH builds with FreeBSD make
+	Fixes automake bug#7884.
+	* lib/am/yacc.am (.BEGIN): New target, hack to support VPATH
+	builds with FreeBSD make.
+	* automake.in (lang_yacc_target_hook): Now update the private
+	make variable `am__yacc_c_outputs', used by the target above.
+	* tests/yacc-dist-nobuild.test: Update.
+	* tests/yacc-subdir-objects-vpath.test: New test.
+	* tests/yacc-cond.test: Likewise.
+	* tests/yacc-cond-minimal.test: New test, still failing with
+	BSD make (but it's not a big deal).
+	* tests/Makefile.am (TESTS): Update.
+
 2011-01-22   Stefano Lattarini  <stefano.lattarini <at> gmail.com>
 
 	configure: look for a yacc program to be used by the testsuite
diff --git a/automake.in b/automake.in
index fa458d6..c72e0bc 100755
--- a/automake.in
+++ b/automake.in
@@ -6082,6 +6082,13 @@ sub lang_yacc_target_hook
 	  }
       }
 
+    # Required to have FreeBSD make working correctly in VPATH builds.
+    # FIXME: It would be nice to add a yacc-generated C file to
+    # $(am__yacc_c_outputs) only if it's required by the build, e.g.
+    # only if it's not declared in a false conditional ...
+    Automake::Variable::define ('am__yacc_c_outputs', VAR_AUTOMAKE, '+',
+                                TRUE, $output, '', INTERNAL, VAR_PRETTY);
+
     if ($yflags_contains_minus_d)
       {
 	(my $output_base = $output) =~ s/$KNOWN_EXTENSIONS_PATTERN$//;
diff --git a/lib/am/yacc.am b/lib/am/yacc.am
index 6d35cd4..8de0edc 100644
--- a/lib/am/yacc.am
+++ b/lib/am/yacc.am
@@ -46,3 +46,23 @@ endif %?MAINTAINER-MODE%
 ?!GENERIC??DIST_SOURCE?	$(am__skipyacc) \
 ## For non-suffix rules, we must emulate a VPATH search on %SOURCE%.
 ?!GENERIC?	$(SHELL) $(YLWRAP) `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE% y.tab.c %OBJ% y.tab.h %BASE%.h y.output %BASE%.output -- %COMPILE%
+
+## This is releated to automake bug#7884.
+## The automake-generated Makefiles usually distribute the yacc-derived C
+## files, so that the final user won't need a working yacc to build the
+## package.  But for some reason, in a VPATH build, FreeBSD make does not
+## see (or decides not to use) the yacc-derived and distributed C files
+## which are in the source directory, and tries to rebuild them (in the
+## build directory).  This might fail if the user hasn't a yacc program
+## installed, or even if he has one which is uncompatible with that used
+## by the developer.  So we use the following hack to ensure that, with
+## BSD make, the distributed, yacc-generated C sources are copied from
+## the source directory into the build directory, before starting the
+## "real" make operations.
+.BEGIN:
+	@for f in $(am__yacc_c_outputs); do \
+	   if test ! -r $$f && test -f $(srcdir)/$$f; then \
+	     echo " cp -p $(srcdir)/$$f $$f"; \
+	     cp -p $(srcdir)/$$f $$f || exit 1; \
+	   else :; fi; \
+	done;
diff --git a/tests/Makefile.am b/tests/Makefile.am
index e639b6c..d595c46 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -807,8 +807,11 @@ yacc6.test \
 yacc7.test \
 yacc8.test \
 yaccdry.test \
+yacc-cond.test \
+yacc-cond-minimal.test \
 yacc-dist-nobuild.test \
 yacc-nodist.test \
+yacc-subdir-objects-vpath.test \
 yaccpp.test \
 yaccvpath.test \
 yacc-d-vpath.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 25fc4a8..74df65d 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1075,8 +1075,11 @@ yacc6.test \
 yacc7.test \
 yacc8.test \
 yaccdry.test \
+yacc-cond.test \
+yacc-cond-minimal.test \
 yacc-dist-nobuild.test \
 yacc-nodist.test \
+yacc-subdir-objects-vpath.test \
 yaccpp.test \
 yaccvpath.test \
 yacc-d-vpath.test \
diff --git a/tests/yacc-cond-minimal.test b/tests/yacc-cond-minimal.test
new file mode 100755
index 0000000..2956f01
--- /dev/null
+++ b/tests/yacc-cond-minimal.test
@@ -0,0 +1,81 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that if a yacc-generated C source or header is not needed in the
+# build tree, make won't generate not copy it there.
+
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AM_CONDITIONAL([COND], [:])
+AC_PROG_CC
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+bin_PROGRAMS = foo bar
+if COND
+foo_SOURCES = main.c
+else
+foo_SOURCES = parse.y main.c
+endif
+
+bar_SOURCES = $(foo_SOURCES)
+bar_YFLAGS = -d
+END
+
+cat > main.c << 'END'
+int main () { return 0; }
+END
+
+mkdir bin
+cat > bin/yacc <<'END'
+#!/bin/sh
+echo "$0 invoked, shouldn't happen!" >&2
+exit 1
+END
+cp bin/yacc bin/bison
+chmod a+x bin/yacc bin/bison
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH
+
+YACC=yacc BISON=bison
+export YACC BISON
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+mkdir build
+cd build
+../configure
+$MAKE
+test ! -r parse.c
+test ! -r bar-parse.c
+test ! -r bar-parse.h
+$MAKE clean
+$sleep
+: > ../parse.c
+: > ../bar-parse.c
+: > ../bar-parse.h
+$MAKE
+test ! -r parse.c
+test ! -r bar-parse.c
+test ! -r bar-parse.h
+
+:
diff --git a/tests/yacc-cond.test b/tests/yacc-cond.test
new file mode 100755
index 0000000..8c7729b
--- /dev/null
+++ b/tests/yacc-cond.test
@@ -0,0 +1,122 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check interaction of Yacc (without the `-d' option) and conditionals.
+
+required=yacc
+. ./defs || Exit 1
+
+set -e
+
+distdir=$me-1.0
+
+cat >> configure.in << 'END'
+AM_CONDITIONAL([COND], [:])
+AC_PROG_CC
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+bin_PROGRAMS = foobar zardoz
+
+AM_YFLAGS = -d
+
+if COND
+foobar_SOURCES = parse.y parse.h
+else
+foobar_SOURCES = parse.y main.c
+endif
+
+if COND
+zardoz_SOURCES = main2.c
+else
+zardoz_SOURCES = parse2.y parse2.h main2.c
+endif
+
+.PHONY: test-objs
+test-objs:
+	test -f parse.$(OBJEXT)
+	test ! -r parse2.$(OBJEXT)
+	test -f main2.$(OBJEXT)
+	test ! -r main.$(OBJEXT)
+END
+
+cat > parse2.y << 'END'
+%{
+int yylex () { return 0; }
+void yyerror (char *s) {}
+%}
+%%
+foobar : 'f' 'o' 'o' 'b' 'a' 'r' {};
+%%
+END
+
+cat > main.c << 'END'
+int main (void) { return 0; }
+END
+
+cp main.c main2.c
+
+cat parse2.y main.c > parse.y
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+$MAKE
+ls -l
+test -f parse.c
+test -f parse.h
+test ! -r parse2.c
+test ! -r parse2.h
+$MAKE test-objs
+
+$MAKE distdir
+ls -l $distdir
+test -f $distdir/parse.c
+test -f $distdir/parse.h
+# For all we know, the `parse2.[ch]' generated files might be required by
+# the end user (if he enable the COND conditional), so we must distribute
+# them.
+test -f $distdir/parse2.c
+test -f $distdir/parse2.h
+
+# Now check that yacc is not needed to build from a distribution tarball.
+
+chmod -R a-w $distdir
+
+mkdir bin
+cat > bin/yacc <<'END'
+#!/bin/sh
+echo "$0 invoked, shouldn't happen!" >&2
+exit 1
+END
+cp bin/yacc bin/bison
+chmod a+x bin/yacc bin/bison
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH
+
+YACC=yacc BISON=bison
+export YACC BISON
+
+mkdir build
+cd build
+../$distdir/configure
+$MAKE
+$MAKE test-objs
+
+:
diff --git a/tests/yacc-dist-nobuild.test b/tests/yacc-dist-nobuild.test
index 9061f57..26f5924 100755
--- a/tests/yacc-dist-nobuild.test
+++ b/tests/yacc-dist-nobuild.test
@@ -79,9 +79,10 @@ cd build
 $MAKE
 
 # Sanity check.
-chmod u+w ../$distdir
-rm -f ../$distdir/parse.c
-chmod a-w ../$distdir
+$sleep
+chmod u+w ../$distdir/parse.y
+echo >>  ../$distdir/parse.y
+chmod a-w ../$distdir/parse.y
 $MAKE >out 2>&1 && { cat out; Exit 1; }
 cat out
 $FGREP parse.c out
diff --git a/tests/yacc-subdir-objects-vpath.test b/tests/yacc-subdir-objects-vpath.test
new file mode 100755
index 0000000..c427e47
--- /dev/null
+++ b/tests/yacc-subdir-objects-vpath.test
@@ -0,0 +1,137 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Some tests for subdir parsers in VPATH builds.
+# See also test `yacc8.test'.
+
+required=yacc
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AM_PROG_CC_C_O
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = subdir-objects
+bin_PROGRAMS = foo/bar baz/qux
+foo_bar_SOURCES = foo/parse.y
+baz_qux_SOURCES = baz/parse.y baz/main.c baz/parse.h
+AM_YFLAGS = -d
+
+.PHONY: test-com test1 test2
+test-com:
+	ls -l . foo $(srcdir) $(srcdir)/foo
+	test ! -r parse.h
+	test ! -r parse.c
+	test ! -r parse.$(OBJEXT)
+	test -f foo/parse.$(OBJEXT)
+	test -f baz/parse.$(OBJEXT)
+test1: test-com
+	test -f foo/parse.c
+	test -f foo/parse.h
+	test -f baz/parse.c
+	test -f baz/parse.h
+test2: test-com
+	test -f $(srcdir)/foo/parse.c
+	test -f $(srcdir)/foo/parse.h
+	test -f $(srcdir)/baz/parse.c
+	test -f $(srcdir)/baz/parse.h
+END
+
+mkdir foo
+cat > foo/parse.y << 'END'
+%{
+int yylex () { return 0; }
+void yyerror (char *s) { return; }
+%}
+%%
+x : 'x' {};
+%%
+int main  (void) { return 0; }
+END
+
+mkdir baz
+cat > baz/parse.y << 'END'
+%{
+int yylex () { return 0; }
+void yyerror (char *s) { return; }
+%}
+%token BAZBAZ
+%%
+x : 'x' {};
+%%
+END
+cat > baz/main.c << 'END'
+#include "baz/parse.h"
+int main (void)
+{
+  return (BAZBAZ);
+}
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+# Try a from-scratch VPATH build.
+mkdir sub1
+cd sub1
+../configure
+$MAKE
+$MAKE test1
+
+# Also, let's see if we can recover from removal of generated header
+# files.
+rm -f foo/parse.h baz/parse.h
+$MAKE foo/parse.h baz/parse.h
+test -f foo/parse.h
+test -f baz/parse.h
+$MAKE clean
+rm -f foo/parse.h baz/parse.h
+$MAKE
+test -f baz/parse.h
+# foo/parse.h is not declares in any *_SOURCES, nor #included by any
+# C file, so that it shouldn't be rebuilt by "make all".
+test ! -r foo/parse.h
+
+cd ..
+
+# Generate yacc-derived files with an in-tree build.
+./configure
+$MAKE foo/parse.c foo/parse.h baz/parse.c baz/parse.h
+$MAKE distclean
+ls -l . foo
+test -f foo/parse.c
+test -f foo/parse.h
+test -f baz/parse.c
+test -f baz/parse.h
+
+# Now try a VPATH build with yacc-generated files already present in
+# the srcdir.
+YACC=false BISON=false
+export YACC BISON
+mkdir sub2
+cd sub2
+../configure
+$MAKE
+$MAKE test2
+
+:
-- 
1.7.2.3

Message #32 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 7884 <at> debbugs.gnu.org
Subject: Re: bug#7884: Yacc and FreeBSD make in VPATH builds
Date: Thu, 27 Jan 2011 23:38:09 +0100

* Stefano Lattarini wrote on Thu, Jan 27, 2011 at 08:26:42PM CET:
> On Thursday 27 January 2011, Ralf Wildenhues wrote:
> > I must admit that I still don't like this a lot.  BSD is free software,
> > we shouldn't have to put ugly workarounds in place for it, and this
> > workaround does have some ugliness to it.
> >
> But IMHO its ugliness pales when confronted with that of other
> VPATH-related workarounds that are currently in automake:
>    `test -f '$<' || echo $(srcdir)`/$<

Sure.  But first off, one ugliness should not hold as excuse for
another, second, we cannot do much about IRIX/Sun make (well, at
least we could not, at the time).

> And moreover, while this last ugliness is imposed to every make
> implementation, even the well-behaved ones, the ugliness introduced
> by my patch only affects the "offending" make implementations
> (here, BSD make).

(FWIW, it's on my table to eliminate the above for GNU make, and
in turn to maybe fix depend2 to use it everywhere for non-GNU, as it
still lacks uses cases.)

> > Here's a radically different idea: get them to fix/improve their make.
> > 
> > What do you think about this?
> >
> That's surely good in the long run and for the future, but it won't help
> with older versions of FreeBSD.
> 
> Also, I'm halfway through an automake patch on the lines I've sketched,
> and I can say that it seems remarkably easy at this point ...

How do you ensure in your patch that 'make clean' does not create files?
Likewise distdir?  Does 'make -n' create files (for any target)?

Thanks,
Ralf

Message #35 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: automake-patches <at> gnu.org
Cc: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>, 7884 <at> debbugs.gnu.org
Subject: Re: [PATCH] yacc: fix VPATH builds with FreeBSD make (was: Re:
	bug#7884: Yacc and FreeBSD make in VPATH builds)
Date: Fri, 28 Jan 2011 13:51:40 +0100

[Message part 1 (text/plain, inline)]

On Thursday 27 January 2011, Ralf Wildenhues wrote:
> Does 'make -n' create files (for any target)?
>
No, and I've updated the `yaccdry.test' testcase to also check for this
(see attached amended patch).

> How do you ensure in your patch that 'make distdir' does not create files?
>
Well, it does, but I don't see any problem with this.  It's perfectly normal
for 'make distdir' to create files in the build directory, no?  And anyway,
this behaviour was pre-existint with FreeBSD make, just in a much worser
form ('yacc' was used to recreate the files, instead of a simple "cp -p").

> How do you ensure in your patch that 'make clean' does not create files?
>
In fact, it does (good catch, BTW); see new testcase in the attached patch.
Yes, this is clearly a wart, but a minor one, and having it is IMHO a price
worth paying in order to have VPATH builds work on FreeBSD.

BTW, if you decide to get rid of said wart in a follow-up patch, that can
probably be done using the FreeBSD make special variable `.TARGETS' (akin
to GNU make's `MAKECMDGOALS').

Thanks,
  Stefano

[0001-yacc-fix-VPATH-builds-with-FreeBSD-make.patch (text/x-patch, inline)]

From e5b393aaf4fe417e8cc8cae8056829f9083cc51c Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Date: Thu, 27 Jan 2011 20:30:56 +0100
Subject: [PATCH] yacc: fix VPATH builds with FreeBSD make

Fixes automake bug#7884.

* lib/am/yacc.am (.BEGIN): New target, hack to support VPATH
builds with FreeBSD make.
* automake.in (lang_yacc_target_hook): Now update the private
make variable `am__yacc_c_outputs', used by the target above.
* tests/yacc-dist-nobuild.test: Update.
* tests/yaccdry.test: Extend.
* tests/yaccvpath.test: Likewise.
* tests/yacc-d-vpath.test: Likewise.
* tests/yacc-subdir-objects-vpath.test: New test.
* tests/yacc-cond.test: Likewise.
* tests/yacc-cond-minimal.test: New test, still failing with
BSD make.
* tests/yaccdry2.test: Likewise.
* tests/Makefile.am (TESTS): Update.
---
 ChangeLog                            |   19 +++++
 automake.in                          |    7 ++
 lib/am/yacc.am                       |   20 +++++
 tests/Makefile.am                    |    4 +
 tests/Makefile.in                    |    4 +
 tests/yacc-cond-minimal.test         |   81 ++++++++++++++++++++
 tests/yacc-cond.test                 |  122 ++++++++++++++++++++++++++++++
 tests/yacc-d-vpath.test              |   14 ++++
 tests/yacc-dist-nobuild.test         |    7 +-
 tests/yacc-subdir-objects-vpath.test |  137 ++++++++++++++++++++++++++++++++++
 tests/yaccdry.test                   |   39 ++++++++++
 tests/yaccdry2.test                  |   77 +++++++++++++++++++
 tests/yaccvpath.test                 |   12 +++
 13 files changed, 540 insertions(+), 3 deletions(-)
 create mode 100755 tests/yacc-cond-minimal.test
 create mode 100755 tests/yacc-cond.test
 create mode 100755 tests/yacc-subdir-objects-vpath.test
 create mode 100755 tests/yaccdry2.test

diff --git a/ChangeLog b/ChangeLog
index 4470c28..da291f3 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,22 @@
+2011-01-28   Stefano Lattarini  <stefano.lattarini <at> gmail.com>
+
+	yacc: fix VPATH builds with FreeBSD make
+	Fixes automake bug#7884.
+	* lib/am/yacc.am (.BEGIN): New target, hack to support VPATH
+	builds with FreeBSD make.
+	* automake.in (lang_yacc_target_hook): Now update the private
+	make variable `am__yacc_c_outputs', used by the target above.
+	* tests/yacc-dist-nobuild.test: Update.
+	* tests/yaccdry.test: Extend.
+	* tests/yaccvpath.test: Likewise.
+	* tests/yacc-d-vpath.test: Likewise.
+	* tests/yacc-subdir-objects-vpath.test: New test.
+	* tests/yacc-cond.test: Likewise.
+	* tests/yacc-cond-minimal.test: New test, still failing with
+	BSD make.
+	* tests/yaccdry2.test: Likewise.
+	* tests/Makefile.am (TESTS): Update.
+
 2011-01-22   Stefano Lattarini  <stefano.lattarini <at> gmail.com>
 
 	configure: look for a yacc program to be used by the testsuite
diff --git a/automake.in b/automake.in
index fa458d6..c72e0bc 100755
--- a/automake.in
+++ b/automake.in
@@ -6082,6 +6082,13 @@ sub lang_yacc_target_hook
 	  }
       }
 
+    # Required to have FreeBSD make working correctly in VPATH builds.
+    # FIXME: It would be nice to add a yacc-generated C file to
+    # $(am__yacc_c_outputs) only if it's required by the build, e.g.
+    # only if it's not declared in a false conditional ...
+    Automake::Variable::define ('am__yacc_c_outputs', VAR_AUTOMAKE, '+',
+                                TRUE, $output, '', INTERNAL, VAR_PRETTY);
+
     if ($yflags_contains_minus_d)
       {
 	(my $output_base = $output) =~ s/$KNOWN_EXTENSIONS_PATTERN$//;
diff --git a/lib/am/yacc.am b/lib/am/yacc.am
index 6d35cd4..8de0edc 100644
--- a/lib/am/yacc.am
+++ b/lib/am/yacc.am
@@ -46,3 +46,23 @@ endif %?MAINTAINER-MODE%
 ?!GENERIC??DIST_SOURCE?	$(am__skipyacc) \
 ## For non-suffix rules, we must emulate a VPATH search on %SOURCE%.
 ?!GENERIC?	$(SHELL) $(YLWRAP) `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE% y.tab.c %OBJ% y.tab.h %BASE%.h y.output %BASE%.output -- %COMPILE%
+
+## This is releated to automake bug#7884.
+## The automake-generated Makefiles usually distribute the yacc-derived C
+## files, so that the final user won't need a working yacc to build the
+## package.  But for some reason, in a VPATH build, FreeBSD make does not
+## see (or decides not to use) the yacc-derived and distributed C files
+## which are in the source directory, and tries to rebuild them (in the
+## build directory).  This might fail if the user hasn't a yacc program
+## installed, or even if he has one which is uncompatible with that used
+## by the developer.  So we use the following hack to ensure that, with
+## BSD make, the distributed, yacc-generated C sources are copied from
+## the source directory into the build directory, before starting the
+## "real" make operations.
+.BEGIN:
+	@for f in $(am__yacc_c_outputs); do \
+	   if test ! -r $$f && test -f $(srcdir)/$$f; then \
+	     echo " cp -p $(srcdir)/$$f $$f"; \
+	     cp -p $(srcdir)/$$f $$f || exit 1; \
+	   else :; fi; \
+	done;
diff --git a/tests/Makefile.am b/tests/Makefile.am
index e639b6c..d0e3e31 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -807,8 +807,12 @@ yacc6.test \
 yacc7.test \
 yacc8.test \
 yaccdry.test \
+yaccdry2.test \
+yacc-cond.test \
+yacc-cond-minimal.test \
 yacc-dist-nobuild.test \
 yacc-nodist.test \
+yacc-subdir-objects-vpath.test \
 yaccpp.test \
 yaccvpath.test \
 yacc-d-vpath.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 25fc4a8..7eca925 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1075,8 +1075,12 @@ yacc6.test \
 yacc7.test \
 yacc8.test \
 yaccdry.test \
+yaccdry2.test \
+yacc-cond.test \
+yacc-cond-minimal.test \
 yacc-dist-nobuild.test \
 yacc-nodist.test \
+yacc-subdir-objects-vpath.test \
 yaccpp.test \
 yaccvpath.test \
 yacc-d-vpath.test \
diff --git a/tests/yacc-cond-minimal.test b/tests/yacc-cond-minimal.test
new file mode 100755
index 0000000..2956f01
--- /dev/null
+++ b/tests/yacc-cond-minimal.test
@@ -0,0 +1,81 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that if a yacc-generated C source or header is not needed in the
+# build tree, make won't generate not copy it there.
+
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AM_CONDITIONAL([COND], [:])
+AC_PROG_CC
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+bin_PROGRAMS = foo bar
+if COND
+foo_SOURCES = main.c
+else
+foo_SOURCES = parse.y main.c
+endif
+
+bar_SOURCES = $(foo_SOURCES)
+bar_YFLAGS = -d
+END
+
+cat > main.c << 'END'
+int main () { return 0; }
+END
+
+mkdir bin
+cat > bin/yacc <<'END'
+#!/bin/sh
+echo "$0 invoked, shouldn't happen!" >&2
+exit 1
+END
+cp bin/yacc bin/bison
+chmod a+x bin/yacc bin/bison
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH
+
+YACC=yacc BISON=bison
+export YACC BISON
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+mkdir build
+cd build
+../configure
+$MAKE
+test ! -r parse.c
+test ! -r bar-parse.c
+test ! -r bar-parse.h
+$MAKE clean
+$sleep
+: > ../parse.c
+: > ../bar-parse.c
+: > ../bar-parse.h
+$MAKE
+test ! -r parse.c
+test ! -r bar-parse.c
+test ! -r bar-parse.h
+
+:
diff --git a/tests/yacc-cond.test b/tests/yacc-cond.test
new file mode 100755
index 0000000..1ebce92
--- /dev/null
+++ b/tests/yacc-cond.test
@@ -0,0 +1,122 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check interaction of Yacc and conditionals.
+
+required=yacc
+. ./defs || Exit 1
+
+set -e
+
+distdir=$me-1.0
+
+cat >> configure.in << 'END'
+AM_CONDITIONAL([COND], [:])
+AC_PROG_CC
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+bin_PROGRAMS = foobar zardoz
+
+AM_YFLAGS = -d
+
+if COND
+foobar_SOURCES = parse.y parse.h
+else
+foobar_SOURCES = parse.y main.c
+endif
+
+if COND
+zardoz_SOURCES = main2.c
+else
+zardoz_SOURCES = parse2.y parse2.h main2.c
+endif
+
+.PHONY: test-objs
+test-objs:
+	test -f parse.$(OBJEXT)
+	test ! -r parse2.$(OBJEXT)
+	test -f main2.$(OBJEXT)
+	test ! -r main.$(OBJEXT)
+END
+
+cat > parse2.y << 'END'
+%{
+int yylex () { return 0; }
+void yyerror (char *s) {}
+%}
+%%
+foobar : 'f' 'o' 'o' 'b' 'a' 'r' {};
+%%
+END
+
+cat > main.c << 'END'
+int main (void) { return 0; }
+END
+
+cp main.c main2.c
+
+cat parse2.y main.c > parse.y
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+$MAKE
+ls -l
+test -f parse.c
+test -f parse.h
+test ! -r parse2.c
+test ! -r parse2.h
+$MAKE test-objs
+
+$MAKE distdir
+ls -l $distdir
+test -f $distdir/parse.c
+test -f $distdir/parse.h
+# For all we know, the `parse2.[ch]' generated files might be required by
+# the end user (if he enable the COND conditional), so we must distribute
+# them.
+test -f $distdir/parse2.c
+test -f $distdir/parse2.h
+
+# Now check that yacc is not needed to build from a distribution tarball.
+
+chmod -R a-w $distdir
+
+mkdir bin
+cat > bin/yacc <<'END'
+#!/bin/sh
+echo "$0 invoked, shouldn't happen!" >&2
+exit 1
+END
+cp bin/yacc bin/bison
+chmod a+x bin/yacc bin/bison
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH
+
+YACC=yacc BISON=bison
+export YACC BISON
+
+mkdir build
+cd build
+../$distdir/configure
+$MAKE
+$MAKE test-objs
+
+:
diff --git a/tests/yacc-d-vpath.test b/tests/yacc-d-vpath.test
index bb09990..4d2fed8 100755
--- a/tests/yacc-d-vpath.test
+++ b/tests/yacc-d-vpath.test
@@ -109,4 +109,18 @@ $MAKE distdir
 $FGREP maude $distdir/parse.c
 $FGREP MAUDE $distdir/parse.h
 
+# Now check that `make maintainer-clean' removes all yacc-derived
+# files (if any) created in the builddir.  Try it two times, to
+# be stricter in face of possible `.BEGIN' targets executed by BSD
+# make.
+cp config.status config.sav
+$MAKE maintainer-clean
+test ! -r parse.c
+test ! -r parse.h
+# Recreate Makefile, depfiles, etc.
+mv -f config.sav config.status; ./config.status
+$MAKE maintainer-clean
+test ! -r parse.c
+test ! -r parse.h
+
 :
diff --git a/tests/yacc-dist-nobuild.test b/tests/yacc-dist-nobuild.test
index 9061f57..26f5924 100755
--- a/tests/yacc-dist-nobuild.test
+++ b/tests/yacc-dist-nobuild.test
@@ -79,9 +79,10 @@ cd build
 $MAKE
 
 # Sanity check.
-chmod u+w ../$distdir
-rm -f ../$distdir/parse.c
-chmod a-w ../$distdir
+$sleep
+chmod u+w ../$distdir/parse.y
+echo >>  ../$distdir/parse.y
+chmod a-w ../$distdir/parse.y
 $MAKE >out 2>&1 && { cat out; Exit 1; }
 cat out
 $FGREP parse.c out
diff --git a/tests/yacc-subdir-objects-vpath.test b/tests/yacc-subdir-objects-vpath.test
new file mode 100755
index 0000000..c427e47
--- /dev/null
+++ b/tests/yacc-subdir-objects-vpath.test
@@ -0,0 +1,137 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Some tests for subdir parsers in VPATH builds.
+# See also test `yacc8.test'.
+
+required=yacc
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AM_PROG_CC_C_O
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = subdir-objects
+bin_PROGRAMS = foo/bar baz/qux
+foo_bar_SOURCES = foo/parse.y
+baz_qux_SOURCES = baz/parse.y baz/main.c baz/parse.h
+AM_YFLAGS = -d
+
+.PHONY: test-com test1 test2
+test-com:
+	ls -l . foo $(srcdir) $(srcdir)/foo
+	test ! -r parse.h
+	test ! -r parse.c
+	test ! -r parse.$(OBJEXT)
+	test -f foo/parse.$(OBJEXT)
+	test -f baz/parse.$(OBJEXT)
+test1: test-com
+	test -f foo/parse.c
+	test -f foo/parse.h
+	test -f baz/parse.c
+	test -f baz/parse.h
+test2: test-com
+	test -f $(srcdir)/foo/parse.c
+	test -f $(srcdir)/foo/parse.h
+	test -f $(srcdir)/baz/parse.c
+	test -f $(srcdir)/baz/parse.h
+END
+
+mkdir foo
+cat > foo/parse.y << 'END'
+%{
+int yylex () { return 0; }
+void yyerror (char *s) { return; }
+%}
+%%
+x : 'x' {};
+%%
+int main  (void) { return 0; }
+END
+
+mkdir baz
+cat > baz/parse.y << 'END'
+%{
+int yylex () { return 0; }
+void yyerror (char *s) { return; }
+%}
+%token BAZBAZ
+%%
+x : 'x' {};
+%%
+END
+cat > baz/main.c << 'END'
+#include "baz/parse.h"
+int main (void)
+{
+  return (BAZBAZ);
+}
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+# Try a from-scratch VPATH build.
+mkdir sub1
+cd sub1
+../configure
+$MAKE
+$MAKE test1
+
+# Also, let's see if we can recover from removal of generated header
+# files.
+rm -f foo/parse.h baz/parse.h
+$MAKE foo/parse.h baz/parse.h
+test -f foo/parse.h
+test -f baz/parse.h
+$MAKE clean
+rm -f foo/parse.h baz/parse.h
+$MAKE
+test -f baz/parse.h
+# foo/parse.h is not declares in any *_SOURCES, nor #included by any
+# C file, so that it shouldn't be rebuilt by "make all".
+test ! -r foo/parse.h
+
+cd ..
+
+# Generate yacc-derived files with an in-tree build.
+./configure
+$MAKE foo/parse.c foo/parse.h baz/parse.c baz/parse.h
+$MAKE distclean
+ls -l . foo
+test -f foo/parse.c
+test -f foo/parse.h
+test -f baz/parse.c
+test -f baz/parse.h
+
+# Now try a VPATH build with yacc-generated files already present in
+# the srcdir.
+YACC=false BISON=false
+export YACC BISON
+mkdir sub2
+cd sub2
+../configure
+$MAKE
+$MAKE test2
+
+:
diff --git a/tests/yaccdry.test b/tests/yaccdry.test
index d30fb80..8eebc67 100755
--- a/tests/yaccdry.test
+++ b/tests/yaccdry.test
@@ -21,6 +21,8 @@ required=yacc
 
 set -e
 
+distdir=$me-1.0
+
 cat >> configure.in << 'END'
 AC_PROG_CC
 AC_PROG_YACC
@@ -49,12 +51,49 @@ END
 $ACLOCAL
 $AUTOMAKE --add-missing
 $AUTOCONF
+
+# Try an in-tree build first.
+
 ./configure
 $MAKE
 
 rm -f parse.h
+
+$MAKE -n
+test -f parse.c
+test ! -f parse.h
+
 $MAKE -n parse.h
 test -f parse.c
 test ! -f parse.h
 
+# Now try VPATH build.
+
+$MAKE distdir
+mkdir build
+cd build
+../$distdir/configure
+
+$MAKE -n
+test ! -f parse.c
+test ! -f parse.h
+
+$MAKE -n parse.h
+test ! -f parse.c
+test ! -f parse.h
+
+rm -f ../$distdir/parse.h
+
+$MAKE -n
+test ! -f parse.c
+test ! -f parse.h
+test ! -f ../$distdir/parse.h
+test -f ../$distdir/parse.c
+
+$MAKE -n parse.h
+test ! -f parse.c
+test ! -f parse.h
+test ! -f ../$distdir/parse.h
+test -f ../$distdir/parse.c
+
 :
diff --git a/tests/yaccdry2.test b/tests/yaccdry2.test
new file mode 100755
index 0000000..22902af
--- /dev/null
+++ b/tests/yaccdry2.test
@@ -0,0 +1,77 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that, in a VPATH build, no yacc-derived C file is created in
+# the build directory by targets like "make clean".
+
+. ./defs || Exit 1
+
+set -e
+
+unset YACC || :
+
+distdir=$me-1.0
+
+cat > my-yacc <<'END'
+#!/bin/sh
+echo 'int main () { return 0; }' > y.tab.c
+: > y.tab.h
+END
+chmod a+x my-yacc
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AC_SUBST([YACC], ['$(abs_builddir)/my-yacc'])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AM_YFLAGS = -d
+bin_PROGRAMS = foo
+foo_SOURCES = parse.y
+END
+
+: > parse.y
+
+$ACLOCAL
+$AUTOMAKE --add-missing
+$AUTOCONF
+
+# From-scratch VPATH build.
+mkdir build-scratch
+cd build-scratch
+../configure
+for target in mostlyclean clean distclean; do
+  $MAKE $target
+  test ! -f parse.c
+  test ! -f parse.h
+done
+
+cd ..
+./configure
+$MAKE distdir
+
+# VPATH build from distributed tarball.
+mkdir build-dist
+cd build-dist
+../$distdir/configure
+for target in mostlyclean clean distclean; do
+  $MAKE $target
+  test ! -f parse.c
+  test ! -f parse.h
+done
+
+:
diff --git a/tests/yaccvpath.test b/tests/yaccvpath.test
index 52092fa..853c5df 100755
--- a/tests/yaccvpath.test
+++ b/tests/yaccvpath.test
@@ -99,4 +99,16 @@ END
 $MAKE distdir
 $FGREP maude $distdir/parse.c
 
+# Now check that `make maintainer-clean' removes all yacc-derived
+# files (if any) created in the builddir.  Try it two times, to
+# be stricter in face of possible `.BEGIN' targets executed by BSD
+# make.
+cp config.status config.sav
+$MAKE maintainer-clean
+test ! -r parse.c
+# Recreate Makefile, depfiles, etc.
+mv -f config.sav config.status; ./config.status
+$MAKE maintainer-clean
+test ! -r parse.c
+
 :
-- 
1.7.2.3

Message #38 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: 7884 <at> debbugs.gnu.org
Subject: Re: bug#7884: Yacc and FreeBSD make in VPATH builds
Date: Fri, 28 Jan 2011 21:55:39 +0100

Hello automakers.

JFTR: I've found a prior report of this same issue in the automake
mailing list archives, and I think it could be useful to link it
here, for completeness:
 <http://lists.gnu.org/archive/html/automake/2007-11/msg00058.html>

Regards,
  Stefano

Message #41 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: automake-patches <at> gnu.org, 7884 <at> debbugs.gnu.org
Subject: Re: [PATCH] yacc: fix VPATH builds with FreeBSD make
Date: Sat, 29 Jan 2011 01:17:12 +0100

* Stefano Lattarini wrote on Fri, Jan 28, 2011 at 01:51:40PM CET:
> On Thursday 27 January 2011, Ralf Wildenhues wrote:
> > Does 'make -n' create files (for any target)?
> >
> No, and I've updated the `yaccdry.test' testcase to also check for this
> (see attached amended patch).

Thanks.

> > How do you ensure in your patch that 'make distdir' does not create files?
> >
> Well, it does, but I don't see any problem with this.

Ah, you're right.  I was confused here.

> > How do you ensure in your patch that 'make clean' does not create files?
> >
> In fact, it does (good catch, BTW); see new testcase in the attached patch.
> Yes, this is clearly a wart, but a minor one, and having it is IMHO a price
> worth paying in order to have VPATH builds work on FreeBSD.

Wait.  When you do '../configure && make clean', then your code creates
the files just before removing them again, no?  That is what I meant.
It is not a disqualifying problem that it does so, but it is not nice.

It becomes more of a problem when things like
  make tags
  make some-arbitrary-user-target

start creating files that they shouldn't have anything to do with.
OTOH,
  make some-prog-that-depends-on-yacc-output

should be built with updated yacc outputs.

> BTW, if you decide to get rid of said wart in a follow-up patch, that can
> probably be done using the FreeBSD make special variable `.TARGETS' (akin
> to GNU make's `MAKECMDGOALS').

I suppose, yes.

Another problem, which might be more serious:

Say I checkout a git tree of my project, do a VPATH build from it on
FreeBSD.  The next day someone has updated foo.y in git, I pull the new
version, type 'make' again inside the old build tree.  This fails to use
the code from the new foo.y code version (because at the time the .BEGIN
rule was invoked, the old $(srcdir)/foo.c was still in place).

I use such setups regularly.  I know they break when the source tree
changes are large, but still, with no source tree graph changes it is a
subtle failure that developers will rightly blame Automake for.  It can
be fixed by comparing time stamps, at the cost of now getting into the
time stamp business that I mentioned earlier.

Then, there is the issue of which make implementations support the
.TARGETS feature.  What if GNU make 3.84 starts supporting it?
Our makefiles then would suddenly cause regressions for GNU make users.

I guess my point is, I fear that your approach is fundamentally not
robust.  I acknowledge that BUILT_SOURCES is not robust either
($(BUILT_SOURCES) being updated routinely as part of 'all', 'check',
'install', but no other special targets).  But at least with dependency
tracking enabled, BUILT_SOURCES is typically needed only the first time
'make' is used in some build tree.

Cheers,
Ralf

Message #44 received at 7884 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
Cc: automake-patches <at> gnu.org, 7884 <at> debbugs.gnu.org
Subject: Re: [PATCH] yacc: fix VPATH builds with FreeBSD make
Date: Sat, 29 Jan 2011 11:31:15 +0100

On Saturday 29 January 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Fri, Jan 28, 2011 at 01:51:40PM CET:
> > On Thursday 27 January 2011, Ralf Wildenhues wrote:
> > > Does 'make -n' create files (for any target)?
> > >
> > No, and I've updated the `yaccdry.test' testcase to also check for this
> > (see attached amended patch).
> 
> Thanks.
> 
> > > How do you ensure in your patch that 'make distdir' does not create files?
> > >
> > Well, it does, but I don't see any problem with this.
> 
> Ah, you're right.  I was confused here.
> 
> > > How do you ensure in your patch that 'make clean' does not create files?
> > >
> > In fact, it does (good catch, BTW); see new testcase in the attached patch.
> > Yes, this is clearly a wart, but a minor one, and having it is IMHO a price
> > worth paying in order to have VPATH builds work on FreeBSD.
> 
> Wait.  When you do '../configure && make clean', then your code creates
> the files just before removing them again, no?
>
Actually no, it's worse, because the files are copied by "make clean", but
removed only by "make maintainer-clean".

> That is what I meant.
> It is not a disqualifying problem that it does so, but it is not nice.
>
See above.

> It becomes more of a problem when things like
>   make tags
>   make some-arbitrary-user-target
> 
> start creating files that they shouldn't have anything to do with.
> OTOH,
>   make some-prog-that-depends-on-yacc-output
> 
> should be built with updated yacc outputs.
> 
> > BTW, if you decide to get rid of said wart in a follow-up patch, that can
> > probably be done using the FreeBSD make special variable `.TARGETS' (akin
> > to GNU make's `MAKECMDGOALS').
> 
> I suppose, yes.
> 
> Another problem, which might be more serious:
> 
> Say I checkout a git tree of my project, do a VPATH build from it on
> FreeBSD.  The next day someone has updated foo.y in git, I pull the new
> version, type 'make' again inside the old build tree.  This fails to use
> the code from the new foo.y code version (because at the time the .BEGIN
> rule was invoked, the old $(srcdir)/foo.c was still in place).
>
No, it should work anyway.  I see three scenarious:

 1. You have a pre-existing `foo.c' in the builddir.  After you update
    the `foo.y' in the srcdir, you run "make".  The .BEGIN target will
    do nothing, because there's already a `foo.c' in the builddir; then,
    since this `foo.c' is older than the `foo.y' in the srcdir, make
    will rebuild it.  From there, everything should go as expected.

 2. You have a pre-existing `foo.c' in the srcdir, but not in the
    builddir.  You update the `foo.y' in the srcdir, which will make
    it newer than `foo.c'.  Then you run "make".  The .BEGIN target
    will copy the `foo.c' from the srcdir in the builddir, using the
    cp -p command; this means that the timestamp of `foo.c' will be
    preserved, so that the `foo.c' in the builddir will result older
    than the `foo.y' in the srcdir -- and will thus be rebuilt by
    make.  At this point, `foo.c' is really up-to-date with the new
    `foo.y', and from there, everything should go as expected.

 3. There's no pre-existing `foo.c' in the srcdir nor in the builddir.
    The .BEGIN target will do nothing, because there's not `foo.c' in
    the srcdir to copy.  Make will then create (in the builddir) the
    `foo.c' derived from `foo.y'.  From there, everything should go as
    expected.

> I use such setups regularly.  I know they break when the source tree
> changes are large, but still, with no source tree graph changes it is a
> subtle failure that developers will rightly blame Automake for.  It can
> be fixed by comparing time stamps, at the cost of now getting into the
> time stamp business that I mentioned earlier.
>
There should no need to; and the fact that the behaviour you want really
works should be already verified by the test `yaccvpath.test' (which you
are free to improve if you like, I obviously won't object ;-)

> Then, there is the issue of which make implementations support the
> .TARGETS feature.
>
You mean the ".BEGIN" feature here, right?

> What if GNU make 3.84 starts supporting it?
>
Then we start checking in the `.BEGIN' recipe that the currently-running
make is really a BSD make (e.g. by checking that ${.MAKE.PID} is
non-empty, or something like that).  If it's not a BSD make, we exit
early from the recipe.

But then, that wouldn't be really necessary, since the setup done by
the `.BEGIN' recipe shouldn't break builds for non-BSD makes: it's just
redundant for them.  So, even if GNU make 3.84 starts supporting the
`.BEGIN' target, Makefiles generated with automake 1.12 should still
be able to build packages using Yacc correctly.

> Our makefiles then would suddenly cause regressions for GNU make users.
>
How so exactly?

> I guess my point is, I fear that your approach is fundamentally not
> robust.  I acknowledge that BUILT_SOURCES is not robust either
> ($(BUILT_SOURCES) being updated routinely as part of 'all', 'check',
> 'install', but no other special targets).  But at least with dependency
> tracking enabled, BUILT_SOURCES is typically needed only the first time
> 'make' is used in some build tree.
>
Which BTW is also the way the `.BEGIN' recipe we're using should
behave, ideally ...

Hmm... Maybe having a similar (internal) variable `am__LINKED_SOURCES'
holding a list of files to be copied from srcdir to builddir *only for
targets 'all', 'check' and 'install'* might be a better stategy?
Should I try to pursue that?

> Cheers,
> Ralf
> 

Regards,
  Stefano

Message #49 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: bug-automake <at> gnu.org
Subject: Re: bug#7884: [PATCH] yacc: fix VPATH builds with FreeBSD make
Date: Mon, 16 Nov 2020 19:17:32 -0700

It seems something more needs to be done with this bug, but I don't know
what. I hope someone with more energy, expertise, and/or time can look
into it and find a resolution.

Marking it as severity normal (seems like it can't be crucially
important after languishing for N years) and with the "help" and
"confirmed" tags. --karl

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.