GNU bug report logs - #16008
semantic/grammar-wy.el cannot be bootstrapped

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 16008 in the body.
You can then email your comments to 16008 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #3 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: submit <at> debbugs.gnu.org
Subject: semantic/grammar-wy.el cannot be bootstrapped
Date: Fri, 29 Nov 2013 21:07:30 -0500

Package: emacs
Version: 24.3.50
User: emacs
Usertags: cedet

semantic/grammar-wy.el is a generated file and therefore ideally should
not be kept in the Emacs VCS.

It is generated by semantic/grammar.el.
semantic/grammar.el requires semantic/grammar-wy.el, therefore
it is not possible to bootstrap without semantic/grammar-wy.el already
present.

I should also note that the version of semantic/grammar-wy.el in the
Emacs trunk does not match up with its sources.

It contains:

  (require 'semantic)

Yet if you regenerate it, it contains:

  (require 'semantic/lex)
  (eval-when-compile (require 'semantic/bovine))

Message #6 received at 16008 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: 16008 <at> debbugs.gnu.org
Cc: cedet-devel <at> lists.sourceforge.net
Subject: Re: bug#16008: semantic/grammar-wy.el cannot be bootstrapped
Date: Fri, 29 Nov 2013 21:22:44 -0500

Glenn Morris wrote:

> It is generated by semantic/grammar.el.
> semantic/grammar.el requires semantic/grammar-wy.el, therefore
> it is not possible to bootstrap without semantic/grammar-wy.el already
> present.

I should have said: it is generated by wisent/grammar.el, which requires
semantic/grammar.el, which requires semantic/grammar-wy.el.

Message #9 received at 16008 <at> debbugs.gnu.org (full text, mbox):

From: David Engster <deng <at> randomsample.de>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 16008 <at> debbugs.gnu.org, cedet-devel <at> lists.sourceforge.net
Subject: Re: bug#16008: semantic/grammar-wy.el cannot be bootstrapped
Date: Sat, 30 Nov 2013 11:00:47 +0100

Glenn Morris writes:
> semantic/grammar-wy.el is a generated file and therefore ideally should
> not be kept in the Emacs VCS.

That's true for all parsers. However, grammar-wy.el is special, as
you've noticed.

> It is generated by semantic/grammar.el.
> semantic/grammar.el requires semantic/grammar-wy.el, therefore
> it is not possible to bootstrap without semantic/grammar-wy.el already
> present.

Yes, it has a circular dependency, because the parser grammar-wy.el
parses grammar files, including itself. In upstream, we have a fallback
parser for this, which is used when grammar-wy.el is not present
yet. Needless to say, you'll soon face chicken/egg problems when you do
changes in grammar.wy, which IMHO cause more pain in the rear than
having generated files under version control. In fact, every time I
worked on grammar.wy, I thought about getting rid of bootstrapping at
compile time and just put all parsers under version control.

> I should also note that the version of semantic/grammar-wy.el in the
> Emacs trunk does not match up with its sources.
>
> It contains:
>
>   (require 'semantic)
>
> Yet if you regenerate it, it contains:
>
>   (require 'semantic/lex)
>   (eval-when-compile (require 'semantic/bovine))

Yes, there may be small differences w.r.t. to requires and such. The
generated parsers should never differ, though.

Problem is: the freeze is near, and I have to do another sync with
upstream, and I'd also really like to get a few things fixed before
that.

-David

Message #12 received at 16008 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Glenn Morris <rgm <at> gnu.org>,  16008 <at> debbugs.gnu.org,
 cedet-devel <at> lists.sourceforge.net
Subject: Re: bug#16008: semantic/grammar-wy.el cannot be bootstrapped
Date: Sun, 04 Apr 2021 19:24:29 -0400

David Engster [2013-11-30 11:00:47] wrote:
> Glenn Morris writes:
>> semantic/grammar-wy.el is a generated file and therefore ideally should
>> not be kept in the Emacs VCS.
> That's true for all parsers. However, grammar-wy.el is special, as
> you've noticed.
>> It is generated by semantic/grammar.el.
>> semantic/grammar.el requires semantic/grammar-wy.el, therefore
>> it is not possible to bootstrap without semantic/grammar-wy.el already
>> present.
> Yes, it has a circular dependency, because the parser grammar-wy.el
> parses grammar files, including itself.  In upstream, we have
> a fallback parser for this, which is used when grammar-wy.el is not
> present yet.

Of course, another option would be to add a sexp-based syntax for wisent
grammars, so that grammar.wy can be rewritten with that sexp-based syntax
and won't need a wisent parser to parse it any more.

In the mean time I suggest the patch below which I recently sent to
emacs-devel.

Eli Zaretskii [2021-04-03 10:58:12] wrote:
> Assuming you tested that during bootstrap (which you say you never
> do), and assuming there's no better way of breaking the circular
> dependency, I'm okay with the change.  But please also change
> admin/make-tarball.txt to say that grm-wy-boot.el should be updated
> the same way as ldefs-boot.el is.  We should also make sure
> grm-wy-boot.el is updated in Git whenever grammar.wy changes.

Glenn, could you arrange to "auto""-update `grm-wy-boot.el` like you do
for `ldefs-boot.el`?


        Stefan


diff --git a/.gitignore b/.gitignore
index b653ef215b..9fe8ecb594 100644
--- a/.gitignore
+++ b/.gitignore
@@ -88,6 +88,7 @@ lisp/cedet/semantic/wisent/javat-wy.el
 lisp/cedet/semantic/wisent/js-wy.el
 lisp/cedet/semantic/wisent/python-wy.el
 lisp/cedet/srecode/srt-wy.el
+lisp/cedet/semantic/grammar-wy.el
 lisp/eshell/esh-groups.el
 lisp/finder-inf.el
 lisp/leim/ja-dic/
diff --git a/admin/grammars/Makefile.in b/admin/grammars/Makefile.in
index aa09d9edf9..800e31762d 100644
--- a/admin/grammars/Makefile.in
+++ b/admin/grammars/Makefile.in
@@ -48,14 +48,11 @@ BOVINE =
 	${bovinedir}/make-by.el \
 	${bovinedir}/scm-by.el
 
-## FIXME Should include this one too:
-##	${cedetdir}/semantic/grammar-wy.el
-## but semantic/grammar.el (which is what we use to generate grammar-wy.el)
-## requires it!
-WISENT = \
-	${wisentdir}/javat-wy.el \
-	${wisentdir}/js-wy.el \
-	${wisentdir}/python-wy.el \
+WISENT = 				   \
+	${cedetdir}/semantic/grammar-wy.el \
+	${wisentdir}/javat-wy.el 	   \
+	${wisentdir}/js-wy.el 		   \
+	${wisentdir}/python-wy.el 	   \
 	${cedetdir}/srecode/srt-wy.el
 
 ALL = ${BOVINE} ${WISENT}
diff --git a/lisp/cedet/semantic/grammar.el b/lisp/cedet/semantic/grammar.el
index dba289fdd7..782327e617 100644
--- a/lisp/cedet/semantic/grammar.el
+++ b/lisp/cedet/semantic/grammar.el
@@ -31,7 +31,12 @@
 (require 'semantic/format)
 ;; FIXME this is a generated file, but we need to load this file to
 ;; generate it!
-(require 'semantic/grammar-wy)
+;; We need `semantic/grammar-wy.el' but we're also needed to generate
+;; that file from `grammar.wy', so to break the dependency, we keep
+;; a bootstrap copy of `grammar-wy.el' in `grm-wy-boot.el'.  See bug#16008.
+(eval-and-compile
+  (unless (require 'semantic/grammar-wy nil t)
+    (load "semantic/grm-wy-boot")))
 (require 'semantic/idle)
 (require 'help-fns)
 (require 'semantic/analyze)
diff --git a/lisp/cedet/semantic/grammar-wy.el b/lisp/cedet/semantic/grm-wy-boot.el
similarity index 100%
rename from lisp/cedet/semantic/grammar-wy.el
rename to lisp/cedet/semantic/grm-wy-boot.el

Message #15 received at 16008 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 16008 <at> debbugs.gnu.org, cedet-devel <at> lists.sourceforge.net
Subject: Re: bug#16008: semantic/grammar-wy.el cannot be bootstrapped
Date: Sun, 04 Apr 2021 20:49:39 -0400

Stefan Monnier wrote:

> In the mean time I suggest the patch below which I recently sent to
> emacs-devel.

Instead of a -boot version, how about regenerating it but not deleting it:

--- a/admin/grammars/Makefile.in
+++ b/admin/grammars/Makefile.in
@@ -51,16 +51,19 @@ BOVINE = \
 	${bovinedir}/make-by.el \
 	${bovinedir}/scm-by.el
 
-## FIXME Should include this one too:
-##	${cedetdir}/semantic/grammar-wy.el
-## but semantic/grammar.el (which is what we use to generate grammar-wy.el)
-## requires it!  https://debbugs.gnu.org/16008
-WISENT = \
+WISENT1 = \
 	${wisentdir}/javat-wy.el \
 	${wisentdir}/js-wy.el \
 	${wisentdir}/python-wy.el \
 	${cedetdir}/srecode/srt-wy.el
 
+DELETE = ${BOVINE} ${WISENT1}
+
+## The last one is special: semantic/grammar.el (which is what generates it)
+## requires it.  https://debbugs.gnu.org/16008
+## So we generate it, but never delete it.
+WISENT = ${WISENT1} ${cedetdir}/semantic/grammar-wy.el
+
 ALL = ${BOVINE} ${WISENT}
 
 .PHONY: all bovine wisent
@@ -107,7 +110,7 @@ distclean:
 bootstrap-clean:
 
 gen-clean:
-	rm -f ${ALL}
+	rm -f ${DELETE}
 
 maintainer-clean: gen-clean distclean
 

> Glenn, could you arrange to "auto""-update `grm-wy-boot.el` like you do
> for `ldefs-boot.el`?

It's admin/update_autogen.

Message #18 received at 16008 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 16008 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca,
 cedet-devel <at> lists.sourceforge.net
Subject: Re: bug#16008: semantic/grammar-wy.el cannot be bootstrapped
Date: Mon, 05 Apr 2021 15:43:07 +0300

> From: Glenn Morris <rgm <at> gnu.org>
> Date: Sun, 04 Apr 2021 20:49:39 -0400
> Cc: 16008 <at> debbugs.gnu.org, cedet-devel <at> lists.sourceforge.net
> 
> Stefan Monnier wrote:
> 
> > In the mean time I suggest the patch below which I recently sent to
> > emacs-devel.
> 
> Instead of a -boot version, how about regenerating it but not deleting it:

Thanks, but doesn't this get us back to the original situation,
whereby some stale generated *.el files were left behind and failed
the build?

Message #23 received at 16008-done <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: 16008-done <at> debbugs.gnu.org
Subject: Re: bug#16008: semantic/grammar-wy.el cannot be bootstrapped
Date: Mon, 19 Apr 2021 11:51:40 -0400

Version: 28.1

Stefan Monnier wrote:

> In the mean time I suggest the patch below which I recently sent to
> emacs-devel.

It seems this patch was applied in 214dfbf, but sadly the discussion
wasn't sent to this bug report.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.