GNU bug report logs - #8616
conflict between build-aux/compile script and coreutils Makefiles

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 8616 in the body.
You can then email your comments to 8616 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Green, Paul" <Paul.Green <at> stratus.com>
To: <bug-coreutils <at> gnu.org>
Subject: conflict between build-aux/compile script and coreutils Makefiles
Date: Wed, 4 May 2011 17:09:13 -0400

[Message part 1 (text/plain, inline)]

Gentle Coreutils Developers,

I am writing to notify you of an issue that I stumbled across while
building coreutils-8.12 on the Stratus OpenVOS platform.

On this platform, we have both gcc and a native cc compiler. While gcc
handles the combination of the -c and -o arguments, our native cc
compiler does not. Thus, the coreutils configure script sets
ac_cv_prog_gcc_cc_c_o=no. This leads to using the build-aux/compile
script, which in turn leads to the following "interesting" result
(edited for clarity):

  CC       version.o
  AR       libver.a
  CC       cp.o
  CC       copy.o  (point 1)
  CC       cp-hash.o
  CC       extent-scan.o
  CCLD     cp.pm
  CC       ginstall-install.o
  CC       ginstall-prog-fprintf.o
  CC       ginstall-copy.o  (point 2)
  CC       ginstall-cp-hash.o
  CC       ginstall-extent-scan.o
  CCLD     ginstall.pm
  CC       mv.o
  CC       remove.o
  CCLD     mv.pm
gcc: copy.o: No such file or directory.
gcc: cp-hash.o: No such file or directory.
gcc: extent-scan.o: No such file or directory.
make[3]: *** [mv.pm] Error 1

The root cause of this failure is as follows. The same issue affects
copy.o, cp-hash.o and extent-scan.o, but I'll just focus on copy.o.

At point 1, the build-aux/compile runs gcc (w/o -o) to produce copy.o.
At point 2, the script produces copy.o again, then renames it to
ginstall-copy.o, thereby deleting copy.o.

The src/Makefile does not list copy.o as a dependency of mv.pm (Oops),
so Make does not rebuild copy.o prior to running the link command.
Hence, the linker (gcc) complains that it cannot find the file.  

On a platform where both cc and $CC handle the combination of -c and -o
correctly, this issue cannot arise.

The solution is either to make the compile script smarter, or to have
the dependencies of mv.pm properly listed, or to have mv.pm depend upon
a renamed copy of copy.o, much as is done for ginstall, or to set the
environment variable ac_cv_prog_cc_gcc_c_o=yes before running configure.
This last technique presumes that the faulty "cc" compiler is never
actually used, which seems to be the case here.

Speaking as someone who is fairly familiar with bash, make, and
autoconf, this was still took a lot longer to figure out that I'd like
to admit.  I believe that I have correctly explained the issue, but I
apologize if there are errors in the above description.

I suspect that you can reproduce this issue by forcing
ac_cv_prog_cc_gcc_c_o=no and then running configure.

At least now this issue will get into the mail archives, and be fodder
for some future web search. :-)

Thanks 
PG 
-- 
Paul Green, Senior Technical Consultant, Stratus Technologies. 
Voice: +1 978-461-7557; FAX: +1 978-461-3610; Mobile: +1 (978) 235-2451;
AIM: PaulGreen
Follow me on Twitter: stratuspaulg

[Message part 2 (text/html, inline)]

Message #8 received at 8616 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: "Green\, Paul" <Paul.Green <at> stratus.com>
Cc: 8616 <at> debbugs.gnu.org, automake <at> gnu.org
Subject: Re: bug#8616: conflict between build-aux/compile script and coreutils
	Makefiles
Date: Thu, 05 May 2011 12:42:07 +0200

Green, Paul wrote:
> Gentle Coreutils Developers,
>
> I am writing to notify you of an issue that I stumbled across while
> building coreutils-8.12 on the Stratus OpenVOS platform.

Hi Paul,

Thanks for the detailed report.
I'm Cc'ing the automake list, since that's
the source of the compile script.

> On this platform, we have both gcc and a native cc compiler. While gcc
> handles the combination of the -c and -o arguments, our native cc
> compiler does not. Thus, the coreutils configure script sets
> ac_cv_prog_gcc_cc_c_o=no. This leads to using the build-aux/compile
> script, which in turn leads to the following "interesting" result
> (edited for clarity):
>
>   CC       version.o
>   AR       libver.a
>   CC       cp.o
>   CC       copy.o  (point 1)
>   CC       cp-hash.o
>   CC       extent-scan.o
>   CCLD     cp.pm
>   CC       ginstall-install.o
>   CC       ginstall-prog-fprintf.o
>   CC       ginstall-copy.o  (point 2)
>   CC       ginstall-cp-hash.o
>   CC       ginstall-extent-scan.o
>   CCLD     ginstall.pm
>   CC       mv.o
>   CC       remove.o
>   CCLD     mv.pm
> gcc: copy.o: No such file or directory.
> gcc: cp-hash.o: No such file or directory.
> gcc: extent-scan.o: No such file or directory.
> make[3]: *** [mv.pm] Error 1
>
> The root cause of this failure is as follows. The same issue affects
> copy.o, cp-hash.o and extent-scan.o, but I'll just focus on copy.o.
>
> At point 1, the build-aux/compile runs gcc (w/o -o) to produce copy.o.
> At point 2, the script produces copy.o again, then renames it to
> ginstall-copy.o, thereby deleting copy.o.
>
> The src/Makefile does not list copy.o as a dependency of mv.pm (Oops),
> so Make does not rebuild copy.o prior to running the link command.

My src/Makefile and src/Makefile.in do include that dependency,
assuming your EXEEXT is ".pm".
Does your version of that file look different from this?

    am__objects_1 = copy.$(OBJEXT) cp-hash.$(OBJEXT) extent-scan.$(OBJEXT)
    am_mv_OBJECTS = mv.$(OBJEXT) remove.$(OBJEXT) $(am__objects_1)
    mv_OBJECTS = $(am_mv_OBJECTS)
    ...
    mv$(EXEEXT): $(mv_OBJECTS) $(mv_DEPENDENCIES) $(EXTRA_mv_DEPENDENCIES)

This is not due to a missing dependency,
but rather that the compile script removes copy.o
in the process of creating ginstall-copy.o.
You can see that by the fact that when you rerun "make"
after the above failure, it does regenerate copy.o.

> Hence, the linker (gcc) complains that it cannot find the file.
>
> On a platform where both cc and $CC handle the combination of -c and -o
> correctly, this issue cannot arise.
>
> The solution is either to make the compile script smarter, or to have
> the dependencies of mv.pm properly listed, or to have mv.pm depend upon
> a renamed copy of copy.o, much as is done for ginstall, or to set the
> environment variable ac_cv_prog_cc_gcc_c_o=yes before running configure.
> This last technique presumes that the faulty "cc" compiler is never
> actually used, which seems to be the case here.
>
> Speaking as someone who is fairly familiar with bash, make, and
> autoconf, this was still took a lot longer to figure out that I'd like
> to admit.  I believe that I have correctly explained the issue, but I
> apologize if there are errors in the above description.
>
> I suspect that you can reproduce this issue by forcing
> ac_cv_prog_cc_gcc_c_o=no and then running configure.

Running this,
  ac_cv_prog_cc_gcc_c_o=no ./configure && make
I was indeed able to reproduce it.

I think you're right that making "compile" smarter is the way to go.
It looks like it could be modified to link (or copy) the source file
to some temporary file name, e.g., copy-xYV4aP.c, compile that
to copy-xYV4aP.o, and rename the latter to the destination .o file.
Of course, it would have to remove the temporary file upon termination--
both irregular and normal.

This would also resolve the parallel build race whereby two or more
programs using the same object file both compile the same source to
the same object and rename that file to a different destination.

To the automake folks, is there any reason not to do that?

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: bug-coreutils <at> gnu.org
Cc: 8616 <at> debbugs.gnu.org, "Green, Paul" <Paul.Green <at> stratus.com>,
	Jim Meyering <jim <at> meyering.net>, automake <at> gnu.org
Subject: Re: bug#8616: conflict between build-aux/compile script and coreutils
	Makefiles
Date: Thu, 5 May 2011 16:43:09 +0200

On Thursday 05 May 2011, Jim Meyering wrote:
> Green, Paul wrote:
> > Gentle Coreutils Developers,
> >
[HUGE CUT]
> 
> To the automake folks, is there any reason not to do that?
> 

Hi Jim.  I'm in a middle of something else right now, so I'm
probably not going to look into this soonish.  Could you please
open a bug report on bug-automake so that I won't forget about
the issue?

Thanks,
  Stefano

Message #17 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: "Green, Paul" <Paul.Green <at> stratus.com>, 8616 <at> debbugs.gnu.org,
	bug-coreutils <at> gnu.org, automake <at> gnu.org
Subject: Re: bug#8616: conflict between build-aux/compile script and coreutils
	Makefiles
Date: Thu, 05 May 2011 16:55:11 +0200

Stefano Lattarini wrote:
> On Thursday 05 May 2011, Jim Meyering wrote:
>> Green, Paul wrote:
>> > Gentle Coreutils Developers,
>> >
> [HUGE CUT]
>>
>> To the automake folks, is there any reason not to do that?
>>
>
> Hi Jim.  I'm in a middle of something else right now, so I'm
> probably not going to look into this soonish.  Could you please
> open a bug report on bug-automake so that I won't forget about
> the issue?

Sure.  I did that via http://debbugs.gnu.org/8621

Message #23 received at 8616 <at> debbugs.gnu.org (full text, mbox):

From: "Green, Paul" <Paul.Green <at> stratus.com>
To: "Jim Meyering" <jim <at> meyering.net>
Cc: 8616 <at> debbugs.gnu.org, automake <at> gnu.org
Subject: RE: bug#8616: conflict between build-aux/compile script and coreutils
	Makefiles
Date: Thu, 5 May 2011 11:18:21 -0400

Jim Meyering wrote:
> 
> Green, Paul wrote:
> > Gentle Coreutils Developers,
> >
> > I am writing to notify you of an issue that I stumbled across while
> > building coreutils-8.12 on the Stratus OpenVOS platform.
> 
> Hi Paul,
> 
> Thanks for the detailed report.
> I'm Cc'ing the automake list, since that's
> the source of the compile script.

Thanks, and sorry for not digging further to notice this!

[snip]
> > The src/Makefile does not list copy.o as a dependency of mv.pm
> > (Oops), so Make does not rebuild copy.o prior to running the
> > link command.
> 
> My src/Makefile and src/Makefile.in do include that dependency,
> assuming your EXEEXT is ".pm".
> Does your version of that file look different from this?
> 
>     am__objects_1 = copy.$(OBJEXT) cp-hash.$(OBJEXT) extent-
> scan.$(OBJEXT)
>     am_mv_OBJECTS = mv.$(OBJEXT) remove.$(OBJEXT) $(am__objects_1)
>     mv_OBJECTS = $(am_mv_OBJECTS)
>     ...
>     mv$(EXEEXT): $(mv_OBJECTS) $(mv_DEPENDENCIES)
> $(EXTRA_mv_DEPENDENCIES)

Yes, my copy of the generated Makefile looks the same as that.

> This is not due to a missing dependency,
> but rather that the compile script removes copy.o
> in the process of creating ginstall-copy.o.
> You can see that by the fact that when you rerun "make"
> after the above failure, it does regenerate copy.o.

Yes, I now see that. I was mistakenly keying off of the names of the
variables in the Makefile. I should have paid more attention to the
actual list of dependencies on the link line.  Sorry about that!

[snip]
> I think you're right that making "compile" smarter is the way to go.
> It looks like it could be modified to link (or copy) the source file
> to some temporary file name, e.g., copy-xYV4aP.c, compile that
> to copy-xYV4aP.o, and rename the latter to the destination .o file.
> Of course, it would have to remove the temporary file upon
termination-
> - both irregular and normal.
> 
> This would also resolve the parallel build race whereby two or more
> programs using the same object file both compile the same source to
> the same object and rename that file to a different destination.
> 
> To the automake folks, is there any reason not to do that?

I agree in general. However, as someone whose operating system only
recently (2009) started supporting file names longer than 32 characters,
I think the compile script has to be careful not to lengthen the name of
the source file. I have noticed a trend in recent years to use longer
and longer file names.  If the compile script can simply replace the
original name by a unique generated name of a reasonable length, that
would be better, in my view.  I'm sure there must still be some
operating systems out there that do not support 255-character file
names.

Thank you for your quick response and guidance.

Thanks
PG

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.