GNU bug report logs - #48060
"make check" leaves temporary files after native-comp merge

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 48060 in the body.
You can then email your comments to 48060 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: bug-gnu-emacs <at> gnu.org
Subject: "make check" leaves temporary files after native-comp merge
Date: Tue, 27 Apr 2021 09:27:21 -0500

After some iterations of running of "make -j8 bootstrap && make -j8
check", my ~/src/emacs is filled with a lot of temporary directories:

  drwx------  2 skangas skangas 4.0K 2021-04-27 03:50 emacs-testsuite-0B1nPY
  drwx------  2 skangas skangas 4.0K 2021-04-26 12:01 emacs-testsuite-0Ei8cf
  drwx------  2 skangas skangas 4.0K 2021-04-27 03:50 emacs-testsuite-0G40w3
  drwx------  2 skangas skangas 4.0K 2021-04-27 02:59 emacs-testsuite-0GcjNo
  drwx------  2 skangas skangas 4.0K 2021-04-27 02:43 emacs-testsuite-0Gwmf7
  drwx------  2 skangas skangas 4.0K 2021-04-26 20:24 emacs-testsuite-0HYLcF
  drwx------  2 skangas skangas 4.0K 2021-04-27 03:49 emacs-testsuite-0Iy0yX
  drwx------  2 skangas skangas 4.0K 2021-04-26 12:01 emacs-testsuite-0JpgU6

I actually have 2922 of these directories.

The content of one of these directories is:

  emacs-testsuite-xdgMCK
  └── 28.0.50-d25e21f7
      └── subr--trampoline-63616c6c2d70726f63657373_call_process_0.eln

  1 directory, 1 file

Others are empty:

  emacs-testsuite-1HQfjD

  0 directories, 0 files

Every time I run "make -j8 check", I get another 584 of these directories.

Could we do something to ensure these directories are always deleted?

Message #8 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 48060 <at> debbugs.gnu.org, Andrea Corallo <akrl <at> sdf.org>
Subject: Re: bug#48060: "make check" leaves temporary files after native-comp
 merge
Date: Tue, 27 Apr 2021 11:15:23 -0400

Stefan Kangas wrote:

> After some iterations of running of "make -j8 bootstrap && make -j8
> check", my ~/src/emacs is filled with a lot of temporary directories:
>
>   drwx------  2 skangas skangas 4.0K 2021-04-27 03:50 emacs-testsuite-0B1nPY

As predicted in https://debbugs.gnu.org/47138#28 ?
(I see that report is still open, BTW. Not saying it should be merged.)

Message #11 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 48060 <at> debbugs.gnu.org
Subject: Re: bug#48060: "make check" leaves temporary files after
 native-comp merge
Date: Tue, 27 Apr 2021 16:53:33 +0000

Stefan Kangas <stefan <at> marxist.se> writes:

> After some iterations of running of "make -j8 bootstrap && make -j8
> check", my ~/src/emacs is filled with a lot of temporary directories:
>
>   drwx------  2 skangas skangas 4.0K 2021-04-27 03:50 emacs-testsuite-0B1nPY
>   drwx------  2 skangas skangas 4.0K 2021-04-26 12:01 emacs-testsuite-0Ei8cf
>   drwx------  2 skangas skangas 4.0K 2021-04-27 03:50 emacs-testsuite-0G40w3
>   drwx------  2 skangas skangas 4.0K 2021-04-27 02:59 emacs-testsuite-0GcjNo
>   drwx------  2 skangas skangas 4.0K 2021-04-27 02:43 emacs-testsuite-0Gwmf7
>   drwx------  2 skangas skangas 4.0K 2021-04-26 20:24 emacs-testsuite-0HYLcF
>   drwx------  2 skangas skangas 4.0K 2021-04-27 03:49 emacs-testsuite-0Iy0yX
>   drwx------  2 skangas skangas 4.0K 2021-04-26 12:01 emacs-testsuite-0JpgU6
>
> I actually have 2922 of these directories.
>
> The content of one of these directories is:
>
>   emacs-testsuite-xdgMCK
>   └── 28.0.50-d25e21f7
>       └── subr--trampoline-63616c6c2d70726f63657373_call_process_0.eln
>
>   1 directory, 1 file
>
> Others are empty:
>
>   emacs-testsuite-1HQfjD
>
>   0 directories, 0 files
>
> Every time I run "make -j8 check", I get another 584 of these directories.
>
> Could we do something to ensure these directories are always deleted?

I guess the correct way to do it would be in the Makefile when tests are
done, am I correct?

The other option would be to have Emacs to clean-up this directory
before exiting, but in case of a crash it wouldn't work.

The last option would be to convince `make-temp-file' we call at
startup.el:553 to create the directory in "/tmp/" or equivalent (not
sure that's possible).

  Andrea

Message #14 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 48060 <at> debbugs.gnu.org, stefan <at> marxist.se
Subject: Re: bug#48060: "make check" leaves temporary files after native-comp
 merge
Date: Tue, 27 Apr 2021 20:35:30 +0300

> Cc: 48060 <at> debbugs.gnu.org
> Date: Tue, 27 Apr 2021 16:53:33 +0000
> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> I guess the correct way to do it would be in the Makefile when tests are
> done, am I correct?
> 
> The other option would be to have Emacs to clean-up this directory
> before exiting, but in case of a crash it wouldn't work.

We usually do the latter.  Crashes in the test suite should be rare.

> The last option would be to convince `make-temp-file' we call at
> startup.el:553 to create the directory in "/tmp/" or equivalent (not
> sure that's possible).

That's only a good solution on platforms where /tmp is periodically
cleaned up, or even disappears when the user logs off.

Message #17 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 48060 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#48060: "make check" leaves temporary files after
 native-comp merge
Date: Tue, 27 Apr 2021 21:25:56 +0200

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs <at> gnu.org> writes:

Hi Andrea,

>> The content of one of these directories is:
>>
>>   emacs-testsuite-xdgMCK
>>   └── 28.0.50-d25e21f7
>>       └── subr--trampoline-63616c6c2d70726f63657373_call_process_0.eln
>>
>>   1 directory, 1 file
>>
>> Others are empty:
>>
>>   emacs-testsuite-1HQfjD
>>
>>   0 directories, 0 files
>>
>> Every time I run "make -j8 check", I get another 584 of these directories.
>>
>> Could we do something to ensure these directories are always deleted?
>
> I guess the correct way to do it would be in the Makefile when tests are
> done, am I correct?
>
> The other option would be to have Emacs to clean-up this directory
> before exiting, but in case of a crash it wouldn't work.

Not an Emacs cleanup. Usually, test functions are responsible for their
own cleanup. You could put the whole test body in an unwind-protect
bodyform, and perform the cleanup in one of the unwindforms.

>   Andrea

Best regards, Michael.

Message #20 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 48060 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#48060: "make check" leaves temporary files after
 native-comp merge
Date: Tue, 27 Apr 2021 20:34:23 +0000

Michael Albinus <michael.albinus <at> gmx.de> writes:

> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs <at> gnu.org> writes:
>
> Hi Andrea,
>
>>> The content of one of these directories is:
>>>
>>>   emacs-testsuite-xdgMCK
>>>   └── 28.0.50-d25e21f7
>>>       └── subr--trampoline-63616c6c2d70726f63657373_call_process_0.eln
>>>
>>>   1 directory, 1 file
>>>
>>> Others are empty:
>>>
>>>   emacs-testsuite-1HQfjD
>>>
>>>   0 directories, 0 files
>>>
>>> Every time I run "make -j8 check", I get another 584 of these directories.
>>>
>>> Could we do something to ensure these directories are always deleted?
>>
>> I guess the correct way to do it would be in the Makefile when tests are
>> done, am I correct?
>>
>> The other option would be to have Emacs to clean-up this directory
>> before exiting, but in case of a crash it wouldn't work.
>
> Not an Emacs cleanup. Usually, test functions are responsible for their
> own cleanup. You could put the whole test body in an unwind-protect
> bodyform, and perform the cleanup in one of the unwindforms.

Hi Michael,

the issue is that here is not really the test function that is creating
this folder, but Emacs it-self that to operate needs a folder to store
and retrieve eln files.

For the test-suite we moved to a different folder than the one we use
for normal runs to have a clean environment to run the test-suite on.

Regards

  Andrea

Message #23 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 48060 <at> debbugs.gnu.org, stefan <at> marxist.se
Subject: Re: bug#48060: "make check" leaves temporary files after
 native-comp merge
Date: Tue, 27 Apr 2021 21:36:38 +0000

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Cc: 48060 <at> debbugs.gnu.org
>> Date: Tue, 27 Apr 2021 16:53:33 +0000
>> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>> 
>> I guess the correct way to do it would be in the Makefile when tests are
>> done, am I correct?
>> 
>> The other option would be to have Emacs to clean-up this directory
>> before exiting, but in case of a crash it wouldn't work.
>
> We usually do the latter.  Crashes in the test suite should be rare.

Right, c62262736c using `kill-emacs-hook' should do the clean-up for all
non crashing sessions.

Regards

  Andrea

Message #26 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 48060 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#48060: "make check" leaves temporary files after
 native-comp merge
Date: Wed, 28 Apr 2021 12:57:03 +0200

Andrea Corallo <akrl <at> sdf.org> writes:

> Hi Michael,

Hi Andrea,

> the issue is that here is not really the test function that is creating
> this folder, but Emacs it-self that to operate needs a folder to store
> and retrieve eln files.

I know.

> For the test-suite we moved to a different folder than the one we use
> for normal runs to have a clean environment to run the test-suite on.

Couldn't you create this different folder inside an ert-deftest, and
remove it at test-case end?

> Regards
>
>   Andrea

Message #29 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 48060 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#48060: "make check" leaves temporary files after
 native-comp merge
Date: Wed, 28 Apr 2021 19:31:04 +0000

Michael Albinus <michael.albinus <at> gmx.de> writes:

> Andrea Corallo <akrl <at> sdf.org> writes:
>
>> Hi Michael,
>
> Hi Andrea,
>
>> the issue is that here is not really the test function that is creating
>> this folder, but Emacs it-self that to operate needs a folder to store
>> and retrieve eln files.
>
> I know.
>
>> For the test-suite we moved to a different folder than the one we use
>> for normal runs to have a clean environment to run the test-suite on.
>
> Couldn't you create this different folder inside an ert-deftest, and
> remove it at test-case end?

No, Emacs needs to be able if necessary to depose .eln files even before
a test is run (this is not specific to native-comp tests), and given we
set HOME to a non existent directory the conventional
"~/.emacs.d/eln-cache" can't be used.

Regards

  Andrea

Message #32 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: 48060 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>,
 Michael Albinus <michael.albinus <at> gmx.de>
Subject: Re: bug#48060: "make check" leaves temporary files after
 native-comp merge
Date: Sun, 02 May 2021 10:30:57 +0200

Andrea Corallo <akrl <at> sdf.org> writes:

>> Couldn't you create this different folder inside an ert-deftest, and
>> remove it at test-case end?
>
> No, Emacs needs to be able if necessary to depose .eln files even before
> a test is run (this is not specific to native-comp tests), and given we
> set HOME to a non existent directory the conventional
> "~/.emacs.d/eln-cache" can't be used.

I don't quite understand how this precludes us from specifying a
directory under test/ for the .eln files?  I.e., 

HOME=/nonexistent ELN_DIR=test/cache-dir emacs ...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #35 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 48060 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>,
 Michael Albinus <michael.albinus <at> gmx.de>
Subject: Re: bug#48060: "make check" leaves temporary files after
 native-comp merge
Date: Sun, 02 May 2021 21:28:14 +0000

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Andrea Corallo <akrl <at> sdf.org> writes:
>
>>> Couldn't you create this different folder inside an ert-deftest, and
>>> remove it at test-case end?
>>
>> No, Emacs needs to be able if necessary to depose .eln files even before
>> a test is run (this is not specific to native-comp tests), and given we
>> set HOME to a non existent directory the conventional
>> "~/.emacs.d/eln-cache" can't be used.
>
> I don't quite understand how this precludes us from specifying a
> directory under test/ for the .eln files?  I.e., 
>
> HOME=/nonexistent ELN_DIR=test/cache-dir emacs ...

I agree, these are just to different cases, specifying one single
directory is certainly possible, doing it test by test inside
ert-deftest makes me a little more skeptical :)

  Andrea

Message #38 received at 48060 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 48060 <at> debbugs.gnu.org, Andrea Corallo <akrl <at> sdf.org>
Subject: Re: bug#48060: "make check" leaves temporary files after
 native-comp merge
Date: Tue, 24 May 2022 14:21:28 +0200

Stefan Kangas <stefan <at> marxist.se> writes:

> After some iterations of running of "make -j8 bootstrap && make -j8
> check", my ~/src/emacs is filled with a lot of temporary directories:
>
>   drwx------  2 skangas skangas 4.0K 2021-04-27 03:50 emacs-testsuite-0B1nPY
>   drwx------  2 skangas skangas 4.0K 2021-04-26 12:01 emacs-testsuite-0Ei8cf
>   drwx------  2 skangas skangas 4.0K 2021-04-27 03:50 emacs-testsuite-0G40w3
>   drwx------  2 skangas skangas 4.0K 2021-04-27 02:59 emacs-testsuite-0GcjNo
>   drwx------  2 skangas skangas 4.0K 2021-04-27 02:43 emacs-testsuite-0Gwmf7
>   drwx------  2 skangas skangas 4.0K 2021-04-26 20:24 emacs-testsuite-0HYLcF
>   drwx------  2 skangas skangas 4.0K 2021-04-27 03:49 emacs-testsuite-0Iy0yX
>   drwx------  2 skangas skangas 4.0K 2021-04-26 12:01 emacs-testsuite-0JpgU6
>
> I actually have 2922 of these directories.

Andrea's previous changes reduced the number of these directories, but
there were still some left over.  I think I've now fixed that, at least
for "make check", and I'm going through the other ert interface
functions, too.

nativecomp still leaves other files over, but I'm opening a new report
for that, and closing this one.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.