GNU bug report logs - #47381
Atomic (fast) install

Previous Next

Package: automake;

Reported by: Дилян Палаузов <dilyan.palauzov <at> aegee.org>

Date: Thu, 25 Mar 2021 10:27:01 UTC

Severity: normal

Done: Karl Berry <karl <at> freefriends.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 47381 in the body.
You can then email your comments to 47381 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-automake <at> gnu.org:
bug#47381; Package automake. (Thu, 25 Mar 2021 10:27:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Дилян Палаузов <dilyan.palauzov <at> aegee.org>:
New bug report received and forwarded. Copy sent to bug-automake <at> gnu.org. (Thu, 25 Mar 2021 10:27:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Дилян Палаузов
 <dilyan.palauzov <at> aegee.org>
To: bug-automake <at> gnu.org
Subject: Atomic (fast) install
Date: Thu, 25 Mar 2021 12:24:13 +0200
Hello,

I use automake with libtool .  The latter has “fast install” option. 
My config.status contains:

enable_fast_install='yes'
fast_install=$enable_fast_install

During “make install” it prints:

 /bin/sh ./libtool   --mode=install /usr/bin/install -c  
lib/libcyrus_min.la lib/libcyrus.la imap/libcyrus_imap.la
sieve/libcyrus_sieve.la '/usr/local/lib'  
libtool: install: /usr/bin/install -c lib/.libs/libcyrus_min.so.0.0.0
/usr/local/lib/libcyrus_min.so.0.0.0                                  
libtool: install: (cd /usr/local/lib && { ln -s -f
libcyrus_min.so.0.0.0 libcyrus_min.so.0 || { rm -f libcyrus_min.so.0 &&
ln -s libcyrus_min.so.0.0.0 libcyrus_min.so.0; }; })                  
libtool: install: (cd /usr/local/lib && { ln -s -f
libcyrus_min.so.0.0.0 libcyrus_min.so || { rm -f libcyrus_min.so && ln
-s libcyrus_min.so.0.0.0 libcyrus_min.so; }; })                       
libtool: install: /usr/bin/install -c lib/.libs/libcyrus_min.lai
/usr/local/lib/libcyrus_min.la                                        
libtool: install: /usr/bin/install -c lib/.libs/libcyrus.so.0.0.0
/usr/local/lib/libcyrus.so.0.0.0                                      
libtool: install: (cd /usr/local/lib && { ln -s -f libcyrus.so.0.0.0
libcyrus.so.0 || { rm -f libcyrus.so.0 && ln -s libcyrus.so.0.0.0
libcyrus.so.0; }; })   
libtool: install: (cd /usr/local/lib && { ln -s -f libcyrus.so.0.0.0
libcyrus.so || { rm -f libcyrus.so && ln -s libcyrus.so.0.0.0
libcyrus.so; }; })         
libtool: install: /usr/bin/install -c lib/.libs/libcyrus.lai
/usr/local/lib/libcyrus.la                                            
libtool: warning: relinking 'imap/libcyrus_imap.la'                   



for some shared libraries, but it does not print it for all shared
libraries.  It does not print it for libcyrus_min.so or libcyrus.so

Moreover (Auto)make install calls:

/bin/sh ./libtool --mode=install /usr/bin/install -c imap/httpd
'/usr/local/libexec'

which triggers these syscalls:

[pid 346601] stat("/usr/local/libexec/httpd", {st_mode=S_IFREG|0755,
st_size=3362912, ...}) = 0
[pid 346601] newfstatat(AT_FDCWD, "imap/.libs/httpd",
{st_mode=S_IFREG|0755, st_size=3362912, ...}, 0) = 0
[pid 346601] newfstatat(AT_FDCWD, "/usr/local/libexec/httpd",
{st_mode=S_IFREG|0755, st_size=3362912, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 346601] unlink("/usr/local/libexec/httpd") = 0
[pid 346601] openat(AT_FDCWD, "imap/.libs/httpd", O_RDONLY) = 3
[pid 346601] fstat(3, {st_mode=S_IFREG|0755, st_size=3362912, ...}) = 0
[pid 346601] openat(AT_FDCWD, "/usr/local/libexec/httpd",
O_WRONLY|O_CREAT|O_EXCL, 0600) = 4
[pid 346601] fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
[pid 346601] ioctl(4, BTRFS_IOC_CLONE or FICLONE, 3) = -1 EOPNOTSUPP
(Operation not supported)
[pid 346601] fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
[pid 346601] mmap(NULL, 139264, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb61f408000
[pid 346601] read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\2\0>\0\1\0\0\0\260\343A\0\0\0\0\0"...,
131072) = 131072
[pid 346601] write(4,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\2\0>\0\1\0\0\0\260\343A\0\0\0\0\0"...,
131072) = 131072
[pid 346601] read(3,
"\377H\211\302H\213\205h\372\377\377\211\331H\211\306H\307\307\4\252\35
3\212\350t\0\6\0H\213E\340"..., 131072) = 131072
[pid 346601] write(4,
"\377H\211\302H\213\205h\372\377\377\211\331H\211\306H\307\307\4\252\35
3\212\350t\0\6\0H\213E\340"..., 131072) = 131072
...
[pid 346601] chmod("/usr/local/libexec/httpd", 0755) = 0
[pid 346601] lstat("/usr/local/libexec/httpd", {st_mode=S_IFREG|0755,
st_size=3362912, ...}) = 0


That said, the installation is not atomic.  The bytes destination file
is replaced byte by byte, rather than writing the result to a temporary
file and then calling rename(2), thus ensuring that the file is always
executable.

Cyrus IMAP is a server, consisting of several executable files
(daemons).  When an executable file notices, that its timestamp has
changed, the daemon terminates and is then started again (from the new
executable).  The problem is, that during this process, there is a
timegap, where the process-file has newer timestamp, but is not
executable yet, as `install` currently moves data to it.

As far the install executable is concerned, I raised the atomicity to
coreutils at debbugs.gnu.org/cgi/bugreport.cgi?bug=47380 .

• Please adjust Automake on “make install” to do (per default or with
an option) atomic install of files, even when libtool is involved: the
file is moved to a temporary destination first, and then rename(2)d, or
the source is moved to the destination, disappearing from the source,
or something like this.

• Why does “fast install” relink some libraries, but others not?

• Since ./libtool starts with 
  #! /bin/sh

do not call
  /bin/sh ./libtool --mode=install /usr/bin/install -c imap/httpd
'/usr/local/libexec'

but 
  ./libtool --mode=install /usr/bin/install -c imap/httpd
'/usr/local/libexec'

Greetings
  Дилян





Information forwarded to bug-automake <at> gnu.org:
bug#47381; Package automake. (Fri, 26 Mar 2021 22:41:02 GMT) Full text and rfc822 format available.

Message #8 received at 47381 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: dilyan.palauzov <at> aegee.org
Cc: 47381 <at> debbugs.gnu.org
Subject: Re: bug#47381: Atomic (fast) install
Date: Fri, 26 Mar 2021 16:39:06 -0600
Thanks for the reports, but:

1) Automake can only run whatever install program has been specified by
the user. It does not seem possible (or sensible) to me to guarantee
atomic installs at the automake level. (This behavior by cyrus imap also
seems undesirable to me, but whatever.)

2) I have never heard of "fast install" before, and have barely ever
touched libtool in general. You'll need to ask the libtool maintainers.

3) I have seen explicit invocation of "/bin/sh <script>" in other
contexts. I'm not sure why (noexec filesystems are all I can think of),
but I don't see a need to change it, either. If it's causing an actual
problem, feel free to report that.

Sorry this is not a very helpful reply, but it's what I've got.
--best, karl.




Reply sent to Karl Berry <karl <at> freefriends.org>:
You have taken responsibility. (Fri, 26 Mar 2021 22:41:03 GMT) Full text and rfc822 format available.

Notification sent to Дилян Палаузов <dilyan.palauzov <at> aegee.org>:
bug acknowledged by developer. (Fri, 26 Mar 2021 22:41:03 GMT) Full text and rfc822 format available.

Information forwarded to bug-automake <at> gnu.org:
bug#47381; Package automake. (Sat, 27 Mar 2021 09:28:01 GMT) Full text and rfc822 format available.

Message #16 received at 47381 <at> debbugs.gnu.org (full text, mbox):

From: Peter Johansson <trojkan <at> gmail.com>
To: Дилян Палаузов
 <dilyan.palauzov <at> aegee.org>, 47381 <at> debbugs.gnu.org
Subject: Re: bug#47381: Atomic (fast) install
Date: Sat, 27 Mar 2021 19:27:21 +1000
On 25/3/21 8:24 pm, Дилян Палаузов wrote:
> • Please adjust Automake on “make install” to do (per default or with
> an option) atomic install of files, even when libtool is involved: the
> file is moved to a temporary destination first, and then rename(2)d, or
> the source is moved to the destination, disappearing from the source,
> or something like this.

As a workaround, I would write a script using the DESTDIR variable. 
First running make install DESTDIR=/some/path (please refer to the 
manual for details) and then move (rename) files from there to their 
final destination - or include the script in the Makefile if certain 
make variables are needed.

Cheers,

Peter





bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sat, 24 Apr 2021 11:24:07 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 339 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.