GNU bug report logs - #59515
nginx: Fails to start on boot while upstream service is not yet running

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 59515 in the body.
You can then email your comments to 59515 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
To: bug-guix <at> gnu.org
Subject: nginx: Fails to start on boot while upstream service is not yet
 running
Date: Wed, 23 Nov 2022 16:34:57 +0100

When I start my personal server with a radicale service behind a nginx
reverse proxy, nginx fails to start.

The relevant part in the log:
Nov 23 16:02:56 localhost shepherd[1]: Service networking has been started.
Nov 23 16:02:56 localhost shepherd[1]: Service radicale has been started.
Nov 23 16:02:56 localhost shepherd[1]: Service ssh-daemon has been started.
Nov 23 16:02:56 localhost shepherd[1]: [nginx] nginx: [emerg] host not
found in upstream "localhost:5232" in
/gnu/store/y29zl57pprwxbcxfx593s16456kxk99y-nginx.conf:15
Nov 23 16:02:56 localhost shepherd[1]: Failed to start nginx in the
background.

The config can be found here:
https://gitlab.com/jonsger/jonsger-guix/-/blob/master/config/baebia.scm#L116
```
(upstream-blocks (list
  (nginx-upstream-configuration
    (name "radicale")
    (servers (list "localhost:5232")))))
```

I wonder whats going wrong here. Is there a way to define that nginx
service should wait until radicale service is started?

~Jonathan

Message #8 received at 59515 <at> debbugs.gnu.org (full text, mbox):

From: mirai <mirai <at> makinata.eu>
To: 59515 <at> debbugs.gnu.org
Subject: nginx: Fails to start on boot while upstream service is not yet
 running
Date: Wed, 23 Nov 2022 17:35:34 +0000

nginx-configuration has a 'shepherd-requirement' parameter that can be used here

Message #11 received at 59515 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
To: 59515 <at> debbugs.gnu.org
Subject: nginx: Fails to start on boot while upstream service is not yet
 running
Date: Fri, 25 Nov 2022 09:56:00 +0100

Ah thanks.

I added `(shepherd-requirement '(radicale))` to my nginx-configuration.
Sadly the issue is not fixed, the situation is the same as before.

I assume that radicale takes a fracture of a second to start. And nginx
starts faster up. See this small experiment:

$ sudo herd stop radicale
$ sudo herd start radicale; curl localhost:5232
Service radicale has been started.
curl: (7) Failed to connect to localhost port 5232 after 0 ms:
Verbindungsaufbau abgelehnt

$ sudo herd stop radicale
$ sudo herd start radicale; sleep 0.2; curl localhost:5232
Service radicale has been started.
Redirected to /.web

Message #14 received at 59515 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
Cc: 59515 <at> debbugs.gnu.org
Subject: Re: bug#59515: nginx: Fails to start on boot while upstream service
 is not yet running
Date: Sun, 27 Nov 2022 22:14:48 +0100

Jonathan Brielmaier <jonathan.brielmaier <at> web.de> skribis:

> I added `(shepherd-requirement '(radicale))` to my nginx-configuration.
> Sadly the issue is not fixed, the situation is the same as before.
>
> I assume that radicale takes a fracture of a second to start. And nginx
> starts faster up. See this small experiment:

That looks like a bug in the radicale service: there’s no
synchronization upon startup:

--8<---------------cut here---------------start------------->8---
     (list (shepherd-service
            (provision '(radicale))
            (documentation "Run the radicale daemon.")
            (requirement '(networking))
            (start #~(make-forkexec-constructor
                      (list #$(file-append package "/bin/radicale")
                        "-C" #$config-file)
                      #:user "radicale"
                      #:group "radicale"))
            (stop #~(make-kill-destructor))))
--8<---------------cut here---------------end--------------->8---

It should wait for a PID file or be started as a systemd or inetd
service.  That way, we’d be sure that shepherd considers it started once
it’s really listening for input.  A quick grep in the source suggests it
creates a PID file somewhere.

HTH!

Ludo’.

Message #17 received at 59515 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 59515 <at> debbugs.gnu.org
Subject: Re: bug#59515: nginx: Fails to start on boot while upstream service
 is not yet running
Date: Mon, 2 Jan 2023 18:09:26 +0100

On 27.11.22 22:14, Ludovic Courtès wrote:> That looks like a bug in the
radicale service: there’s no
> synchronization upon startup:

I think its rather an issue with nscd, networking or some other base
service.

Changing the nginx configuration from
(servers (list "localhost:5232"))
to
(servers (list "127.0.0.1:5232"))
resolves the issue for me :)

> It should wait for a PID file or be started as a systemd or inetd
> service.  That way, we’d be sure that shepherd considers it started once
> it’s really listening for input.  A quick grep in the source suggests it
> creates a PID file somewhere.

I couldn't find any PID file on my server of radicale. But there seems
to by a .lock file: /var/lib/radicale/collections/.Radicale.lock
No idea if that would be helpful...

~Jonathan

Message #22 received at 59515-done <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
To: 59515-done <at> debbugs.gnu.org
Subject: nginx: Fails to start on boot while upstream service is not yet
 running
Date: Wed, 22 Mar 2023 00:02:13 +0100

Closing.
Workaround with using "127.0.0.1" instead of "localhost" provided.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.