GNU logs - #77390, boring messages


Message sent to bug-guix@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: bug#77390: Shepherd: bug in timer scheduling on daylight savings time change
Resent-From: "Timo Wilken" <guix@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Sun, 30 Mar 2025 20:05:02 +0000
Resent-Message-ID: <handler.77390.B.174336508927453 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: report 77390
X-GNU-PR-Package: guix
X-GNU-PR-Keywords: 
To: 77390 <at> debbugs.gnu.org
X-Debbugs-Original-To: <bug-guix@HIDDEN>
Received: via spool by submit <at> debbugs.gnu.org id=B.174336508927453
          (code B ref -1); Sun, 30 Mar 2025 20:05:02 +0000
Received: (at submit) by debbugs.gnu.org; 30 Mar 2025 20:04:49 +0000
Received: from localhost ([127.0.0.1]:38175 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1tyytj-00078e-Us
	for submit <at> debbugs.gnu.org; Sun, 30 Mar 2025 16:04:48 -0400
Received: from lists.gnu.org ([2001:470:142::17]:52410)
 by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.84_2) (envelope-from <timo@HIDDEN>) id 1tyyth-000783-EZ
 for submit <at> debbugs.gnu.org; Sun, 30 Mar 2025 16:04:46 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <timo@HIDDEN>) id 1tyytR-0006ZC-2j
 for bug-guix@HIDDEN; Sun, 30 Mar 2025 16:04:31 -0400
Received: from mx2.mythic-beasts.com ([2a00:1098:0:82:1000:0:2:1])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <timo@HIDDEN>) id 1tyytN-00021E-1I
 for bug-guix@HIDDEN; Sun, 30 Mar 2025 16:04:27 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=twilken.net
 ; s=mythic-beasts-k1; h=To:From:Subject:Date;
 bh=uotxTyu6NZJZ61+GBfSqAGv1Akk1RbYpaC1geEiZozc=; b=UO75YAqP10Nz5Lpyd2LqRM1TAd
 XT7GqU/trfMbHI7J7j45eRGyLkrxPqTG5N77UG6ZRZesvk1usa4U01pG72iOjErVwGNI4Bb6RKo5Y
 qJx8REbLjYV7SvhtAY0iMdieckQYMe20b0UD4e/MuVTyytRQCY4yTXHT0OTrsylWHkI9JLXBuCO3x
 SP3yBn6E5YnIqd0TGTeDfvBYv0F7Ggsb7SyYUrRqv5kIFQFZnXX76oW98cO6IdyM++iNcZE2J/0QR
 oI7aT59QRIJCpflTYKgoxwxj4hbp+L/MF0BCiuLyUz3TDRdHCrJf474bM9tTkv5H1WotwNv/faydQ
 nfc3Uvkg==;
Received: by mailhub-hex-d.mythic-beasts.com with esmtpsa (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2)
 (envelope-from <timo@HIDDEN>) id 1tyytK-008UvH-Sv
 for bug-guix@HIDDEN; Sun, 30 Mar 2025 21:04:23 +0100
Mime-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8
Date: Sun, 30 Mar 2025 22:04:07 +0200
Message-Id: <D8TVBYDNEVB3.2M6JZ0D96HFML@HIDDEN>
From: "Timo Wilken" <guix@HIDDEN>
X-Mailer: aerc 0.20.1
X-BlackCat-Spam-Score: 7
X-Spam-Status: No, score=0.7
Received-SPF: pass client-ip=2a00:1098:0:82:1000:0:2:1;
 envelope-from=timo@HIDDEN; helo=mx2.mythic-beasts.com
X-Spam_score_int: -43
X-Spam_score: -4.4
X-Spam_bar: ----
X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.0 (+)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

Hi,

Last night, my server's time changed from CET to CEST automatically, as it =
is
set to Europe/Berlin time.

I have a Shepherd timer set to run every night at 02:00 local time:
https://cgit.twilken.net/dotfiles/tree/tw/system.scm?h=3Dcd39f5d3340c0bcd59=
bb80d3ff4f12741db75e07#n136

--8<---------------cut here
   (simple-service 'disk-maintenance shepherd-root-service-type
     (list (shepherd-service
            (provision '(guix-gc))
            (requirement '(user-processes file-systems))
            (documentation "Delete unused files from Guix' store.")
            (modules '((shepherd service timer)))
            (start #~(make-timer-constructor
                      (calendar-event #:hours '(2) #:minutes '(0))
                      (command (list "guix" "gc" "-d" "2w"))))
            (stop #~(make-timer-destructor))
            (actions (list (shepherd-action
                            (name 'trigger) (procedure #~trigger-timer)
                            (documentation "Collect garbage now.")))))))
--8<---------------cut here

Last night, this hit the exact moment of the CET/CEST change and seems to h=
ave
confused the Shepherd a lot, to the point where it kept spamming the same
message to /var/log/messages, and used up 80% of my system's total memory,
according to htop. (This led to various things getting oom-killed, and the
system grinding to a halt until I had a look at it in the afternoon.)

First, at 01:00, the Shepherd spawned about 30 processes for this one timer=
:

--8<---------------cut here
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1691.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'nextcloud-cron' spawned p=
rocess 1692.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for nextc=
loud-cron.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'gpsplot' spawned process =
1693.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for gpspl=
ot.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'dynamic-dns-ipv4.api.myth=
ic-beasts.com' spawned process 1694.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for dynam=
ic-dns-ipv4.api.mythic-beasts.com.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1698.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1703.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1713.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1715.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1723.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1726.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1732.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1736.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1740.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1750.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1754.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1761.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1766.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1768.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1778.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1785.
2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1791.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1806.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1811.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1818.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1822.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1829.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1838.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1848.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1851.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1862.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1872.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1876.
2025-03-30 01:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 01:00:02 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1883.
2025-03-30 01:00:02 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 03:00:00 localhost linux: [46479.948177] perf: interrupt took to=
o long (2504 > 2500), lowering kernel.perf_event_max_sample_rate to 79750
2025-03-30 03:00:01 localhost linux: [47490.137060] perf: interrupt took to=
o long (3138 > 3130), lowering kernel.perf_event_max_sample_rate to 63500
2025-03-30 03:00:01 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1887.
2025-03-30 03:00:01 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 03:00:02 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1896.
2025-03-30 03:00:02 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 03:00:03 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1910.
2025-03-30 03:00:03 localhost shepherd[1]: Registering new logger for guix-=
gc.
2025-03-30 03:00:03 localhost shepherd[1]: Timer 'guix-gc' spawned process =
1915.
--8<---------------cut here

I then get 731(!!) lines like the following over the course of 3 seconds:

--8<---------------cut here
2025-03-30 03:00:03 localhost shepherd[1]: Waiting anew for timer 'guix-gc'=
 (resuming from sleep state?).
[...]
2025-03-30 03:00:06 localhost shepherd[1]: Waiting anew for timer 'guix-gc'=
 (resuming from sleep state?).
--8<---------------cut here

The Shepherd then gets more confused and starts spamming the same message o=
ver
and over again, but now without newlines, leading to lines like:

--8<---------------cut here
2025-03-30 03:00:06 localhost shepherd[1]: Waiting anew for timer 'guix-gc'=
 (resuming from sleep state?).Waiting anew for timer 'guix-gc' (resuming fr=
om sleep state?).Waiting anew for timer 'guix-gc' (resuming from sleep stat=
e?).Waiting anew for timer 'guix-gc' (resuming from sleep state?).Waiting a=
new for timer 'guix-gc' (resuming from sleep state?).Waiting anew for timer=
 'guix-gc' (resuming from sleep state?).Waiting anew for timer 'guix-gc' (r=
esuming from sleep state?).Waiting anew for timer 'guix-gc' (resuming from =
sleep state?).Waiting anew for
2025-03-30 03:00:06 localhost shepherd[1]:  timer 'guix-gc' (resuming from =
sleep state?).Waiting anew for timer 'guix-gc' (resuming from sleep state?)=
.Waiting anew for timer 'guix-gc' (resuming from sleep state?).Waiting anew=
 for timer 'guix-gc' (resuming from sleep state?).Waiting anew for timer 'g=
uix-gc' (resuming from sleep state?).Waiting anew for timer 'guix-gc' (resu=
ming from sleep state?).Waiting anew for timer 'guix-gc' (resuming from sle=
ep state?).Waiting anew for timer 'guix-gc' (resuming from sleep state?).Wa=
iting anew for timer 'guix-gc'
--8<---------------cut here

Then there's some output from the various "guix gc" processes spawned by th=
e
timer, which of course all step on each other's toes. The Shepherd keeps
happily writing the above lines out in large blocks.

It goes on like this until the afternoon, when the OOM kills start (probabl=
y
when I start SSH'ing in to see what's going on):

--8<---------------cut here
2025-03-30 15:21:44 localhost linux: [93869.155828] sshd-session invoked oo=
m-killer: gfp_mask=3D0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=3D0, =
oom_score_adj=3D0
--8<---------------cut here

After a bit of that, I decide to reboot the machine.

I've saved the system logs from that night - let me know if you need more d=
etails.

I'm running Shepherd 1.0.3, as packaged in Guix:

--8<---------------cut here
Generation 235	Mar 29 2025 12:07:15	(current)
  file name: /var/guix/profiles/system-235-link
  canonical file name: /gnu/store/p7infvmm678nysvdg3xvxlr83xxkj66j-system
  label: GNU with Linux-Libre 6.13.8
  bootloader: grub-efi
  root device: UUID: c63af3e6-3c2b-43d2-b1e6-944f09a10e0f
  kernel: /gnu/store/d7z9027yrs1vfmpvds10iqfl69cyvixn-linux-libre-6.13.8/bz=
Image
  channels:
    tw:
      repository URL: https://git.twilken.net/dotfiles
      branch: master
      commit: 39abf96d68ce500abbd55f9bdbd68e1c63282d36
    guix:
      repository URL: https://git.savannah.gnu.org/git/guix.git
      branch: master
      commit: 67f4a2feadbb6cc04a61b5a3bc7b8f4bace2a0bb
    [non-free channels omitted]
--8<---------------cut here

Cheers,
Timo




Message sent:


Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: "Timo Wilken" <guix@HIDDEN>
Subject: bug#77390: Acknowledgement (Shepherd: bug in timer scheduling on
 daylight savings time change)
Message-ID: <handler.77390.B.174336508927453.ack <at> debbugs.gnu.org>
References: <D8TVBYDNEVB3.2M6JZ0D96HFML@HIDDEN>
X-Gnu-PR-Message: ack 77390
X-Gnu-PR-Package: guix
Reply-To: 77390 <at> debbugs.gnu.org
Date: Sun, 30 Mar 2025 20:05:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-guix@HIDDEN

If you wish to submit further information on this problem, please
send it to 77390 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
77390: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D77390
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems


Message received at control <at> debbugs.gnu.org:


Received: (at control) by debbugs.gnu.org; 31 Mar 2025 13:10:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Mar 31 09:10:14 2025
Received: from localhost ([127.0.0.1]:40311 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1tzEu5-0002G0-Jz
	for submit <at> debbugs.gnu.org; Mon, 31 Mar 2025 09:10:14 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:40272)
 by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1tzEu3-0002Cr-1S
 for control <at> debbugs.gnu.org; Mon, 31 Mar 2025 09:10:11 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1tzEtw-0003J3-Ad
 for control <at> debbugs.gnu.org; Mon, 31 Mar 2025 09:10:05 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to:
 references; bh=uIs8M3B4Pe7/y98hFk8Bn+1Q0jLwve+PhB/UFjnKQfY=; b=YlHXb2qncuPsyH
 6QRtBCDLCMzdiCV29/3yYUW7UIE4KscO8NG4Eb/mt+nANccM6Ce/aIX+ukKtuHnaHuDRRnk8JtC00
 G/ES1REt0RqToRD9q3Z8huVeh2jzTaKIRWs54pMac3YBah6q356X0tGja7jjSrGauh2Dk2nxDBXt0
 ynKiezwnsKiiUZcMsW6saGXG869O8D4vaUH3hXQE4fErqRKi8ZEzGoobAqY2NQWRCEQCXHw/MS0Ks
 scxzEE/ihsFsm26ZlamJXJcgFlNUsRW+8mRRkrJpxTPClSJVyRCDQekhUWpFAoGjLq1Ry9sKarVPk
 I7kX50HyoJbXQyjzfi7Q==;
Date: Mon, 31 Mar 2025 15:09:38 +0200
Message-Id: <8734etmjn1.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Subject: control message for bug #77390
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

merge 77390 77401
quit





Message sent to bug-guix@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: bug#77390: Shepherd: bug in timer scheduling on daylight savings time change
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Mon, 31 Mar 2025 13:18:04 +0000
Resent-Message-ID: <handler.77390.B77390.174342703722682 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 77390
X-GNU-PR-Package: guix
X-GNU-PR-Keywords: 
To: "Timo Wilken" <guix@HIDDEN>
Cc: 77390 <at> debbugs.gnu.org
Received: via spool by 77390-submit <at> debbugs.gnu.org id=B77390.174342703722682
          (code B ref 77390); Mon, 31 Mar 2025 13:18:04 +0000
Received: (at 77390) by debbugs.gnu.org; 31 Mar 2025 13:17:17 +0000
Received: from localhost ([127.0.0.1]:40347 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1tzF0u-0005sz-7f
	for submit <at> debbugs.gnu.org; Mon, 31 Mar 2025 09:17:17 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:44394)
 by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1tzF0p-0005l1-0m
 for 77390 <at> debbugs.gnu.org; Mon, 31 Mar 2025 09:17:13 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1tzF0i-0004fw-TF; Mon, 31 Mar 2025 09:17:04 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
 From; bh=k6iuGWCMNCWxWGVItwVxIa1IK+XgNQKjEHkCdqMNvLU=; b=aBV/vPiBomi32g2Ni8ZZ
 +7LQgti0T2LOB9+ZmWMcG8TLxvtzhS/6gRkyu3nKi6MtL2mMd9/MJanOVBe4laPQravEt83PzsgHN
 MQkRIRXbPWDVhte4Dm0iYt2A7T2hjPusns7BQhsFsc+iDCfjNOMfOykCRYUphJ1H3rna05CQUNeBD
 2K+YJ+m/ezmWb/WbrF856EMwBkAWmoB5CecVU1IEZ7LSpRVbd9Au1Sj+FCMJ/6CTrnuV8Q6tPGlpi
 hhvk3Tg0P81QOzmVXrh9LLWT29br8vREVWk6XjM8gy4o0zZZZBCqyVwbECh52L4sk1jYtV3D+hR5v
 DSGav3JAVbPRwQ==;
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
In-Reply-To: <D8TVBYDNEVB3.2M6JZ0D96HFML@HIDDEN> (Timo Wilken's message
 of "Sun, 30 Mar 2025 22:04:07 +0200")
References: <D8TVBYDNEVB3.2M6JZ0D96HFML@HIDDEN>
Date: Mon, 31 Mar 2025 15:17:00 +0200
Message-ID: <87semtl4qb.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi Timo,

"Timo Wilken" <guix@HIDDEN> skribis:

> Last night, this hit the exact moment of the CET/CEST change and seems to=
 have
> confused the Shepherd a lot, to the point where it kept spamming the same
> message to /var/log/messages, and used up 80% of my system's total memory,
> according to htop. (This led to various things getting oom-killed, and the
> system grinding to a halt until I had a look at it in the afternoon.)
>
> First, at 01:00, the Shepherd spawned about 30 processes for this one tim=
er:
>
> --8<---------------cut here
> 2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned proces=
s 1691.
> 2025-03-30 01:00:00 localhost shepherd[1]: Timer 'nextcloud-cron' spawned=
 process 1692.
> 2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for nex=
tcloud-cron.
> 2025-03-30 01:00:00 localhost shepherd[1]: Timer 'gpsplot' spawned proces=
s 1693.
> 2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for gps=
plot.
> 2025-03-30 01:00:00 localhost shepherd[1]: Timer 'dynamic-dns-ipv4.api.my=
thic-beasts.com' spawned process 1694.
> 2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for dyn=
amic-dns-ipv4.api.mythic-beasts.com.
> 2025-03-30 01:00:00 localhost shepherd[1]: Timer 'guix-gc' spawned proces=
s 1698.
> 2025-03-30 01:00:00 localhost shepherd[1]: Registering new logger for gui=
x-gc.

I saw the exact same problem on berlin (aka. ci.guix.gnu.org):

  https://issues.guix.gnu.org/77401

but not on the other servers I co-administer; I suspect the difference
is that these other servers did not have a timer running at 01:00.

So I guess that the fix for <https://issues.guix.gnu.org/75622> was
bogus somehow?

To be continued=E2=80=A6

Ludo=E2=80=99.




Message received at control <at> debbugs.gnu.org:


Received: (at control) by debbugs.gnu.org; 31 Mar 2025 13:18:05 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Mar 31 09:18:05 2025
Received: from localhost ([127.0.0.1]:40355 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1tzF1g-0006UR-3r
	for submit <at> debbugs.gnu.org; Mon, 31 Mar 2025 09:18:05 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:41594)
 by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1tzF1L-00068j-NM
 for control <at> debbugs.gnu.org; Mon, 31 Mar 2025 09:17:44 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1tzF1F-0004le-Qr
 for control <at> debbugs.gnu.org; Mon, 31 Mar 2025 09:17:38 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to:
 references; bh=1zbc2wG5AFWXO8eq9So/SrnPDnJ4Z7TpF604FQZmPbI=; b=HyCPv7BfNWQ+pV
 lgDnZszSWBMs6Qi61HiycGp0HoMki9chLsB+iW5dLjFyfEzE5HDvMMqD783Jxre1uBi/JtglqoYNv
 CImQgkOTc/cvIZFNcjrAnAUfsW1x+f0b/caMWKmNckL8sEa2TRERyVfMx1TxG4yHpvw+cBrRpYGgK
 XvBOY9H1/O7LCUwhL8w2pODnmonlUMJIwmMPRrJtnXQuPuvIr5sHJkAgQo/JWa1+i3Anc0XvQv/UU
 7Q7h0q6+nBX+y+ic9jALjjn+5meRP2znvRtdrIyOitooTRWZs1OitOBoHWS7r+ErHzvnJthPMuD/d
 x/k6vlDK8g+nzCbKhGUA==;
Date: Mon, 31 Mar 2025 15:17:29 +0200
Message-Id: <87r02dl4pi.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Subject: control message for bug #77401
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

severity 77401 important
quit





Message received at control <at> debbugs.gnu.org:


Received: (at control) by debbugs.gnu.org; 31 Mar 2025 20:29:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Mar 31 16:29:03 2025
Received: from localhost ([127.0.0.1]:43091 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1tzLkk-0000Sz-R6
	for submit <at> debbugs.gnu.org; Mon, 31 Mar 2025 16:29:03 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:48698)
 by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1tzLki-0000Rt-Be
 for control <at> debbugs.gnu.org; Mon, 31 Mar 2025 16:29:00 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1tzLkc-00031J-Pf
 for control <at> debbugs.gnu.org; Mon, 31 Mar 2025 16:28:54 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to:
 references; bh=exCrlfDXhc1MQBEOvZui8/BrQDunupVBEN6UGZZRkvA=; b=iDt7XfXNSr4qT8
 0fxExQc5uZVflfu8q6XtTFwGD3KD1VyxzxcISnKXDGT2Rref7i0RouR5iqXjQzywE426PM8w+j/7w
 bLTfliPEnv0Uzrz88tD5QZuwx6E4JRZubNShxw+p/5Ue47cDvQewI5IkLJeBbFK/SjSMSvWgr0Qgy
 dniIQhFddjcjBBLafrsiUV9Kgl77n2Du4dZrPVOKV/bR3CMNFtqHu2MbdK5X7SmgZV+gqh5cmve3a
 DrvhIWV10PgQHFWTDuK8eWDKwN+pE1mAN3Gp+s5YJSPH9CotFne6N+3QVq62MOokbW6IEX/ONJhKO
 MzWYkOEPZ+Jf5JIhZaXA==;
Date: Mon, 31 Mar 2025 22:28:51 +0200
Message-Id: <87jz85j664.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Subject: control message for bug #77401
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

retitle 77401 [Shepherd] Timer spawns many commands on DST change (CET -> CEST)
quit






Last modified: Mon, 31 Mar 2025 20:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.