GNU logs - #41865, boring messages

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#41865: ns_select behavior in "recent" (2013+, 10.9/Mavericks+) mac-osx
Resent-From: Lin Xu <lin@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Mon, 15 Jun 2020 02:39:02 +0000
Resent-Message-ID: <handler.41865.B.15921887322011 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 41865 <at> debbugs.gnu.org
MIME-Version: 1.0
From: Lin Xu <lin@HIDDEN>
Date: Sun, 14 Jun 2020 16:23:55 -0700
Message-ID: <CA+c-4VUHKiNAC4+h-nv7qvjbCcp8EJSPSTkjuh9GLK89Z8OZLw@HIDDEN>
Content-Type: multipart/alternative; boundary="000000000000c29e4305a8139cb6"
Received-SPF: none client-ip=2607:f8b0:4864:20::632;
 envelope-from=lin@HIDDEN; helo=mail-pl1-x632.google.com
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001,
 SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=_AUTOLEARN
X-Spam_action: no action
X-Spam-Score: -2.3 (--)
X-Mailman-Approved-At: Sun, 14 Jun 2020 22:38:49 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

--000000000000c29e4305a8139cb6
Content-Type: text/plain; charset="UTF-8"

Hi,

This might be more of a discussion than a bug, or a documentation
suggestion, or something like that.

During debugging of a performance issue, I discovered that `ns_select` in
nsterm.m uses `scheduledTimerWithTimeInterval` to rewake the run loop when
it is called with "only" a timeout. This would occur frequently if a timer
is scheduled with `run-with-timer` only. For example, emacs -Q with

(run-with-timer 1 1 (lambda ()
                      (print (format "%s" (format-time-string "%s"))
                             'external-debugging-output)))

And tab away, eg. have `focus-out` occur, e.g. when
`applicationDidResignActive` occurs. You will notice that after a little
time (this depends on whether you are using battery or not), the unix epoch
time printed will be delayed by ~10 seconds, and then (of course) code in
`timer.el` repeats the timer to catch up.

This is because in 2013, OSX Mavericks added "appnap", which is a feature
designed to conserve battery life and power consumption. One thing it does
is to lower timer frequencies for background tasks.

More context:

1. elisp documentation "timers" clearly already says that a timer might be
run multiple times after being delayed if "repeat" is given.
2. `timer-max-repeats` exists as a defcustom, with a default of 10 (!).
3. `suspend-frame' in TTY will do something similar, but it won't run any
timers while suspended, of course; but, when you unsuspend all the timers
"catch up".
4. I grepped built-in elisp. There are actually only very few cases where
people use repeat for `run-with-timer`, `autorevert` being a major one.
5. I grepped major emacs packages, company, lsp, doom, spacemacs, etc. and
there's almost no usage of `run-with-timer` with repeat.

A few options occur to me:

1. Add more words in documentation in the timers node to clarify that
things like suspending, OS decisions, GC, could impact when a rescheduled
timer might run again. Based on my brief survey, almost everyone really
wants "run again, once". Make it even clearer that is probably what you
really want.

2. Support the usual use case better by making `timer-max-repeats` bindable
around `run-with-timer`, so that one could control how often a late timer
is rerun in a row. It's more likely that the developer of a timer really
knows how often it should run then a user, to be honest.

3. Relatedly, consider changing the default value of `timer-max-repeats' to
1. There seems to be no point having this value be configurable to me
actually. Timer code must already be robust to being rate limited. In what
use case is 10 better than 1? Changing defaults is always impactful,
though... so maybe not.

4. And/or, extend `run-with-timer` with a new API argument, or have a new
function `run-with-timer-exactly-once`, or something, to better cater for
the more common use case.

5. In either case, update `auto-revert` to run `auto-revert-buffers` once
exactly. There's no point running the revert timer handler again and again
in very quick succession - the default being 10 - after being suspended or
delayed. Especially in the case of remote connections, this might be
expensive. Also, various packages use those hooks to do further heavyweight
actions (magit, lsp, eg.)

6. Extend ns* layer code to understand the timer list, so that timers can
be scheduled at the OS level, which allows one to suggest to the OS that
certain tasks are user-initiated or background tasks - e.g "refresh this
buffer tailing" versus "autosave/gc on idle". MacOS has this level of
granularity. This would be a giant undertaking.

My suggestion would be 1+2+5.

(4) seems a little adventurous, and perhaps overkill. However, guiding
developers to do the right thing easily is the key goal of API design, so
maybe it's worth the extra API surface to support. I would hypothesize that
power management & control of scheduled tasks is likely to spread to other
OSes - I haven't checked but I wouldn't be surprised it exists on Windows
already - so there might be value there for the longer term future.

(5) is scary work, and I looked at the number of reverted attempts to
improve `ns_select` in `nsterm.m` since 2013. It would also imply deeply
understanding/changing how the various process.c / keyboard.c / read /
event handling code works; most of that is OS platform independent and
pretty gnarly, so it would be tough going. Perhaps in some long term future
it might be interesting to attempt - deeper integration of OS scheduling
features into emacs would probably improve latency and performance. I
mentioned it here because in a pure sense this would be the "best" fix, but
I think the tradeoffs are not worth it, unless there is platform wide
support for that across Linux, windows & mac osx.

If there is interest for 1 + 2 + 5, I would be willing to submit a patch
along those lines in parts for each.

Thanks,

Lin

--000000000000c29e4305a8139cb6
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi,<div><br></div><div>This might be more of a discussion =
than a bug, or a documentation suggestion, or something like that.<br><div>=
<br></div><div>During debugging of a performance issue, I discovered that `=
ns_select` in nsterm.m uses `scheduledTimerWithTimeInterval` to rewake the =
run loop when it is called with &quot;only&quot; a timeout. This would occu=
r frequently if a timer is scheduled with `run-with-timer` only. For exampl=
e, emacs -Q with</div><div><br></div><div>(run-with-timer 1 1 (lambda ()<br=
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 (print (format &quot;%s&quot; (format-time-string &quot;%s&quot;))<br>=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0&#39;external-debugging-output)))<br></div><=
div><br></div><div>And tab away, eg. have `focus-out` occur, e.g. when `app=
licationDidResignActive` occurs. You will notice that after a little time (=
this depends on whether you are using battery or not), the unix epoch time =
printed will be delayed by ~10 seconds, and then (of course) code in `timer=
.el` repeats the timer to catch up.=C2=A0</div><div></div><div><br></div><d=
iv>This is because in 2013, OSX Mavericks added &quot;appnap&quot;, which i=
s a feature designed to conserve battery life and power consumption. One th=
ing it does is to lower timer frequencies for background tasks.</div></div>=
<div><br>More context:</div><div><br></div><div>1. elisp documentation &quo=
t;timers&quot; clearly already says that a timer might be run multiple time=
s after being delayed if &quot;repeat&quot; is given.<br></div><div>2. `tim=
er-max-repeats` exists as a defcustom, with a default of 10 (!).=C2=A0</div=
><div>3. `suspend-frame&#39; in TTY will do something similar, but it won&#=
39;t run any timers while suspended, of course; but, when you unsuspend all=
 the timers &quot;catch up&quot;.</div><div>4. I grepped=C2=A0built-in elis=
p. There are actually only very few cases where people use repeat for `run-=
with-timer`, `autorevert` being a major one.</div><div>5. I grepped major e=
macs packages, company, lsp, doom, spacemacs, etc. and there&#39;s almost n=
o usage of `run-with-timer` with=C2=A0repeat.</div><div><br></div><div>A fe=
w options occur to me:</div><div><br></div><div>1. Add more words in docume=
ntation in the timers node to clarify that things like suspending, OS decis=
ions, GC, could impact when a rescheduled timer might run again. Based on m=
y brief survey, almost everyone really wants &quot;run again, once&quot;. M=
ake it even clearer that is probably what you really want.</div><div><br></=
div><div>2. Support the usual use case better by making `timer-max-repeats`=
 bindable around `run-with-timer`, so that one could=C2=A0control how often=
 a late timer is rerun in a row. It&#39;s more likely that the developer of=
 a timer really knows how often it should run then a user, to be honest.=C2=
=A0</div><div><br></div><div>3. Relatedly, consider changing the default va=
lue of `timer-max-repeats&#39; to 1. There seems to be no point having this=
 value be configurable to me actually. Timer code must already be robust to=
 being rate limited. In what use case is 10 better than 1? Changing default=
s is always impactful, though... so maybe not.<br></div><div><br></div><div=
>4. And/or, extend `run-with-timer` with a new API argument, or have a new =
function `run-with-timer-exactly-once`, or something, to better cater for t=
he more common use case.=C2=A0</div><div><br></div><div></div><div>5. In ei=
ther case, update `auto-revert` to run `auto-revert-buffers` once exactly. =
There&#39;s no point running the revert timer handler again and again in ve=
ry quick succession - the default being 10 - after being suspended or delay=
ed. Especially in the case of remote connections, this might be expensive. =
Also, various  packages use those hooks to do further heavyweight actions (=
magit, lsp, eg.)</div><div><br></div><div>6. Extend ns* layer code to under=
stand the timer list, so that timers can be scheduled at the OS level, whic=
h allows one to suggest to the OS that certain tasks are user-initiated or =
background tasks - e.g &quot;refresh this buffer tailing&quot; versus &quot=
;autosave/gc on idle&quot;. MacOS has this level of granularity. This would=
 be a giant undertaking.</div><div><br></div><div>My suggestion would be 1+=
2+5.=C2=A0</div><div><br></div><div>(4) seems a little adventurous, and per=
haps overkill. However, guiding developers to do the right thing easily is =
the key goal of API design, so maybe it&#39;s worth the extra API surface t=
o support. I would hypothesize that power management &amp; control of sched=
uled tasks is likely to spread to other OSes - I haven&#39;t checked but I =
wouldn&#39;t be surprised it exists on Windows already - so there might be =
value there for the longer term future.</div><div><br></div><div>(5) is sca=
ry work, and I looked at the number of reverted attempts to improve `ns_sel=
ect` in `nsterm.m` since 2013. It would also imply deeply understanding/cha=
nging how the various process.c / keyboard.c / read / event handling code w=
orks; most of that is OS platform independent and pretty gnarly, so it woul=
d be tough going. Perhaps in some long term future it might be interesting =
to attempt - deeper integration of OS scheduling features into emacs would =
probably improve latency and performance. I mentioned it here because in a =
pure sense this would be the &quot;best&quot; fix, but I think the tradeoff=
s are not worth it, unless there is platform wide support for that across L=
inux, windows &amp; mac osx.</div><div><br></div><div>If there is interest =
for 1=C2=A0+ 2 + 5, I would be willing to submit a patch along those lines =
in parts for each.</div><div><br></div><div>Thanks,</div><div><br></div><di=
v>Lin</div></div>

--000000000000c29e4305a8139cb6--

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Lin Xu <lin@HIDDEN>
Subject: bug#41865: Acknowledgement (ns_select behavior in "recent"
 (2013+, 10.9/Mavericks+) mac-osx)
Message-ID: <handler.41865.B.15921887322011.ack <at> debbugs.gnu.org>
References: <CA+c-4VUHKiNAC4+h-nv7qvjbCcp8EJSPSTkjuh9GLK89Z8OZLw@HIDDEN>
X-Gnu-PR-Message: ack 41865
X-Gnu-PR-Package: emacs
Reply-To: 41865 <at> debbugs.gnu.org
Date: Mon, 15 Jun 2020 02:39:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 41865 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
41865: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D41865
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 2 Sep 2021 22:39:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 02 18:39:43 2021
Received: from localhost ([127.0.0.1]:42306 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mLvN0-0001gU-PE
	for submit <at> debbugs.gnu.org; Thu, 02 Sep 2021 18:39:43 -0400
Received: from outbound.soverin.net ([116.202.126.228]:48711)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <alan@HIDDEN>) id 1mLvMl-0001g2-NF
 for control <at> debbugs.gnu.org; Thu, 02 Sep 2021 18:39:41 -0400
Received: from smtp.soverin.net (unknown [10.10.3.24])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (No client certificate requested)
 by outbound.soverin.net (Postfix) with ESMTPS id C5FBA53
 for <control <at> debbugs.gnu.org>; Thu,  2 Sep 2021 22:39:21 +0000 (UTC)
Received: from smtp.soverin.net (smtp.soverin.net [159.69.232.138]) by
 soverin.net
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=idiocy.org; s=soverin;
 t=1630622361; bh=s7AMkvcsXHzMkz5vJA2jX4984Po96IbJ0c12YUttLxQ=;
 h=To:From:Subject:Date:From;
 b=nU/T+kXGYlaeZh36v18nNs/kHmia0PjN++va6Tb0QE7ooUT1ugRf5hq5jAV3GROTs
 fHDrkxBsnVVZzNzejDjCbGTM1XzhZS517cnkRpg0tCgcjhmWrHt4HkdPd/zcIAt0bs
 2Q/YoJ7guBEilm6Xt9nM1DEAnrfXaZL88ynP9oIJSsnKcDVlVjUO72G6tqk/xccQQM
 tPKUkh1tFVvXakuX/hu5pVCo5TdNcle8FzUATxXowwK/wRf/Em9Sb5nZmJKJVUJut3
 YuMBkub3/IuE5L4ftcLH/qTXXGrQ41J8+fLZPM8WAEIt/N4tG5Jq6B/3wg+I9e4HRP
 4JT2ClrnbcRfg==
Received: from alan by faroe.holly.idiocy.org with local (Exim 4.94.2)
 (envelope-from <alan@HIDDEN>) id 1mLvMd-001A6Y-JL
 for control <at> debbugs.gnu.org; Thu, 02 Sep 2021 23:39:19 +0100
To: control <at> debbugs.gnu.org
From: Alan Third <alan@HIDDEN>
Subject: control message for bug #41865
Message-Id: <E1mLvMd-001A6Y-JL@HIDDEN>
Date: Thu, 02 Sep 2021 23:39:19 +0100
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

merge 41865 24849
quit

Last modified: Thu, 2 Sep 2021 22:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.