GNU bug report logs - #49127
Performance degradation in encode_coding_object

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Victor Nawothnig <victor.nawothnig@HIDDEN>; dated Sun, 20 Jun 2021 08:19:03 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 49127 <at> debbugs.gnu.org:


Received: (at 49127) by debbugs.gnu.org; 24 Jun 2021 16:50:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 24 12:50:01 2021
Received: from localhost ([127.0.0.1]:44008 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lwSYD-0007tX-CK
	for submit <at> debbugs.gnu.org; Thu, 24 Jun 2021 12:50:01 -0400
Received: from eggs.gnu.org ([209.51.188.92]:35836)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1lwSYB-0007tI-NH
 for 49127 <at> debbugs.gnu.org; Thu, 24 Jun 2021 12:50:00 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:49550)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1lwSY6-0004iw-Cj; Thu, 24 Jun 2021 12:49:54 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2224
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1lwSY5-0006d4-ED; Thu, 24 Jun 2021 12:49:54 -0400
Date: Thu, 24 Jun 2021 19:49:41 +0300
Message-Id: <83y2azryvu.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: victor.nawothnig@HIDDEN
In-Reply-To: <83eecwvrd0.fsf@HIDDEN> (message from Eli Zaretskii on Sun, 20
 Jun 2021 12:04:59 +0300)
Subject: Re: bug#49127: Performance degradation in encode_coding_object
References: <ACD4B349-8EAE-4697-99B0-0C3F2C678E93@HIDDEN>
 <83eecwvrd0.fsf@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 49127
Cc: 49127 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Ping!  Could you please respond to my requests below?  I'd like to
make some progress with this bug report.

> Date: Sun, 20 Jun 2021 12:04:59 +0300
> From: Eli Zaretskii <eliz@HIDDEN>
> Cc: 49127 <at> debbugs.gnu.org
> 
> > Date: Sun, 20 Jun 2021 08:30:24 +0200
> > From:  Victor Nawothnig via "Bug reports for GNU Emacs,
> >  the Swiss army knife of text editors" <bug-gnu-emacs@HIDDEN>
> > 
> > With gprof/prof_events I have nailed the problem to be encode_coding_object looping over all markers. In degenerate cases this list can contain millions of markers. Traversing this list is particularly slow because of the indirection being a singly linked list. Based on the fact that a GC remedies this, I’m assuming this list contains mostly  unreachable markers. When stepping through encode_coding_object with GDB after a GC this list of markers shrinks to small double digit numbers from millions.
> > 
> > The source of these markers appears to be looking-at in the font locking code of haskell-mode, this assumption is based on the fact that commenting out the uses of looking-at in haskell-mode prevents the accumulation of markers and thus the slowdown.
> 
> Do you understand why using looking-at causes creation of markers?  If
> so, can you show the details of why this happens?
> 
> > One contributing factor to all of this, is that for lsp-mode to perform adequately, one needs a relatively high gc-cons-threshold, which means GCs that would clean up the markers run more rarely, leading to higher accumulation of markers over time.
> 
> Yes, playing with GC threshold is usually a bad idea, but it is hard
> to explain to people why, and they keep doing that, to their cost.
> 
> > This problem only triggers in terminal frames, but not in GUI frames. Setting GDB breakpoints suggests that the GUI frame never even calls into encode_coding_object.
> 
> Can you should a backtrace from the call to encode_coding_object,
> including the Lisp backtrace (via the "xbacktrace" command)?
> 
> > So far I’m torn on whether this is a bug in the haskell-mode font locking code or in Emacs. What do you think?
> 
> Let's revisit this question after we have all the data I requested
> above, okay?
> 
> Thanks.
> 
> 
> 
> 





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#49127; Package emacs. Full text available.

Message received at 49127 <at> debbugs.gnu.org:


Received: (at 49127) by debbugs.gnu.org; 20 Jun 2021 09:04:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jun 20 05:04:57 2021
Received: from localhost ([127.0.0.1]:33013 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lutNw-0003lB-Q7
	for submit <at> debbugs.gnu.org; Sun, 20 Jun 2021 05:04:57 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39812)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1lutNu-0003kt-KS
 for 49127 <at> debbugs.gnu.org; Sun, 20 Jun 2021 05:04:55 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:34928)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1lutNp-0000Za-AK; Sun, 20 Jun 2021 05:04:49 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2437
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1lutNo-0000tD-Ti; Sun, 20 Jun 2021 05:04:49 -0400
Date: Sun, 20 Jun 2021 12:04:59 +0300
Message-Id: <83eecwvrd0.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Victor Nawothnig <victor.nawothnig@HIDDEN>
In-Reply-To: <ACD4B349-8EAE-4697-99B0-0C3F2C678E93@HIDDEN>
 (bug-gnu-emacs@HIDDEN)
Subject: Re: bug#49127: Performance degradation in encode_coding_object
References: <ACD4B349-8EAE-4697-99B0-0C3F2C678E93@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 49127
Cc: 49127 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> Date: Sun, 20 Jun 2021 08:30:24 +0200
> From:  Victor Nawothnig via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@HIDDEN>
> 
> With gprof/prof_events I have nailed the problem to be encode_coding_object looping over all markers. In degenerate cases this list can contain millions of markers. Traversing this list is particularly slow because of the indirection being a singly linked list. Based on the fact that a GC remedies this, I’m assuming this list contains mostly  unreachable markers. When stepping through encode_coding_object with GDB after a GC this list of markers shrinks to small double digit numbers from millions.
> 
> The source of these markers appears to be looking-at in the font locking code of haskell-mode, this assumption is based on the fact that commenting out the uses of looking-at in haskell-mode prevents the accumulation of markers and thus the slowdown.

Do you understand why using looking-at causes creation of markers?  If
so, can you show the details of why this happens?

> One contributing factor to all of this, is that for lsp-mode to perform adequately, one needs a relatively high gc-cons-threshold, which means GCs that would clean up the markers run more rarely, leading to higher accumulation of markers over time.

Yes, playing with GC threshold is usually a bad idea, but it is hard
to explain to people why, and they keep doing that, to their cost.

> This problem only triggers in terminal frames, but not in GUI frames. Setting GDB breakpoints suggests that the GUI frame never even calls into encode_coding_object.

Can you should a backtrace from the call to encode_coding_object,
including the Lisp backtrace (via the "xbacktrace" command)?

> So far I’m torn on whether this is a bug in the haskell-mode font locking code or in Emacs. What do you think?

Let's revisit this question after we have all the data I requested
above, okay?

Thanks.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#49127; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 20 Jun 2021 08:18:54 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jun 20 04:18:54 2021
Received: from localhost ([127.0.0.1]:32994 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lusfN-0000CJ-L8
	for submit <at> debbugs.gnu.org; Sun, 20 Jun 2021 04:18:53 -0400
Received: from lists.gnu.org ([209.51.188.17]:38224)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <victor.nawothnig@HIDDEN>) id 1luqyZ-0003xI-EK
 for submit <at> debbugs.gnu.org; Sun, 20 Jun 2021 02:30:35 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:51338)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <victor.nawothnig@HIDDEN>)
 id 1luqyZ-0001qi-5E
 for bug-gnu-emacs@HIDDEN; Sun, 20 Jun 2021 02:30:35 -0400
Received: from mr85p00im-zteg06023901.me.com ([17.58.23.192]:45945)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <victor.nawothnig@HIDDEN>)
 id 1luqyW-0002c1-0A
 for bug-gnu-emacs@HIDDEN; Sun, 20 Jun 2021 02:30:34 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com;
 s=1a1hai; t=1624170627;
 bh=On+gBYlSFol7/uYoWXY0dGO7ecWymoDIKqDDornfQDY=;
 h=Content-Type:From:Mime-Version:Subject:Message-Id:Date:To;
 b=xVclvyqhEW67enSt+g6bi7KGHfRX7bF8i0v8kxlhtT+gsRX9rH75zk93JFrxwRas1
 mJbn6VAUhjYEi79jcaXio4ASKF6gwSw+R6E/F5PKWXvPIT+wAOOels5o/5d8o5DuQp
 JwacKkP5BDQ9PlbEbgvZNuqGX71VE1uzAJjN/JEjGR7r9vog/8qWwqmzbhSlVgtKug
 21XGHkp3swu0cwQZOM72lTpaqGs95GomXPYDpQM6KNb4mwj7WDrPpmZXmWeP2kU8yQ
 fFsep5Fu8GAT5RxPVFSDMUzMsM4vK63onrelGT4kAXItTT0b+D8XAxzlxPGr6dbE02
 1MgOLZTfsIZuA==
Received: from smtpclient.apple (p508bae06.dip0.t-ipconnect.de [80.139.174.6])
 by mr85p00im-zteg06023901.me.com (Postfix) with ESMTPSA id
 69BF768023F
 for <bug-gnu-emacs@HIDDEN>; Sun, 20 Jun 2021 06:30:27 +0000 (UTC)
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From: Victor Nawothnig <victor.nawothnig@HIDDEN>
Mime-Version: 1.0 (1.0)
Subject: Performance degradation in encode_coding_object
Message-Id: <ACD4B349-8EAE-4697-99B0-0C3F2C678E93@HIDDEN>
Date: Sun, 20 Jun 2021 08:30:24 +0200
To: bug-gnu-emacs@HIDDEN
X-Mailer: iPad Mail (18F72)
X-Proofpoint-Virus-Version: =?UTF-8?Q?vendor=3Dfsecure_engine=3D1.1.170-22c6f66c430a71ce266a39bfe25bc?=
 =?UTF-8?Q?2903e8d5c8f:6.0.391,18.0.790,17.0.607.475.0000000_definitions?=
 =?UTF-8?Q?=3D2021-06-19=5F15:2021-06-18=5F01,2021-06-19=5F15,2020-04-07?=
 =?UTF-8?Q?=5F01_signatures=3D0?=
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0
 malwarescore=0
 bulkscore=0 clxscore=1011 adultscore=0 mlxscore=0 spamscore=0 phishscore=0
 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2106200043
Received-SPF: pass client-ip=17.58.23.192;
 envelope-from=victor.nawothnig@HIDDEN; helo=mr85p00im-zteg06023901.me.com
X-Spam_score_int: -8
X-Spam_score: -0.9
X-Spam_bar: /
X-Spam_report: (-0.9 / 5.0 requ) BAYES_40=-0.001, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Sun, 20 Jun 2021 04:18:53 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

Hi,

All of the following applies to Emacs 27.1 and 28.0.50.

Im currently debugging a performance degradation in haskell-mode. When editi=
ng code in a terminal frame, over time as I make modifications to source cod=
e, redrawing lines during scrolling becomes continuously slower to the point=
 that it sometimes takes up 200ms for a single line to draw. This problem di=
sappears once the GC runs.

With gprof/prof_events I have nailed the problem to be encode_coding_object l=
ooping over all markers. In degenerate cases this list can contain millions o=
f markers. Traversing this list is particularly slow because of the indirect=
ion being a singly linked list. Based on the fact that a GC remedies this, I=
=E2=80=99m assuming this list contains mostly  unreachable markers. When ste=
pping through encode_coding_object with GDB after a GC this list of markers s=
hrinks to small double digit numbers from millions.

The source of these markers appears to be looking-at in the font locking cod=
e of haskell-mode, this assumption is based on the fact that commenting out t=
he uses of looking-at in haskell-mode prevents the accumulation of markers a=
nd thus the slowdown.

One contributing factor to all of this, is that for lsp-mode to perform adeq=
uately, one needs a relatively high gc-cons-threshold, which means GCs that w=
ould clean up the markers run more rarely, leading to higher accumulation of=
 markers over time.

This problem only triggers in terminal frames, but not in GUI frames. Settin=
g GDB breakpoints suggests that the GUI frame never even calls into encode_c=
oding_object.

So far I=E2=80=99m torn on whether this is a bug in the haskell-mode font lo=
cking code or in Emacs. What do you think?

Kind regards,
Victor=




Acknowledgement sent to Victor Nawothnig <victor.nawothnig@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#49127; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Thu, 24 Jun 2021 17:00:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.