GNU bug report logs - #46881
28.0.50; pdumper dumping causes way too many syscalls

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Pip Cet <pipcet@HIDDEN>; dated Tue, 2 Mar 2021 20:35:01 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 13:16:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 08:16:58 2021
Received: from localhost ([127.0.0.1]:33212 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lIAK9-00049d-Mq
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 08:16:58 -0500
Received: from mail-oi1-f175.google.com ([209.85.167.175]:35934)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lIAK8-00044Q-Dh
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 08:16:57 -0500
Received: by mail-oi1-f175.google.com with SMTP id j1so2467027oiw.3
 for <46881 <at> debbugs.gnu.org>; Fri, 05 Mar 2021 05:16:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=H0SXET0ERFvOY8lTy7BnLVCeaYnyZz1ClNI5Aa6Rr1k=;
 b=KHBoAln4dkiPDt1K7dk/CD05wWfrGFrKkpTKjHkQPYDBZjKKFnNi6P3oRVe/MvrNjA
 XSzX4gLRvsqDDSqlvSScbcn7ZVmEwMUl/qEam6y7ytwRSTDiCJB6ZGAdgQDVQr/X3anG
 fsbeL4Iun4Lit+SgWujb8uKwamzSUx8u00y5tL9uGpA88mMrI7tcptdMj8szJuhxYeO/
 FN6Ue5nux35oPs+5G9b/rA7Q9Wp70x+E6JnNzeymQaxXWipY+jHdPOuJURRM9hSDOSuL
 vXE8HrRq1A2vH9rOj6lxeiNKA37W+Wf4obLz/hY4NpsyCehw3KjzJl7iKIMXrZrBX/jD
 V06Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=H0SXET0ERFvOY8lTy7BnLVCeaYnyZz1ClNI5Aa6Rr1k=;
 b=Uts56fRDR4fyEDNKdFLptg5fpO+5oOqjcifcbcvZB0U2Q/Br6jNInQ5q+/5aIWLY3N
 3vKI4TiJUkbeEiifnkHLHuuhp8fqEgCadE1EGgMPV3UpHnsJH/hT3+xv+xKR0prGgOHm
 L+9EAsUs0CC/uacUGO/c+2TFqXyINHKj2vgEI7HkZoTEo+8ViE0HGNa6LA20MIHcfqct
 yhSACMjy+gpRi+f1AHkoOyWfMS0+rE/Y9RbJstuHvWr8H50AxLiYgyrqgLTRN4EKzQ76
 rfiFnlcxVNOfoDbxxX67n5EQcGYnRuqCF2XqvEz0wKpA2234uZMK2EPitqVRkyQSIDMn
 oZDA==
X-Gm-Message-State: AOAM530epCuOWzRjHlEGLHr5X8tiMp5X4JdbS7E/XZGrSa5V9Z1XwWpg
 ZFwQrwZsK4jFOR+HzR4uC20jfz527cY07gO2Gx0=
X-Google-Smtp-Source: ABdhPJyrccCB5s04d5/bgzi+YG0ag4UF7eDxUZlwZGS/HcavT4D2asqz4WCtxIsxj8i5S6vrhF1jlW0+QMshPDXyH1M=
X-Received: by 2002:aca:4c0f:: with SMTP id z15mr3194359oia.44.1614950210855; 
 Fri, 05 Mar 2021 05:16:50 -0800 (PST)
MIME-Version: 1.0
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 <83a6riysnv.fsf@HIDDEN>
 <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
 <83zgzixcfy.fsf@HIDDEN>
 <CAOqdjBfch1Tg57EKJvUXJHt+gdJ=6kZzg-2dPAisyy9je=e9SQ@HIDDEN>
 <83v9a5yfc0.fsf@HIDDEN>
In-Reply-To: <83v9a5yfc0.fsf@HIDDEN>
From: Pip Cet <pipcet@HIDDEN>
Date: Fri, 5 Mar 2021 13:16:14 +0000
Message-ID: <CAOqdjBdPP6ogXOTGB0DDP=yZ6CZ9vmzaS4pwfrRmZirsh+M1qQ@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Eli Zaretskii <eliz@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Daniel Colascione <dancol@HIDDEN>,
 eggert@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Fri, Mar 5, 2021 at 12:07 PM Eli Zaretskii <eliz@HIDDEN> wrote:
> > From: Pip Cet <pipcet@HIDDEN>
> > Date: Fri, 5 Mar 2021 09:54:32 +0000
> > Cc: Daniel Colascione <dancol@HIDDEN>, eggert@HIDDEN, 46881 <at> debbugs.gnu.org
> >
> > My patch:
> >
> > real    0m1.988s
> > user    0m1.916s
> > sys    0m0.073s
> >
> > fwrite-based patch:
> >
> > real    0m3.576s
> > user    0m2.571s
> > sys    0m1.006s
>
> 30% slowdown and 1.5 sec absolute time difference doesn't sound bad
> enough to me to

It's a 30% slowdown of the entire dump process, including the
CPU-intensive part which loads Emacs. I think you get a better idea of
the performance difference from the "sys" numbers above.

And the absolute time difference is more than that, because Emacs is
dumped twice during each build; the first dump file is about 2.5 times
the size of the ultimate dump file, so my guess (as I said before,
unfortunately Intel decided to make this system not have a predictable
CPU clock, so I can't really run good benchmarks) is we're talking
about 4.5 seconds here.

> justify a homemade solution.

"Create a buffer in memory and do all the IO at once" is such an old
solution that even the GNU Coding Standards explicitly recommend it
(albeit for input files):

You could keep the entire input file in memory and scan it there
instead of using stdio

>I say let's go with stdio.

Maybe setbuffer(3) could help us here? I could run some benchmarks for
that if the idea isn't out of the question.

> > > > Also, we're not currently using fseek-and-write anywhere in Emacs.
> > >
> > > I don't see why this would be important.
> >
> > Because the stream returned by emacs_fopen might not be generally seekable?
>
> I don't see how that could happen.

It has, to me, but I'm willing to accept I did some inadvisable things first.

> > By preparing the data in memory and writing it in one go, which
> > doesn't require any of the major complications of implementing
> > buffered streams.
>
> There are no complications I can see, not in our sources.  (And you
> don't actually write it in one go anyway, see emacs_full_write.)

Er, precisely. I was the one saying there are no complications, so we
shouldn't let the idea of "implementing our own buffered streams"
scare us, because that is a complicated project but it's also not what
we are doing.

> So let's go with the stdio solution, please.

Should I add a sync after every seek to make absolutely certain,
rather than merely likely, this will destroy someone's flash chip one
day?

Pip




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 12:49:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 07:49:59 2021
Received: from localhost ([127.0.0.1]:33153 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI9u2-0008Bl-Tn
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 07:49:59 -0500
Received: from quimby.gnus.org ([95.216.78.240]:34262)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <larsi@HIDDEN>) id 1lI9u0-0008BW-Q9
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 07:49:57 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org;
 s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:
 References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:
 Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender:
 Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:
 List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=gjm4us0t1R5jS+aoIrtqLbN4JhaTd6Exl9H4KfBtOeA=; b=IlorhVDZ7Mz2WTNUybESlvSpgM
 OZusR4NN6B2nLAv5WHth2zbobOmDklqRzB7F3gB+k7qFTVkelfgMGfCNuaWXiDg5f0dOWDBHZBTym
 Kx4zm1r1/ZRIEAqZXIBKDNRDMaNYZQHthu8zQ1zFWlmrWQ5GMhC0WQhk9hOV9n0iJNdY=;
Received: from cm-84.212.220.105.getinternet.no ([84.212.220.105] helo=xo)
 by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.92) (envelope-from <larsi@HIDDEN>)
 id 1lI9tn-0006O0-TM; Fri, 05 Mar 2021 13:49:48 +0100
From: Lars Ingebrigtsen <larsi@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN>
 <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 <83a6riysnv.fsf@HIDDEN>
 <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
 <83zgzixcfy.fsf@HIDDEN>
 <CAOqdjBfch1Tg57EKJvUXJHt+gdJ=6kZzg-2dPAisyy9je=e9SQ@HIDDEN>
 <83v9a5yfc0.fsf@HIDDEN>
Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAABGdBTUEAALGPC/xhBQAAACBj
 SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAG1BMVEU+RoM3R4Uscq4n
 hsItS41TWpOwr8o1iL7///9ogBk0AAAAAWJLR0QIht6VegAAAAd0SU1FB+UDBQwvMinyOvoAAAGs
 SURBVDjLdZNLcoMwDIbJDWJad49c2nWtCex7A+iYPcngrBtm8PUr+QFOQkVMHH2W5N+xikI824FG
 sQfYikMJoJAGBNNK00sciwO8VgDvJXxDJQDKqvScQPVcgdLQgL0KIPIIabp9YITMAUgjDZuggI6/
 AxBgognZmYknlFCJwtTR3wljhlGGKQNng9E6Y6eYlXQMGRjsFLNSDbusgFLR43MxaLaIt8EOIabc
 wEQChxX4VBd7ZUDirJkGS+6+zoCVspNm9AA8aC6uZeAPy1jeWAAfF2xjcSKSgTqFiBk9CEc7rGC5
 IPpdxTM3D0AkQARjKpy9jPX/6vHU+4gZFzvmQNes3HrQZeBH16lGY/Ob0OsoEPFj+ge0d8Ak0GI7
 5qBOOtoZfzN/qTOQ39MXDAKXc3vDrwyoFSCizgAmQLtCteUq6ecqMA9R92AN4QCsVRTIVm1+ijim
 CDLuoFJl4KxQaepP1BCXRHDzLas2N6Vl0MzeC/TB1jlHY44ANT6YB8uVlrlb7P8KPmnaMXDpTse2
 5A7rWaDzpoC2xbVoD5omfBlSd6zd7ht3H3RH8QdZYeMm+rfuuwAAACV0RVh0ZGF0ZTpjcmVhdGUA
 MjAyMS0wMy0wNVQxMjo0Nzo1MCswMDowMCUNay8AAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDMt
 MDVUMTI6NDc6NTArMDA6MDBUUNOTAAAAAElFTkSuQmCC
X-Now-Playing: Brigid Mae Power's _Head Above the Water_: "The Blacksmith"
Date: Fri, 05 Mar 2021 13:49:42 +0100
In-Reply-To: <83v9a5yfc0.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 05 Mar
 2021 14:06:55 +0200")
Message-ID: <877dmlsr2x.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 @@CONTACT_ADDRESS@@ for details.
 Content preview: Eli Zaretskii <eliz@HIDDEN> writes: > 30% slowdown and 1.5
 sec absolute time difference doesn't sound bad > enough to me to justify
 a homemade solution. I say let's go with > stdio. Seems significant to me
 -- we're building Emacs a lot, and this bit can't be parallelised. And the
 savings in electricity alone should make us go for the most efficient
 solution.
 Content analysis details:   (-2.9 points, 5.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -1.0 ALL_TRUSTED            Passed through trusted hosts only via SMTP
 -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
 [score: 0.0000]
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, eggert@HIDDEN, Pip Cet <pipcet@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Eli Zaretskii <eliz@HIDDEN> writes:

> 30% slowdown and 1.5 sec absolute time difference doesn't sound bad
> enough to me to justify a homemade solution.  I say let's go with
> stdio.

Seems significant to me -- we're building Emacs a lot, and this bit
can't be parallelised.  And the savings in electricity alone should make
us go for the most efficient solution.

There doesn't seem to be any significant drawbacks to doing it the
efficient way, either.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 12:07:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 07:07:29 2021
Received: from localhost ([127.0.0.1]:33068 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI9Ev-00053A-3t
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 07:07:29 -0500
Received: from eggs.gnu.org ([209.51.188.92]:50112)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1lI9Et-00052z-Ad
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 07:07:27 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:35305)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1lI9En-0008Le-8j; Fri, 05 Mar 2021 07:07:21 -0500
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1570
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1lI9Ed-0006Mh-IQ; Fri, 05 Mar 2021 07:07:19 -0500
Date: Fri, 05 Mar 2021 14:06:55 +0200
Message-Id: <83v9a5yfc0.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Pip Cet <pipcet@HIDDEN>
In-Reply-To: <CAOqdjBfch1Tg57EKJvUXJHt+gdJ=6kZzg-2dPAisyy9je=e9SQ@HIDDEN>
 (message from Pip Cet on Fri, 5 Mar 2021 09:54:32 +0000)
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 <83a6riysnv.fsf@HIDDEN>
 <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
 <83zgzixcfy.fsf@HIDDEN>
 <CAOqdjBfch1Tg57EKJvUXJHt+gdJ=6kZzg-2dPAisyy9je=e9SQ@HIDDEN>
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, dancol@HIDDEN, eggert@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

> From: Pip Cet <pipcet@HIDDEN>
> Date: Fri, 5 Mar 2021 09:54:32 +0000
> Cc: Daniel Colascione <dancol@HIDDEN>, eggert@HIDDEN, 46881 <at> debbugs.gnu.org
> 
> My patch:
> 
> real    0m1.988s
> user    0m1.916s
> sys    0m0.073s
> 
> fwrite-based patch:
> 
> real    0m3.576s
> user    0m2.571s
> sys    0m1.006s

30% slowdown and 1.5 sec absolute time difference doesn't sound bad
enough to me to justify a homemade solution.  I say let's go with
stdio.

> > > Also, we're not currently using fseek-and-write anywhere in Emacs.
> >
> > I don't see why this would be important.
> 
> Because the stream returned by emacs_fopen might not be generally seekable?

I don't see how that could happen.

> > Since we open the file in
> > binary mode, fseek should work correctly even on non-Posix systems.
> 
> I guess I should have used emacs_fopen :-)

Yes, of course.  Especially as with fopen there are problems with
non-ASCII file names on MS-Windows.

> By preparing the data in memory and writing it in one go, which
> doesn't require any of the major complications of implementing
> buffered streams.

There are no complications I can see, not in our sources.  (And you
don't actually write it in one go anyway, see emacs_full_write.)

So let's go with the stdio solution, please.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 10:23:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 05:23:56 2021
Received: from localhost ([127.0.0.1]:32938 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI7ci-0002Hi-2k
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 05:23:56 -0500
Received: from mx.sdf.org ([205.166.94.24]:58264)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <akrl@HIDDEN>) id 1lI7cg-0002HZ-BT
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 05:23:55 -0500
Received: from mab (ma.sdf.org [205.166.94.33])
 by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id 125ANjC3003950
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits) verified NO);
 Fri, 5 Mar 2021 10:23:45 GMT
From: Andrea Corallo <akrl@HIDDEN>
To: Pip Cet <pipcet@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN>
 <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 <83a6riysnv.fsf@HIDDEN>
 <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
 <83zgzixcfy.fsf@HIDDEN>
 <CAOqdjBfch1Tg57EKJvUXJHt+gdJ=6kZzg-2dPAisyy9je=e9SQ@HIDDEN>
Date: Fri, 05 Mar 2021 10:23:45 +0000
In-Reply-To: <CAOqdjBfch1Tg57EKJvUXJHt+gdJ=6kZzg-2dPAisyy9je=e9SQ@HIDDEN>
 (Pip Cet's message of "Fri, 5 Mar 2021 09:54:32 +0000")
Message-ID: <xjfzgzh6gr2.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>, eggert@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Pip Cet <pipcet@HIDDEN> writes:

[...]

> By preparing the data in memory and writing it in one go, which
> doesn't require any of the major complications of implementing
> buffered streams.

Preparing data in memory might also be seen as a small step in the
direction of having pdumper as a generic de/serializer.  IMO would be
helpful for a number of tasks (native compiler included).

  Andrea




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 09:55:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 04:55:17 2021
Received: from localhost ([127.0.0.1]:32905 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI7Az-0001Zv-1N
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 04:55:17 -0500
Received: from mail-oo1-f53.google.com ([209.85.161.53]:43607)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lI7Aw-0001Zh-OS
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 04:55:15 -0500
Received: by mail-oo1-f53.google.com with SMTP id x19so280470ooj.10
 for <46881 <at> debbugs.gnu.org>; Fri, 05 Mar 2021 01:55:14 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=/pUVpAbPc7Snr4jWcgI+Fr34P94CvOSjnbZxL7DVfXU=;
 b=k4Dnu2pWWSsfRJn16TNN3tfWhtXAxXhvCo+knS4NNTUuSu43WRATaf7Kgh0W9k0ydH
 24CIXmVIK794htiv2bPmOsvX7ZbuRY7y8kPWgFNg+/bE8XbdTIe2FIcGBDbupOWailTz
 c+CXt37E6BHifUh62oApnwbrZtnlqDqPDaXf9EAY3CXU0LzRrtjP9c8pfniYzIo2gm4Z
 kRC5Kux5sRMHksbPR1ZcQA0VwS8c3UagF/OhFFtTwfDL5Hyn1+4FEGWfsG95Exio6beC
 ex4hErZWpAeleZ3YYMrNr1oshOyM2LJioTEqaVo64Nkt0OkafA3LKZGmIP2WgoV+zsuN
 CdzQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=/pUVpAbPc7Snr4jWcgI+Fr34P94CvOSjnbZxL7DVfXU=;
 b=jnD5Io+GgbdOLO8sUnhekX9cOoCKMZ7c+NIWq2i+5C+8baZZ9oH1wh5QX5A9yRTiw6
 chkdEh+Ofbcj/EX8r5lm1xxFi9Huq5t+OBhqHvp0McZjmlQwT90zEF4ZIFt0V0HhPefU
 dtJr9Qwec1xRVbyLXPzJsP4dug8mQhG2pFz+DDOAWY7CNYIrYAceWovWyYRta0FMNzP/
 ypt7wjObOljvqM9LF8fw6LEqJ8QiA1L+ZiFHjcrXvTMN+dbV3DFQC+1ETfTkL4hSXNvX
 AYooUwhMsBXikUNB7KgXYRSJS6GmL3jpVaNIKSixV5oQPWXkU/p+vuzozQb3SBUY5D/+
 v5sw==
X-Gm-Message-State: AOAM532dvDTA1CKuVp70nSJ3ueI1eJDIymzUyNY9vN4c0coRMdkAz5fk
 perIGSQVFHZy0XgygY7gQGlvelnwc4L/fL3G1Sg=
X-Google-Smtp-Source: ABdhPJzwUz0HRGSnxn97A5/I2FLppqHNkracf205PzStwH++5E9gyq/88OUef5GOlI5zuICJfNzWCwag/vRDs9B+ZJk=
X-Received: by 2002:a4a:2a0a:: with SMTP id k10mr7039762oof.88.1614938109021; 
 Fri, 05 Mar 2021 01:55:09 -0800 (PST)
MIME-Version: 1.0
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 <83a6riysnv.fsf@HIDDEN>
 <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
 <83zgzixcfy.fsf@HIDDEN>
In-Reply-To: <83zgzixcfy.fsf@HIDDEN>
From: Pip Cet <pipcet@HIDDEN>
Date: Fri, 5 Mar 2021 09:54:32 +0000
Message-ID: <CAOqdjBfch1Tg57EKJvUXJHt+gdJ=6kZzg-2dPAisyy9je=e9SQ@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Eli Zaretskii <eliz@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Daniel Colascione <dancol@HIDDEN>,
 eggert@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Fri, Mar 5, 2021 at 7:55 AM Eli Zaretskii <eliz@HIDDEN> wrote:
> > From: Pip Cet <pipcet@HIDDEN>
> > Date: Fri, 5 Mar 2021 07:38:27 +0000
> > Cc: Daniel Colascione <dancol@HIDDEN>, eggert@HIDDEN, 46881 <at> debbugs.gnu.org
> >
> > > I'm not sure I understand: what's wrong with fseek?
> >
> > Nothing, assuming you're fine with the current performance. Many libcs
> > aren't going to be smart enough to avoid I/O when you fseek through a
> > "large" file and write a word here and there, and my suspicion is that
> > would include glibc.
>
> Could we benchmark the two implementations instead of acting on
> suspicions?

Sure.

My patch:

real    0m1.988s
user    0m1.916s
sys    0m0.073s

fwrite-based patch:

real    0m3.576s
user    0m2.571s
sys    0m1.006s

This is as I expected: glibc just isn't doing a very good job for this
buffered stream.

> In general, I'd prefer not to reinvent the wheel, and trust modern
> libc's that they are efficient enough in handling buffered streams,
> unless we have hard evidence to the contrary.

We do, now.

> If nothing else, it
> would prevent people asking, like Daniel did, why didn't we use stdio
> in the first place.

I think it's a very good question (in fact, the brach I'm working on
is called pdumper-fwrite because I decided only after creating it that
all the seeking would hurt performance too much). I'll try including a
comment explaining why.

> > Also, we're not currently using fseek-and-write anywhere in Emacs.
>
> I don't see why this would be important.

Because the stream returned by emacs_fopen might not be generally seekable?

> Since we open the file in
> binary mode, fseek should work correctly even on non-Posix systems.

I guess I should have used emacs_fopen :-)

> > We're talking about a file which Emacs is going to have to keep in
> > memory anyway, when reading the dump. The only case in which there
> > might be a problem is if the build machine has significantly less
> > available memory than the machine we intend to run on, and I just
> > don't think that's going to happen.
>
> You are thinking about memory consumption, while I am thinking how to
> avoid implementing our own private buffered streams.

By preparing the data in memory and writing it in one go, which
doesn't require any of the major complications of implementing
buffered streams.

Pip




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 09:42:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 04:42:31 2021
Received: from localhost ([127.0.0.1]:32881 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI6yd-0001Eq-69
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 04:42:31 -0500
Received: from mail-oi1-f174.google.com ([209.85.167.174]:39773)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lI6yb-0001Ee-Or
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 04:42:30 -0500
Received: by mail-oi1-f174.google.com with SMTP id z126so1838416oiz.6
 for <46881 <at> debbugs.gnu.org>; Fri, 05 Mar 2021 01:42:29 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc:content-transfer-encoding;
 bh=D3uqr5W55Z7Yrg+ynSUew8pX3D7ASLuo5MTdlD9SVPM=;
 b=RSlcyhA3KSavK/a4uEbd203l9l7vUkhNKSgsrc6etmpQQ+L7lHmM9S+YFF+aox7qAR
 35V0d2XNYHr6gwUnZFITfvresFoRJLPuywZ9yE9vBL7Dx5h6EdPorQbNivJJUiwXqVUJ
 7UQCAj6COYZadZ0cSUnHxwJTKYcgMabF1kKheubEtwhPmgWSgsJk8XwMRNj1+NyUwe07
 JXfmGPE0a8vsUlUDgXd1vUSpktafMObcOluVAJKOr0Rg2MXzhK0QXjTqIxoOAzLbW2PE
 Zm75rYpSCydTu/IfReVqiJ+EbvkQVZGoh057s9uvakjYuF9oCcrUB+gRTkbPxLTc0HUi
 JmCA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc:content-transfer-encoding;
 bh=D3uqr5W55Z7Yrg+ynSUew8pX3D7ASLuo5MTdlD9SVPM=;
 b=LWSgVTU5uDX/WHiyd/XbZytmYMaDcHiZHG+kKaKY+axeAFnZT8YGnRks1xXbb4a/Gv
 glxaE6kvhgbmQaNuDr4WGabXNq9fV8U4sG47/tdrITCFSOxLgAsuSIUZQ9PHSYX6x7cU
 yo0aNXaIUZjs/3lQxZgYIxmxILGd4qWtC27EUdReoVG1dral8zpC31qj6u97rVxO5/vA
 1kKMgIUoO9Cw80IAmPfz9HCVGxiIGmpeUvXwn0gn1gP8QdN5QGLxe0GKUspWiiZ5+bMk
 PX3k7DE+AI/m8/oRhR0NzDG48wYFMkBqmtviB9ZV1z5QdC3TWqt7YQx1z+sE+pcG8/jy
 eDhA==
X-Gm-Message-State: AOAM530urZZFlfUaK1g899kqexxg7OIO2lAGY2rXheX9O8VLLV0OhZJs
 QHEhPNeB56a0zb33ffhhEWJwcDzbxQYSE5frFvY5SFIIl19qww==
X-Google-Smtp-Source: ABdhPJxc1FMLpBtHwhRXFjg5qKRPjNLZRZFJ0HlksfD53dIJjuwYWXgkPMOuzXEIpJHywifToyTZxBp4zZLFqLn3OVY=
X-Received: by 2002:aca:d905:: with SMTP id q5mr6185981oig.30.1614937343876;
 Fri, 05 Mar 2021 01:42:23 -0800 (PST)
MIME-Version: 1.0
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 <83a6riysnv.fsf@HIDDEN>
 <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
 <87y2f254f5.fsf@HIDDEN>
In-Reply-To: <87y2f254f5.fsf@HIDDEN>
From: Pip Cet <pipcet@HIDDEN>
Date: Fri, 5 Mar 2021 09:41:47 +0000
Message-ID: <CAOqdjBfXp36b++KuxFhh+roYRggy+7nvM0sQbS4jx0jcvabMUw@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Andreas Schwab <schwab@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>, eggert@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Fri, Mar 5, 2021 at 9:35 AM Andreas Schwab <schwab@HIDDEN> wrote=
:
> On M=C3=A4r 05 2021, Pip Cet wrote:
>
> > We're talking about a file which Emacs is going to have to keep in
> > memory anyway, when reading the dump.
>
> While reading the dump, you only have the data once, and you don't have
> to realloc the whole data.

Correct. I think a build memory usage of 28 MB for a 10 MB dump file
is something we can live with...

Pip




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 09:35:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 04:35:32 2021
Received: from localhost ([127.0.0.1]:32871 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI6rs-000144-ML
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 04:35:32 -0500
Received: from mail-out.m-online.net ([212.18.0.9]:52540)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <whitebox@HIDDEN>) id 1lI6rq-00013u-7X
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 04:35:31 -0500
Received: from frontend01.mail.m-online.net (unknown [192.168.8.182])
 by mail-out.m-online.net (Postfix) with ESMTP id 4DsMzN49hKz1qtdy;
 Fri,  5 Mar 2021 10:35:27 +0100 (CET)
Received: from localhost (dynscan1.mnet-online.de [192.168.6.70])
 by mail.m-online.net (Postfix) with ESMTP id 4DsMzM5MPvz1qqkq;
 Fri,  5 Mar 2021 10:35:27 +0100 (CET)
X-Virus-Scanned: amavisd-new at mnet-online.de
Received: from mail.mnet-online.de ([192.168.8.182])
 by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new,
 port 10024)
 with ESMTP id IM447b6YijDw; Fri,  5 Mar 2021 10:35:27 +0100 (CET)
X-Auth-Info: fhSJwioBJAvJNsQ/GmLU0kYqA/+5GVM3Jh8ffWYZkuSrPplrSvCCijjRsl11t7pP
Received: from igel.home (ppp-46-244-183-40.dynamic.mnet-online.de
 [46.244.183.40])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by mail.mnet-online.de (Postfix) with ESMTPSA;
 Fri,  5 Mar 2021 10:35:27 +0100 (CET)
Received: by igel.home (Postfix, from userid 1000)
 id 76A0F2C362B; Fri,  5 Mar 2021 10:35:26 +0100 (CET)
From: Andreas Schwab <schwab@HIDDEN>
To: Pip Cet <pipcet@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN>
 <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 <83a6riysnv.fsf@HIDDEN>
 <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
X-Yow: Someone is DROOLING on my collar!!
Date: Fri, 05 Mar 2021 10:35:26 +0100
In-Reply-To: <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
 (Pip Cet's message of "Fri, 5 Mar 2021 07:38:27 +0000")
Message-ID: <87y2f254f5.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.4 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>, eggert@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.4 (-)

On Mär 05 2021, Pip Cet wrote:

> We're talking about a file which Emacs is going to have to keep in
> memory anyway, when reading the dump.

While reading the dump, you only have the data once, and you don't have
to realloc the whole data.

Andreas.

-- 
Andreas Schwab, schwab@HIDDEN
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 07:55:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 02:55:07 2021
Received: from localhost ([127.0.0.1]:60916 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI5Ih-00072J-6G
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 02:55:07 -0500
Received: from eggs.gnu.org ([209.51.188.92]:52276)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1lI5Ie-00071k-UJ
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 02:55:05 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:53251)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1lI5IX-0000OO-QA; Fri, 05 Mar 2021 02:54:57 -0500
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1971
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1lI5IX-0005oi-6t; Fri, 05 Mar 2021 02:54:57 -0500
Date: Fri, 05 Mar 2021 09:54:41 +0200
Message-Id: <83zgzixcfy.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Pip Cet <pipcet@HIDDEN>
In-Reply-To: <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
 (message from Pip Cet on Fri, 5 Mar 2021 07:38:27 +0000)
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 <83a6riysnv.fsf@HIDDEN>
 <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, dancol@HIDDEN, eggert@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

> From: Pip Cet <pipcet@HIDDEN>
> Date: Fri, 5 Mar 2021 07:38:27 +0000
> Cc: Daniel Colascione <dancol@HIDDEN>, eggert@HIDDEN, 46881 <at> debbugs.gnu.org
> 
> > I'm not sure I understand: what's wrong with fseek?
> 
> Nothing, assuming you're fine with the current performance. Many libcs
> aren't going to be smart enough to avoid I/O when you fseek through a
> "large" file and write a word here and there, and my suspicion is that
> would include glibc.

Could we benchmark the two implementations instead of acting on
suspicions?

In general, I'd prefer not to reinvent the wheel, and trust modern
libc's that they are efficient enough in handling buffered streams,
unless we have hard evidence to the contrary.  If nothing else, it
would prevent people asking, like Daniel did, why didn't we use stdio
in the first place.

> Also, we're not currently using fseek-and-write anywhere in Emacs.

I don't see why this would be important.  Since we open the file in
binary mode, fseek should work correctly even on non-Posix systems.
Am I missing something?

> We're talking about a file which Emacs is going to have to keep in
> memory anyway, when reading the dump. The only case in which there
> might be a problem is if the build machine has significantly less
> available memory than the machine we intend to run on, and I just
> don't think that's going to happen.

You are thinking about memory consumption, while I am thinking how to
avoid implementing our own private buffered streams.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 07:39:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 02:39:12 2021
Received: from localhost ([127.0.0.1]:60900 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI53H-0006fF-L9
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 02:39:12 -0500
Received: from mail-oi1-f180.google.com ([209.85.167.180]:36144)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lI53F-0006f2-Fq
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 02:39:10 -0500
Received: by mail-oi1-f180.google.com with SMTP id j1so1546666oiw.3
 for <46881 <at> debbugs.gnu.org>; Thu, 04 Mar 2021 23:39:09 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=VZ0z9pEvfnZzbQ0sPXcm6gvsEnaOshuFZVeRShqCOBs=;
 b=psOPkvKQ24fOwb2ymV0kqfJQqZ33IJ1QiJgCfFYlaPEh5j/5nhAcR/ZLf6qHZcAZDs
 tiNTOBv6yub40h+HsironL7uzCbpgRSjmQyrvrmJW8iL6J4baVPDKZfmHuBlFXlxgPRW
 WvpVj2SZ108TxyPYV3A9WAinFu5xf3gvLn53MV8ylmEwnUy4Baim1oj35KTsKGlsB9hb
 ThpWb7XoZE5VoaySaorsisE7dLFfuYTEvScZCfYA4rH+nyZY2HVJb4OdArAuYrklC0i3
 zM6+trT4AqX3MoGvGH8rBYuP5qRWTs2TYT2ODv2/i/OPR+IFySndmLKXc03uMjFSy4Dk
 r6yQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=VZ0z9pEvfnZzbQ0sPXcm6gvsEnaOshuFZVeRShqCOBs=;
 b=by9gFRev5bfXVYv/1rjjfLRPZOx4m98Zi63ESMIUhcWKkVvGddF2hMzJ9oNPoBB2HR
 LrquK7av2wKj/V4KddZXN2v5enSZ3olKKkLncn6i8OKacyRiPyrSc+QijyUrniX/B72T
 cv+9gM17EE07743b+QbFAgV5fFCgAIrQ6gcPvGgCXwvx3uzk/2XccXb4yWr7SRlGCzwa
 Mvq9Mz/DWZEZzp56++EZ5s8DSATe2k4zh+Et8w1ReNRrqDc+aOKVG5nYDEar9S5em7uY
 O319dDNAKjw2xuTsAllIcUzubYlUlP7thbMgSspoJi9ppyAo980aafjeXisadnQOsKj+
 GHpw==
X-Gm-Message-State: AOAM5330VczhMc2KA4jOaDymOwIbHLuMhK2xRcV70/AXFBoR3vnd8RX2
 cUo5Rj/9hNeayY3WBkBY++u9/z6iCbTqUqZcV5w=
X-Google-Smtp-Source: ABdhPJwGm0E5IrOyoWiHIdVuB2ubCbv9nAM1U73re/fe0kHvackB7JamNXOYoBqltDF2U1wbUJRTwG+1L9NXcxYDN84=
X-Received: by 2002:aca:4c0f:: with SMTP id z15mr2190051oia.44.1614929943855; 
 Thu, 04 Mar 2021 23:39:03 -0800 (PST)
MIME-Version: 1.0
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 <83a6riysnv.fsf@HIDDEN>
In-Reply-To: <83a6riysnv.fsf@HIDDEN>
From: Pip Cet <pipcet@HIDDEN>
Date: Fri, 5 Mar 2021 07:38:27 +0000
Message-ID: <CAOqdjBdztYrqL6iU5+N72ATk7-7bzvnFOpqkFwc3u7mZRQgSqQ@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Eli Zaretskii <eliz@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Daniel Colascione <dancol@HIDDEN>,
 eggert@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Fri, Mar 5, 2021 at 7:19 AM Eli Zaretskii <eliz@HIDDEN> wrote:
> > From: Pip Cet <pipcet@HIDDEN>
> > Date: Fri, 5 Mar 2021 02:30:13 +0000
> > Cc: Eli Zaretskii <eliz@HIDDEN>, Paul Eggert <eggert@HIDDEN>, 46881 <at> debbugs.gnu.org
> >
> > > It looks fine, but wouldn't dumping to a FILE* (with internal buffering)
> > > do the same basic thing in a simpler way?
> >
> > I initially set out to do that, but decided against it. We don't just
> > write sequentially (when FILE I/O helps, a little), we also have the
> > seek-and-fixup phase, and it didn't seem any simpler at that point..
>
> I'm not sure I understand: what's wrong with fseek?

Nothing, assuming you're fine with the current performance. Many libcs
aren't going to be smart enough to avoid I/O when you fseek through a
"large" file and write a word here and there, and my suspicion is that
would include glibc.

Also, we're not currently using fseek-and-write anywhere in Emacs.

We're talking about a file which Emacs is going to have to keep in
memory anyway, when reading the dump. The only case in which there
might be a problem is if the build machine has significantly less
available memory than the machine we intend to run on, and I just
don't think that's going to happen.

Pip




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 07:19:26 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 05 02:19:26 2021
Received: from localhost ([127.0.0.1]:60870 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI4kA-00068n-2A
	for submit <at> debbugs.gnu.org; Fri, 05 Mar 2021 02:19:26 -0500
Received: from eggs.gnu.org ([209.51.188.92]:45688)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1lI4k7-00068X-SO
 for 46881 <at> debbugs.gnu.org; Fri, 05 Mar 2021 02:19:25 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:52967)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1lI4k2-0003sk-6I; Fri, 05 Mar 2021 02:19:18 -0500
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3748
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1lI4k0-0007wX-1s; Fri, 05 Mar 2021 02:19:18 -0500
Date: Fri, 05 Mar 2021 09:19:00 +0200
Message-Id: <83a6riysnv.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Pip Cet <pipcet@HIDDEN>
In-Reply-To: <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
 (message from Pip Cet on Fri, 5 Mar 2021 02:30:13 +0000)
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
 <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, dancol@HIDDEN, eggert@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

> From: Pip Cet <pipcet@HIDDEN>
> Date: Fri, 5 Mar 2021 02:30:13 +0000
> Cc: Eli Zaretskii <eliz@HIDDEN>, Paul Eggert <eggert@HIDDEN>, 46881 <at> debbugs.gnu.org
> 
> > It looks fine, but wouldn't dumping to a FILE* (with internal buffering)
> > do the same basic thing in a simpler way?
> 
> I initially set out to do that, but decided against it. We don't just
> write sequentially (when FILE I/O helps, a little), we also have the
> seek-and-fixup phase, and it didn't seem any simpler at that point..

I'm not sure I understand: what's wrong with fseek?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 5 Mar 2021 02:30:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 04 21:30:57 2021
Received: from localhost ([127.0.0.1]:60702 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lI0Ez-0007Hi-8B
	for submit <at> debbugs.gnu.org; Thu, 04 Mar 2021 21:30:57 -0500
Received: from mail-ot1-f54.google.com ([209.85.210.54]:43412)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lI0Ex-0007HV-DV
 for 46881 <at> debbugs.gnu.org; Thu, 04 Mar 2021 21:30:56 -0500
Received: by mail-ot1-f54.google.com with SMTP id v12so363540ott.10
 for <46881 <at> debbugs.gnu.org>; Thu, 04 Mar 2021 18:30:55 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=Yz+5IQwz1PHNAgvclmhCEIHgyfGRA67qmVd0f2CpqfY=;
 b=aQGhbbaYcIu6PXyoVeiOaBgqs0O9MaZcBo7eRMscwwfCxVMYj+Q0uZxFSu9sD5nRkR
 l+tfV3OqPlX+uLkn0ANbHbESCePX4Y0Gj9UStdPAnOx+lZvHquco3e4O5HfgDYvl4JHh
 7eujLJn9FrJBBU9iYL9KOG++UB27bxaGKzMr/OMyu0PvY+tE8/lcFZr0NrUnO/lI0Y4T
 omd/5hs4jOZbQpDU5kFST5Plx/a2f3RFYeZes+MBT8Em7EAtkP3Wh+6JmWcVPvezOjMh
 kF/97XiZmpSi7ZGHMCYl0P8Na5oJtHYN4NZKmGs7MYPC7DusLL05lyFEKpugir9Hu14c
 0XzQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=Yz+5IQwz1PHNAgvclmhCEIHgyfGRA67qmVd0f2CpqfY=;
 b=MGuVYQ/vxXa9gzx0pGl38dL0oZSoRFMOEIxgkVza26uDzAPiZwjQZA26IsB16FF51L
 IpqL3ht2+42PnSPMiOiIc8BsWtno8YqiyxhjSMWySkkAfF8/qTFaK/w/pRqh6XsVn1WV
 CsCcIJFBBw2kbryP1bi8wk3+mEpl+XqFwlxqjypHHjTM8kP+oD+4BCkSfdhRTvQo496C
 dfBVFNMNeq62csS8wFZcx0jZzAWcdMu2CWsVPDQnzMfand3hreyPkHuPmreYtn0/UY3x
 6aVmF5nxrLs56OHTaqYDeKG3ZHONQtfalF27mUwhXk8ltj7W4jLqPTWm9o5Ax9b20BQn
 VyLQ==
X-Gm-Message-State: AOAM533LU+s0syYr9oAO0FAkAk4MuoysLsVbMgQji7NVm7pMQhv1W91Q
 NYONz46rxDs1ybcC07k5bQfdBj8RmM9lxgyoBt4=
X-Google-Smtp-Source: ABdhPJwIQH7cO+h2H+YYei8PWyPiS1L921F16Fu5I/aqTG9jzxcPhYtm2585s40h7ZZryU0sTAknPMQfYfgTorNTIw0=
X-Received: by 2002:a05:6830:1e51:: with SMTP id
 e17mr6132947otj.292.1614911449852; 
 Thu, 04 Mar 2021 18:30:49 -0800 (PST)
MIME-Version: 1.0
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN> <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
In-Reply-To: <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
From: Pip Cet <pipcet@HIDDEN>
Date: Fri, 5 Mar 2021 02:30:13 +0000
Message-ID: <CAOqdjBe8qDmYO8K93oA9k7Zf12cORWDeAAoFAUzACDc4K+SmyA@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Daniel Colascione <dancol@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>,
 Paul Eggert <eggert@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Thu, Mar 4, 2021 at 10:26 PM Daniel Colascione <dancol@HIDDEN> wrote:
> On 3/3/21 12:51 AM, Eli Zaretskii wrote:
> > Daniel, Paul: any comments?  In particular, is it safe to allocate
> > large amounts of memory off the heap while dumping?  A couple of
> > places in pdumper.c says some parts of code should call malloc.
>
> It looks fine, but wouldn't dumping to a FILE* (with internal buffering)
> do the same basic thing in a simpler way?

I initially set out to do that, but decided against it. We don't just
write sequentially (when FILE I/O helps, a little), we also have the
seek-and-fixup phase, and it didn't seem any simpler at that point..

> There aren't any particular
> constraints on the environment _during_ the dump: we even make new lisp
> objects. It's when loading the dump, early in initialization, that you
> have to be careful.

Thanks!

Pip




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 4 Mar 2021 22:26:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 04 17:26:41 2021
Received: from localhost ([127.0.0.1]:60555 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHwQa-0001WI-TA
	for submit <at> debbugs.gnu.org; Thu, 04 Mar 2021 17:26:41 -0500
Received: from dancol.org ([96.126.100.184]:35692)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dancol@HIDDEN>) id 1lHwQW-0001W8-3X
 for 46881 <at> debbugs.gnu.org; Thu, 04 Mar 2021 17:26:38 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; 
 s=x;
 h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:
 Message-ID:From:References:Cc:To:Subject:Sender:Reply-To:Content-ID:
 Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
 :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:
 List-Post:List-Owner:List-Archive;
 bh=AH3jYq7hr+YyCbABU+2+sUNtbROHnDTbfObLdyzhWms=; b=UB24kcXCIPvrgAOb/19/PtG/IL
 7WwiRVbTX03Y5aL5u4LckDkYGagGLrp5vzbW2Pf5LtDMM8MhDPpCqikgYT+PfIplfynUEuoIdoI74
 QAOw/Sxv214AUCXu0wSDXUutWFNqRvS2lPWAEqD0EQy6xkqsgeqiOEyu1EXzivEsAzcf5OHKfiv7o
 FEl8aJYOz5xM7ZseepNowy7USVozRyXS6qNzoJ18UB+OLfCFzgPIfNRhXcEVGgg5GaquO9McMZZBg
 3NzMwaC+LLDBKbLTiI2gayi0gEBwn5/l09WUTBXEniNAFiXHyCa4LNZkWrJQ7uHCrrMn8QgS6s4zk
 YRlX37RA==;
Received: from [97.104.73.87] (port=46596 helo=[192.168.1.148])
 by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.89) (envelope-from <dancol@HIDDEN>)
 id 1lHwQU-0003cW-5G; Thu, 04 Mar 2021 14:26:34 -0800
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Eli Zaretskii <eliz@HIDDEN>, Pip Cet <pipcet@HIDDEN>,
 Paul Eggert <eggert@HIDDEN>
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN>
From: Daniel Colascione <dancol@HIDDEN>
Message-ID: <90e99fc5-280d-63bb-9bc4-3efe89b9f9e2@HIDDEN>
Date: Thu, 4 Mar 2021 17:26:32 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.7.1
MIME-Version: 1.0
In-Reply-To: <83r1kw6b06.fsf@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On 3/3/21 12:51 AM, Eli Zaretskii wrote:

>> From: Pip Cet <pipcet@HIDDEN>
>> Date: Tue, 2 Mar 2021 20:45:04 +0000
>>
>> On Tue, Mar 2, 2021 at 8:35 PM Pip Cet <pipcet@HIDDEN> wrote:
>>> I've looked into the problem, and it seems easy to solve and worth it
>>> in terms of debuggability and performance.
>> Very rough benchmarks, but this seems to be clearly worth it:
>>
>> Performance:
>> With patch:
>> real    0m3.861s
>> user    0m3.776s
>> sys    0m0.085s
>>
>> Without patch:
>> real    0m7.001s
>> user    0m4.476s
>> sys    0m2.511s
>>
>> Number of syscalls:
>> With patch: 415442
>> Without patch: 2028307
>>
>>> Patch will be attached once this has a bug number.
>> And here's the patch. Testing would be very appreciated.
>>
>> I'm unsure about the precise usage of dump_off vs ptrdiff_t here; I
>> don't think it matters, but suggestions, nitpicks, and comments, on
>> this or any other aspect, would be very appreciated.
>>  From 92ee138852b34ede2f43dd7f93f310fc746bb3bf Mon Sep 17 00:00:00 2001
>> From: Pip Cet <pipcet@HIDDEN>
>> Date: Tue, 2 Mar 2021 20:38:23 +0000
>> Subject: [PATCH] Prepare pdumper dump file in memory, write it in one go
>>   (Bug#46881)
>>
>> * src/pdumper.c (struct dump_context): Add buf, buf_size, max_offset fields.
>> (grow_buffer): New function.
>> (dump_write): Use memcpy, not an actual emacs_write.
>> (dump_seek): Keep track of maximum seen offset.
>> (Fdump_emacs_portable): Write out the file contents when done.
>> ---
>>   src/pdumper.c | 20 ++++++++++++++++++--
>>   1 file changed, 18 insertions(+), 2 deletions(-)
>>
>> diff --git a/src/pdumper.c b/src/pdumper.c
>> index 337742fda4ade..62ddad8ee5e34 100644
>> --- a/src/pdumper.c
>> +++ b/src/pdumper.c
>> @@ -473,6 +473,10 @@ dump_fingerprint (char const *label,
>>   {
>>     /* Header we'll write to the dump file when done.  */
>>     struct dump_header header;
>> +  /* Data that will be written to the dump file.  */
>> +  void *buf;
>> +  ptrdiff_t buf_size;
>> +  ptrdiff_t max_offset;
>>   
>>     Lisp_Object old_purify_flag;
>>     Lisp_Object old_post_gc_hook;
>> @@ -581,6 +585,13 @@ dump_fingerprint (char const *label,
>>   
>>   /* Dump file creation */
>>   
>> +static void dump_grow_buffer (struct dump_context *ctx)
>> +{
>> +  ctx->buf = xrealloc (ctx->buf, ctx->buf_size = (ctx->buf_size ?
>> +						  (ctx->buf_size * 2)
>> +						  : 1024 * 1024));
>> +}
>> +
>>   static dump_off dump_object (struct dump_context *ctx, Lisp_Object object);
>>   static dump_off dump_object_for_offset (struct dump_context *ctx,
>>   					Lisp_Object object);
>> @@ -747,8 +758,9 @@ dump_write (struct dump_context *ctx, const void *buf, dump_off nbyte)
>>     eassert (nbyte == 0 || buf != NULL);
>>     eassert (ctx->obj_offset == 0);
>>     eassert (ctx->flags.dump_object_contents);
>> -  if (emacs_write (ctx->fd, buf, nbyte) < nbyte)
>> -    report_file_error ("Could not write to dump file", ctx->dump_filename);
>> +  while (ctx->offset + nbyte > ctx->buf_size)
>> +    dump_grow_buffer (ctx);
>> +  memcpy ((char *)ctx->buf + ctx->offset, buf, nbyte);
>>     ctx->offset += nbyte;
>>   }
>>   
>> @@ -828,6 +840,8 @@ dump_tailq_pop (struct dump_tailq *tailq)
>>   static void
>>   dump_seek (struct dump_context *ctx, dump_off offset)
>>   {
>> +  if (ctx->max_offset < ctx->offset)
>> +    ctx->max_offset = ctx->offset;
>>     eassert (ctx->obj_offset == 0);
>>     if (lseek (ctx->fd, offset, SEEK_SET) < 0)
>>       report_file_error ("Setting file position",
>> @@ -4159,6 +4173,8 @@ DEFUN ("dump-emacs-portable",
>>     ctx->header.magic[0] = dump_magic[0];
>>     dump_seek (ctx, 0);
>>     dump_write (ctx, &ctx->header, sizeof (ctx->header));
>> +  if (emacs_write (ctx->fd, ctx->buf, ctx->max_offset) < ctx->max_offset)
>> +    report_file_error ("Could not write to dump file", ctx->dump_filename);
>>   
>>     dump_off
>>       header_bytes = header_end - header_start,
>> -- 
>> 2.30.1
> Thanks.
>
> Daniel, Paul: any comments?  In particular, is it safe to allocate
> large amounts of memory off the heap while dumping?  A couple of
> places in pdumper.c says some parts of code should call malloc.

It looks fine, but wouldn't dumping to a FILE* (with internal buffering) 
do the same basic thing in a simpler way? There aren't any particular 
constraints on the environment _during_ the dump: we even make new lisp 
objects. It's when loading the dump, early in initialization, that you 
have to be careful.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 4 Mar 2021 07:26:06 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 04 02:26:05 2021
Received: from localhost ([127.0.0.1]:57554 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHiMu-0001YM-D1
	for submit <at> debbugs.gnu.org; Thu, 04 Mar 2021 02:26:05 -0500
Received: from mail-ot1-f46.google.com ([209.85.210.46]:32946)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lHiMs-0001Y9-Pw
 for 46881 <at> debbugs.gnu.org; Thu, 04 Mar 2021 02:25:55 -0500
Received: by mail-ot1-f46.google.com with SMTP id j8so890800otc.0
 for <46881 <at> debbugs.gnu.org>; Wed, 03 Mar 2021 23:25:54 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to;
 bh=RSykA11zPfpk99eUA+3H9NyfJn/uSfzUhZ5+YCNxFng=;
 b=E78xJTA7uWmlzYQJE1IeHqQNFFY6+Wtp7Ar74ep4OvuHiHfL5q3LhLABY2j3w4b5VB
 Cst0ltm5mRlx2nKOKOqFfHa5pSHOoqdISjylBHceSa5rCrA510MdiII2TbedCrWVF0bR
 wCTngaw+hyyR4PbFAm+pI0VtFYCx9nm6Ldz48Lsycw49d54rlFVvdB1GvurOGAvmNFdS
 ZK34w7T4k6HI/vPGnpEd3BjMJV6rTg6M7e0lm9GV8ZdAvoFX+JLnP2gg66shQttpcyKU
 i3JQss61a9E6+m5M1F0k/nHB9b72kCi3QzJkXtGN5NadFRcDBvkBcGL6HNdP3mRmur9s
 uyEg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to;
 bh=RSykA11zPfpk99eUA+3H9NyfJn/uSfzUhZ5+YCNxFng=;
 b=Es7j4L/xZW13zFc0l9ayd0JHYyhXh/gHcdktwEzong/fi49Cd9yvrFa4qull8CSx5x
 0YiiOD6XTAdYyd8kyyCI+7QDVn90GYLqXsoV744ntBJAJ778aw8MKarxIXyS1a1ngflM
 6LqMuGexN44E6G1Xxzei3GMcwp2LoZxDySxilpHX5t7vatWqA222hGrClorzmYICsU2S
 RmTxueiBvLbTjhbF3dHzEE0N45Dy64Gj0tsWpj/8eFouY+6XB6cJaR9h7cIIPjHD66Al
 m2Qp4s61IKz8y/Qd7qKMS2wdpK41lF7l+bPMGcjFGu17Bbe+l+HCs/Lw5j42rfN2ksEm
 D58A==
X-Gm-Message-State: AOAM532o/AO7vFQYcqkzg4PbXK41/gfgme5KnXNVwo4X+UtbiZY3Y33V
 KSMY9s9Dhk3q3xcwwj/EoGj3RArmI6WHFs8NSUA=
X-Google-Smtp-Source: ABdhPJzfXsuMZZGujGssQAPGbkQ0pGdkuJA0UB77t7oy2v3f+nRA7ValeOLHaxX5xlx0HSEJQiMSWPdwnW+fKfihdro=
X-Received: by 2002:a05:6830:1011:: with SMTP id
 a17mr221042otp.154.1614842749289; 
 Wed, 03 Mar 2021 23:25:49 -0800 (PST)
MIME-Version: 1.0
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <YD6pKJUrra3Rt2b9@HIDDEN>
 <CAOqdjBfnTu2kW1qrHbn_scU08B2gy9rM6eHPdu2GRFPuZ=XaMA@HIDDEN>
 <YD/qR9UC3jwLQs0Q@HIDDEN>
In-Reply-To: <YD/qR9UC3jwLQs0Q@HIDDEN>
From: Pip Cet <pipcet@HIDDEN>
Date: Thu, 4 Mar 2021 07:25:13 +0000
Message-ID: <CAOqdjBeZVSoC8cVY5o3LTK-uowVBj64r1FBqwxyh_LcwB3J0aw@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Alan Third <alan@HIDDEN>, Pip Cet <pipcet@HIDDEN>,
 46881 <at> debbugs.gnu.org
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On Wed, Mar 3, 2021 at 7:58 PM Alan Third <alan@HIDDEN> wrote:
> On Wed, Mar 03, 2021 at 07:10:28AM +0000, Pip Cet wrote:
> > > My quick test on macOS by doing:
> > >
> > > rm src/*.pdmp
> > > time make
> > >
> > > sees it going from ~26s without patch to ~10s with patch, so a
> > > considerable improvement.
> >
> > Thanks for testing!
> >
> > > > > Patch will be attached once this has a bug number.
> > > >
> > > > And here's the patch. Testing would be very appreciated.
> > >
> > > It appears to work fine here, but I don't know if there's anything
> > > specific to test other than just running Emacs.
> >
> > I suspect there may be problems on systems with very little memory. Do
> > you have an easy way to determine maximum resident size (on Debian
> > GNU/Linux, /bin/time works)? It would be interesting to see if that's
> > actually different.
>
> I tried using time -l the same as above and both came  out with
> roughly the same values, but I suspect that's probably just me getting
> the values for running make.

Thanks.

> Is there a better way of testing dumping?

I guess you could run "time  ./temacs --batch  -l loadup
--temacs=pbootstrap" directly (in src/)...

I realize that's not an answer to your question, but IME, dumping bugs
will often lead to immediate failures to bootstrap, whereas GC bugs
are tricky and don't. Since this bug doesn't affect GC in any way, I
think we've done enough initial testing.

Pip




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 19:58:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 03 14:58:12 2021
Received: from localhost ([127.0.0.1]:57064 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHXdM-0003jR-I7
	for submit <at> debbugs.gnu.org; Wed, 03 Mar 2021 14:58:12 -0500
Received: from outbound.soverin.net ([116.202.65.218]:43347)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <alan@HIDDEN>) id 1lHXdK-0003jE-DZ
 for 46881 <at> debbugs.gnu.org; Wed, 03 Mar 2021 14:58:11 -0500
Received: from smtp.soverin.net (unknown [10.10.3.24])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (No client certificate requested)
 by outbound.soverin.net (Postfix) with ESMTPS id 8F72160293
 for <46881 <at> debbugs.gnu.org>; Wed,  3 Mar 2021 19:58:02 +0000 (UTC)
Received: from smtp.soverin.net (smtp.soverin.net [159.69.232.138]) by
 soverin.net
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=idiocy.org; s=soverin;
 t=1614801482; bh=NX9Txa0W5RxAPS57duXK5gx3K+SNhSY779l6SVevE8E=;
 h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
 b=OA3Tv6McME/Xc9gQLKlTxJhKAUwdL+hLgwlLrUocZX5dKjdWiVtkiXEBHAjP2ackY
 HctFnCYWjSl8/j7FinLxj3F+VrvI/20xWNiYpZiP/Yp5V+EU2QE56b+ab6YBrtFE8c
 RdBgZCl3G0pijOsLgggYmgmPZgaH/3+a3/m4/4GsVjlj8VTw2EQhZ3lk0CWEaU/aHp
 GJeLmOeqz4HcTPe9uYxeP9b2Z0OO4ph3QI3HcWI86zC7SPaoxWj6P775TzGiosofhs
 VJch/fK6TbuctnWF1Kdz+vEF6/36FVEHHSzMnAsCeh3Xc0QrJpUeZ5LyXN8V6XRpAB
 N4B2fbWMZZFsQ==
Received: by breton.holly.idiocy.org (Postfix, from userid 501)
 id 6D72D202AC618E; Wed,  3 Mar 2021 19:57:59 +0000 (GMT)
Date: Wed, 3 Mar 2021 19:57:59 +0000
From: Alan Third <alan@HIDDEN>
To: Pip Cet <pipcet@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
Message-ID: <YD/qR9UC3jwLQs0Q@HIDDEN>
Mail-Followup-To: Alan Third <alan@HIDDEN>, Pip Cet <pipcet@HIDDEN>,
 46881 <at> debbugs.gnu.org
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <YD6pKJUrra3Rt2b9@HIDDEN>
 <CAOqdjBfnTu2kW1qrHbn_scU08B2gy9rM6eHPdu2GRFPuZ=XaMA@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAOqdjBfnTu2kW1qrHbn_scU08B2gy9rM6eHPdu2GRFPuZ=XaMA@HIDDEN>
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

On Wed, Mar 03, 2021 at 07:10:28AM +0000, Pip Cet wrote:
> > My quick test on macOS by doing:
> >
> > rm src/*.pdmp
> > time make
> >
> > sees it going from ~26s without patch to ~10s with patch, so a
> > considerable improvement.
> 
> Thanks for testing!
> 
> > > > Patch will be attached once this has a bug number.
> > >
> > > And here's the patch. Testing would be very appreciated.
> >
> > It appears to work fine here, but I don't know if there's anything
> > specific to test other than just running Emacs.
> 
> I suspect there may be problems on systems with very little memory. Do
> you have an easy way to determine maximum resident size (on Debian
> GNU/Linux, /bin/time works)? It would be interesting to see if that's
> actually different.

I tried using time -l the same as above and both came  out with
roughly the same values, but I suspect that's probably just me getting
the values for running make.

Is there a better way of testing dumping?
-- 
Alan Third




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 19:35:40 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 03 14:35:40 2021
Received: from localhost ([127.0.0.1]:57036 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHXHY-0003Bp-2l
	for submit <at> debbugs.gnu.org; Wed, 03 Mar 2021 14:35:40 -0500
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:57290)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eggert@HIDDEN>) id 1lHXHX-0003Bc-00
 for 46881 <at> debbugs.gnu.org; Wed, 03 Mar 2021 14:35:39 -0500
Received: from localhost (localhost [127.0.0.1])
 by zimbra.cs.ucla.edu (Postfix) with ESMTP id D2E591600F3;
 Wed,  3 Mar 2021 11:35:32 -0800 (PST)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
 by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
 with ESMTP id UMigWG2Oy0O3; Wed,  3 Mar 2021 11:35:32 -0800 (PST)
Received: from localhost (localhost [127.0.0.1])
 by zimbra.cs.ucla.edu (Postfix) with ESMTP id 2EF4F1600CC;
 Wed,  3 Mar 2021 11:35:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
 by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id 931QkZ5XMszY; Wed,  3 Mar 2021 11:35:32 -0800 (PST)
Received: from [192.168.1.9] (cpe-23-243-218-95.socal.res.rr.com
 [23.243.218.95])
 by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 048BE1600BC;
 Wed,  3 Mar 2021 11:35:31 -0800 (PST)
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Pip Cet <pipcet@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN>
 <CAOqdjBcLB-zzV=Kbh6xcJHuATZBy5je-UtA4SwM07ZvtyE3RmQ@HIDDEN>
From: Paul Eggert <eggert@HIDDEN>
Organization: UCLA Computer Science Department
Message-ID: <e87b8253-ff9d-135c-0d9c-a271430f9833@HIDDEN>
Date: Wed, 3 Mar 2021 11:35:31 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.7.1
MIME-Version: 1.0
In-Reply-To: <CAOqdjBcLB-zzV=Kbh6xcJHuATZBy5je-UtA4SwM07ZvtyE3RmQ@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Daniel Colascione <dancol@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

On 3/2/21 11:35 PM, Pip Cet wrote:
> IIUC, the prohibition on calling malloc, if it is still a concern,
> applies only when loading the dump, not while writing it.

That's my understanding as well.

> My main concern is the possibility of a partly-written dump file,
> since we no longer turn "!UMPEDGNUEMACS" into "DUMPEDGNUEMACS" after
> the dump. Maybe it would make sense to restore that feature?

Wouldn't hurt, though I'd make it low priority.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 15:09:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 03 10:09:34 2021
Received: from localhost ([127.0.0.1]:56615 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHT82-0000Se-LT
	for submit <at> debbugs.gnu.org; Wed, 03 Mar 2021 10:09:34 -0500
Received: from quimby.gnus.org ([95.216.78.240]:40160)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <larsi@HIDDEN>) id 1lHT80-0000SO-Ci
 for 46881 <at> debbugs.gnu.org; Wed, 03 Mar 2021 10:09:33 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org;
 s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:
 References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:
 Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender:
 Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:
 List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=dWcvz+PRXKBKPusURui7a7AcOm+ihfRCKPJ+xgNRmCE=; b=RrAyIQ/t0jwcKEzBRiEWb3w3fr
 DcyVjjH77m2bN5fSpyjJdovZgCIV8iuQKzr2KpmurclmlhSKjybnUYoU/nqM6klOpSi/yJjLWRf+8
 fYTHrv/Q+fuJHVIL353fWN1+BybXxP/hWXsv3OPk2jOuVnWwa8aFKuy1j+jd8aa0xSwQ=;
Received: from cm-84.212.220.105.getinternet.no ([84.212.220.105] helo=xo)
 by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.92) (envelope-from <larsi@HIDDEN>)
 id 1lHT7k-0001G5-Jr; Wed, 03 Mar 2021 16:09:19 +0100
From: Lars Ingebrigtsen <larsi@HIDDEN>
To: Pip Cet <pipcet@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN>
 <CAOqdjBcLB-zzV=Kbh6xcJHuATZBy5je-UtA4SwM07ZvtyE3RmQ@HIDDEN>
X-Now-Playing: Front 242's _Geography_: "He Runs Too Fast For Us"
Date: Wed, 03 Mar 2021 16:09:15 +0100
In-Reply-To: <CAOqdjBcLB-zzV=Kbh6xcJHuATZBy5je-UtA4SwM07ZvtyE3RmQ@HIDDEN>
 (Pip Cet's message of "Wed, 3 Mar 2021 07:35:45 +0000")
Message-ID: <87tupsw9yc.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 @@CONTACT_ADDRESS@@ for details.
 Content preview: Pip Cet <pipcet@HIDDEN> writes: > Since debuggability is
 such a concern, we probably shouldn't leak the > buffer memory. Revised patch
 attached. (This patch also removes the > lseek() syscalls; while not quite
 as numerous as the rea [...] 
 Content analysis details:   (-2.9 points, 5.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -1.0 ALL_TRUSTED            Passed through trusted hosts only via SMTP
 -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
 [score: 0.0000]
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>,
 Daniel Colascione <dancol@HIDDEN>, Paul Eggert <eggert@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Pip Cet <pipcet@HIDDEN> writes:

> Since debuggability is such a concern, we probably shouldn't leak the
> buffer memory. Revised patch attached. (This patch also removes the
> lseek() syscalls; while not quite as numerous as the read() ones,
> those did clutter up straces here).

I've tried the patch on a couple of systems here, and the resulting
Emacs works fine (as expected), and the pdumping is significantly faster
here, too.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 07:36:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 03 02:36:32 2021
Received: from localhost ([127.0.0.1]:55060 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHM3b-0005oy-MQ
	for submit <at> debbugs.gnu.org; Wed, 03 Mar 2021 02:36:32 -0500
Received: from mail-oi1-f176.google.com ([209.85.167.176]:32780)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lHM3X-0005og-FX
 for 46881 <at> debbugs.gnu.org; Wed, 03 Mar 2021 02:36:30 -0500
Received: by mail-oi1-f176.google.com with SMTP id a13so24990995oid.0
 for <46881 <at> debbugs.gnu.org>; Tue, 02 Mar 2021 23:36:27 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=+EECznTjamyCy/0zBbo7zaWPtTrz+NZQaFEIg71G6L8=;
 b=OyxWogHD1aqtW7C98jy5/rqH272gJ7yTlXMVLA5H5zBqqxUmhqJ8orR4jOSBgzWsYF
 KFs887UQD8Kj+kTNIwRFacM0dsp/oIi+iXvhvPyIemILHJtYJ6v/iOXaWzMEMRMy86Ty
 poszsU9oNAW2Tth8ozljY37IvWEjzQwDpb7U5wUmVKTBpaZRgN8HJ73K1WYQXS51Vv6V
 LDcKNTF47OjXIGOxJx/B2xVuF5LInqkb9902GXC+bNHXAHHQ2iZixLe+y06dsZtauMaY
 WZZpJB7Zb6v68ETh/V4Pn2t48Px9hqBxoi1D4HhbU4btNGdd9OKC5PVEvFDuJCO8MfoI
 OfQQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=+EECznTjamyCy/0zBbo7zaWPtTrz+NZQaFEIg71G6L8=;
 b=j5Pb+j1rI593i89onfQZUZ9l3N/cFdylRyg6kItjQlAtKlXC1xltSyAJSFmu9yVVGb
 BochGszsIlfZ7EN0QZjSsAY5fp7uVWy2lV/8AjZJeqkJL+rnxZ/U31o7yf6l1SekndBc
 RwUDUdfyGUWN8I2RKII0NK7EWD18RdPoQhOM+q9QVBZZv/pk+Ugb0YhP8LQWjeM3gJPH
 mNgU1e6L514IlaMNuSX5bxkjyt4UqwZ4sDMzJjeHUHwx+IcMDR1MXi1bIoGDEuE5Ui8l
 G8rxwfSzwQ9u1JMuAS48NbrBNj+jnVa9lINJmZP1n3ghRiALRiXULkWCJMnMv4sYh/Th
 brew==
X-Gm-Message-State: AOAM531vKJA1RWJ9VSwH1gHeGFiIjf+lk5MkkUHMahvvTPDTcRGpM9CI
 W1VIozKCj/ieXhpsEDbFu0A+5C+yOTMJoM5ZBDs=
X-Google-Smtp-Source: ABdhPJwhcJOmI9ybAmsRtall1VyrbFUVFaWB4MoQ2UZ2NfJUJJxRA/Zc/xHYjJG1pH4Ar+zcOjifczLTCo/KEcoadvw=
X-Received: by 2002:aca:d905:: with SMTP id q5mr2758632oig.30.1614756981917;
 Tue, 02 Mar 2021 23:36:21 -0800 (PST)
MIME-Version: 1.0
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <83r1kw6b06.fsf@HIDDEN>
In-Reply-To: <83r1kw6b06.fsf@HIDDEN>
From: Pip Cet <pipcet@HIDDEN>
Date: Wed, 3 Mar 2021 07:35:45 +0000
Message-ID: <CAOqdjBcLB-zzV=Kbh6xcJHuATZBy5je-UtA4SwM07ZvtyE3RmQ@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Eli Zaretskii <eliz@HIDDEN>
Content-Type: multipart/mixed; boundary="0000000000004e66b405bc9ce8e0"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org, Daniel Colascione <dancol@HIDDEN>,
 Paul Eggert <eggert@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--0000000000004e66b405bc9ce8e0
Content-Type: text/plain; charset="UTF-8"

On Wed, Mar 3, 2021 at 5:51 AM Eli Zaretskii <eliz@HIDDEN> wrote:
> > From: Pip Cet <pipcet@HIDDEN>
> > Date: Tue, 2 Mar 2021 20:45:04 +0000
> >
> > On Tue, Mar 2, 2021 at 8:35 PM Pip Cet <pipcet@HIDDEN> wrote:
> > > I've looked into the problem, and it seems easy to solve and worth it
> > > in terms of debuggability and performance.

Since debuggability is such a concern, we probably shouldn't leak the
buffer memory. Revised patch attached. (This patch also removes the
lseek() syscalls; while not quite as numerous as the read() ones,
those did clutter up straces here).

> In particular, is it safe to allocate
> large amounts of memory off the heap while dumping?

Even if it isn't, we'd still be faster re-running the dump after
growing the dumper image than the current approach is.

>A couple of
> places in pdumper.c says some parts of code should call malloc.

IIUC, the prohibition on calling malloc, if it is still a concern,
applies only when loading the dump, not while writing it.

My main concern is the possibility of a partly-written dump file,
since we no longer turn "!UMPEDGNUEMACS" into "DUMPEDGNUEMACS" after
the dump. Maybe it would make sense to restore that feature?

Pip

--0000000000004e66b405bc9ce8e0
Content-Type: text/x-patch; charset="US-ASCII"; 
	name="0001-Prepare-pdumper-dump-file-in-memory-write-it-in-one-.patch"
Content-Disposition: attachment; 
	filename="0001-Prepare-pdumper-dump-file-in-memory-write-it-in-one-.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_klt4kmx30>
X-Attachment-Id: f_klt4kmx30

RnJvbSBmYWU2N2MwMjk1NWE1YmJlYTE2ZDU1NGI4ZTczNWRjOGJlZjZhOWUyIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBQaXAgQ2V0IDxwaXBjZXRAZ21haWwuY29tPgpEYXRlOiBUdWUs
IDIgTWFyIDIwMjEgMjA6Mzg6MjMgKzAwMDAKU3ViamVjdDogW1BBVENIXSBQcmVwYXJlIHBkdW1w
ZXIgZHVtcCBmaWxlIGluIG1lbW9yeSwgd3JpdGUgaXQgaW4gb25lIGdvCiAoQnVnIzQ2ODgxKQoK
KiBzcmMvcGR1bXBlci5jIChzdHJ1Y3QgZHVtcF9jb250ZXh0KTogQWRkIGJ1ZiwgYnVmX3NpemUs
IG1heF9vZmZzZXQgZmllbGRzLgooZHVtcF9ncm93X2J1ZmZlcik6IE5ldyBmdW5jdGlvbi4KKGR1
bXBfd3JpdGUpOiBVc2UgbWVtY3B5LCBub3QgYW4gYWN0dWFsIGVtYWNzX3dyaXRlLgooZHVtcF9z
ZWVrKTogS2VlcCB0cmFjayBvZiBtYXhpbXVtIHNlZW4gb2Zmc2V0LiBEb24ndCBhY3R1YWxseSBz
ZWVrLgooRmR1bXBfZW1hY3NfcG9ydGFibGUpOiBXcml0ZSBvdXQgdGhlIGZpbGUgY29udGVudHMg
d2hlbiBkb25lLgotLS0KIHNyYy9wZHVtcGVyLmMgfCAyNyArKysrKysrKysrKysrKysrKysrKysr
LS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCAyMiBpbnNlcnRpb25zKCspLCA1IGRlbGV0aW9ucygtKQoK
ZGlmZiAtLWdpdCBhL3NyYy9wZHVtcGVyLmMgYi9zcmMvcGR1bXBlci5jCmluZGV4IDMzNzc0MmZk
YTRhZGUuLjI5ZDFjYjg2MmUwN2YgMTAwNjQ0Ci0tLSBhL3NyYy9wZHVtcGVyLmMKKysrIGIvc3Jj
L3BkdW1wZXIuYwpAQCAtNDczLDYgKzQ3MywxMCBAQCBkdW1wX2ZpbmdlcnByaW50IChjaGFyIGNv
bnN0ICpsYWJlbCwKIHsKICAgLyogSGVhZGVyIHdlJ2xsIHdyaXRlIHRvIHRoZSBkdW1wIGZpbGUg
d2hlbiBkb25lLiAgKi8KICAgc3RydWN0IGR1bXBfaGVhZGVyIGhlYWRlcjsKKyAgLyogRGF0YSB0
aGF0IHdpbGwgYmUgd3JpdHRlbiB0byB0aGUgZHVtcCBmaWxlLiAgKi8KKyAgdm9pZCAqYnVmOwor
ICBkdW1wX29mZiBidWZfc2l6ZTsKKyAgZHVtcF9vZmYgbWF4X29mZnNldDsKIAogICBMaXNwX09i
amVjdCBvbGRfcHVyaWZ5X2ZsYWc7CiAgIExpc3BfT2JqZWN0IG9sZF9wb3N0X2djX2hvb2s7CkBA
IC01ODEsNiArNTg1LDEzIEBAIGR1bXBfZmluZ2VycHJpbnQgKGNoYXIgY29uc3QgKmxhYmVsLAog
DAogLyogRHVtcCBmaWxlIGNyZWF0aW9uICovCiAKK3N0YXRpYyB2b2lkIGR1bXBfZ3Jvd19idWZm
ZXIgKHN0cnVjdCBkdW1wX2NvbnRleHQgKmN0eCkKK3sKKyAgY3R4LT5idWYgPSB4cmVhbGxvYyAo
Y3R4LT5idWYsIGN0eC0+YnVmX3NpemUgPSAoY3R4LT5idWZfc2l6ZSA/CisJCQkJCQkgIChjdHgt
PmJ1Zl9zaXplICogMikKKwkJCQkJCSAgOiA4ICogMTAyNCAqIDEwMjQpKTsKK30KKwogc3RhdGlj
IGR1bXBfb2ZmIGR1bXBfb2JqZWN0IChzdHJ1Y3QgZHVtcF9jb250ZXh0ICpjdHgsIExpc3BfT2Jq
ZWN0IG9iamVjdCk7CiBzdGF0aWMgZHVtcF9vZmYgZHVtcF9vYmplY3RfZm9yX29mZnNldCAoc3Ry
dWN0IGR1bXBfY29udGV4dCAqY3R4LAogCQkJCQlMaXNwX09iamVjdCBvYmplY3QpOwpAQCAtNzQ3
LDggKzc1OCw5IEBAIGR1bXBfd3JpdGUgKHN0cnVjdCBkdW1wX2NvbnRleHQgKmN0eCwgY29uc3Qg
dm9pZCAqYnVmLCBkdW1wX29mZiBuYnl0ZSkKICAgZWFzc2VydCAobmJ5dGUgPT0gMCB8fCBidWYg
IT0gTlVMTCk7CiAgIGVhc3NlcnQgKGN0eC0+b2JqX29mZnNldCA9PSAwKTsKICAgZWFzc2VydCAo
Y3R4LT5mbGFncy5kdW1wX29iamVjdF9jb250ZW50cyk7Ci0gIGlmIChlbWFjc193cml0ZSAoY3R4
LT5mZCwgYnVmLCBuYnl0ZSkgPCBuYnl0ZSkKLSAgICByZXBvcnRfZmlsZV9lcnJvciAoIkNvdWxk
IG5vdCB3cml0ZSB0byBkdW1wIGZpbGUiLCBjdHgtPmR1bXBfZmlsZW5hbWUpOworICB3aGlsZSAo
Y3R4LT5vZmZzZXQgKyBuYnl0ZSA+IGN0eC0+YnVmX3NpemUpCisgICAgZHVtcF9ncm93X2J1ZmZl
ciAoY3R4KTsKKyAgbWVtY3B5ICgoY2hhciAqKWN0eC0+YnVmICsgY3R4LT5vZmZzZXQsIGJ1Ziwg
bmJ5dGUpOwogICBjdHgtPm9mZnNldCArPSBuYnl0ZTsKIH0KIApAQCAtODI4LDEwICs4NDAsOSBA
QCBkdW1wX3RhaWxxX3BvcCAoc3RydWN0IGR1bXBfdGFpbHEgKnRhaWxxKQogc3RhdGljIHZvaWQK
IGR1bXBfc2VlayAoc3RydWN0IGR1bXBfY29udGV4dCAqY3R4LCBkdW1wX29mZiBvZmZzZXQpCiB7
CisgIGlmIChjdHgtPm1heF9vZmZzZXQgPCBjdHgtPm9mZnNldCkKKyAgICBjdHgtPm1heF9vZmZz
ZXQgPSBjdHgtPm9mZnNldDsKICAgZWFzc2VydCAoY3R4LT5vYmpfb2Zmc2V0ID09IDApOwotICBp
ZiAobHNlZWsgKGN0eC0+ZmQsIG9mZnNldCwgU0VFS19TRVQpIDwgMCkKLSAgICByZXBvcnRfZmls
ZV9lcnJvciAoIlNldHRpbmcgZmlsZSBwb3NpdGlvbiIsCi0gICAgICAgICAgICAgICAgICAgICAg
IGN0eC0+ZHVtcF9maWxlbmFtZSk7CiAgIGN0eC0+b2Zmc2V0ID0gb2Zmc2V0OwogfQogCkBAIC00
MTU5LDYgKzQxNzAsMTIgQEAgREVGVU4gKCJkdW1wLWVtYWNzLXBvcnRhYmxlIiwKICAgY3R4LT5o
ZWFkZXIubWFnaWNbMF0gPSBkdW1wX21hZ2ljWzBdOwogICBkdW1wX3NlZWsgKGN0eCwgMCk7CiAg
IGR1bXBfd3JpdGUgKGN0eCwgJmN0eC0+aGVhZGVyLCBzaXplb2YgKGN0eC0+aGVhZGVyKSk7Cisg
IGlmIChlbWFjc193cml0ZSAoY3R4LT5mZCwgY3R4LT5idWYsIGN0eC0+bWF4X29mZnNldCkgPCBj
dHgtPm1heF9vZmZzZXQpCisgICAgcmVwb3J0X2ZpbGVfZXJyb3IgKCJDb3VsZCBub3Qgd3JpdGUg
dG8gZHVtcCBmaWxlIiwgY3R4LT5kdW1wX2ZpbGVuYW1lKTsKKyAgeGZyZWUgKGN0eC0+YnVmKTsK
KyAgY3R4LT5idWYgPSBOVUxMOworICBjdHgtPmJ1Zl9zaXplID0gMDsKKyAgY3R4LT5tYXhfb2Zm
c2V0ID0gMDsKIAogICBkdW1wX29mZgogICAgIGhlYWRlcl9ieXRlcyA9IGhlYWRlcl9lbmQgLSBo
ZWFkZXJfc3RhcnQsCi0tIAoyLjMwLjEKCg==
--0000000000004e66b405bc9ce8e0--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 07:11:13 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 03 02:11:13 2021
Received: from localhost ([127.0.0.1]:55030 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHLf7-0005Dc-5n
	for submit <at> debbugs.gnu.org; Wed, 03 Mar 2021 02:11:13 -0500
Received: from mail-oo1-f46.google.com ([209.85.161.46]:38681)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lHLf3-0005DN-TD
 for 46881 <at> debbugs.gnu.org; Wed, 03 Mar 2021 02:11:11 -0500
Received: by mail-oo1-f46.google.com with SMTP id f26so5451516oog.5
 for <46881 <at> debbugs.gnu.org>; Tue, 02 Mar 2021 23:11:09 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=ohuOvlUVhjnXo88gJbDdHE0o+bd1U4Y1hNfdQ5HU7+A=;
 b=TdVVj+jXpk9iCc0QbhdwAqheQ4uL/NVsohBHY65b4OQPjgs9Fw9XWvpMsADl9sXv6R
 PIZCdYmbwVVjQYLeNKA0IiH8xGx1OzhUSPAuwumJi+/PEdGRDJoyuBTvYuzJFO8TYtwA
 J0W4Iu3SqjJrSGRHZaQtuc8trDGoPP+0bz0QjclLvNYkFxtLsVn6B5KolajG/sfyHvGy
 qPFIp9+L1hsnsWCmH7pP8xFZ4VgmBmCrU/OHcVDpN9QlCtxUh8NWVTghY+yNlCCRalyr
 OIqCcCdb35s5A+A21zcKW1Fuq0t32zW1KMzUClh91AyEaCEZQzKXOPPXTgJhzlnrQNOX
 hA3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=ohuOvlUVhjnXo88gJbDdHE0o+bd1U4Y1hNfdQ5HU7+A=;
 b=rm/GI8yhU2RQ/KfPcXDNQiqmM0PHvYpEaf37OJD1P486hrhXqwI+HyhCRBL/yMZRrt
 AyFrWjLc+ns2sqvYsB+65QVgTpsEx43FH6zV+OhHJHT3G2GAMqXuHr5yB54RS1hmK7dH
 o3aWc0sZXrZ6uwt2PIKYI9dUy9X27tEncc6C8jxnmhx7F/sLGDtmZ2VEcQk/TkVuYTCx
 kyEm3O3rrymwpSTuB/jiaBvFvwvSjVw2i7CklmZSGS+neww+wPvWp7tCFR1x9xRZ2Iml
 f6wxaz4cQwreDrOPA/2Z0+vtc6W3jSl8/ZvrTOKyQPIDsnc34r1c2ostsvmUdMSzAW12
 rOZA==
X-Gm-Message-State: AOAM530HNHCincOZ4uj7LqEU8tjF3pDsER/vLWSllq3a02yoLpnIgcSM
 j1L7S4VGa2AksrLwdvpmrP2oAYzI3qA0YxMoUqc=
X-Google-Smtp-Source: ABdhPJwDiWksJBkLfj1cBZo0thE98VJ2g8+ALlu2EPNzM/ePAwZHKRJ9nvGlZDFcCUsGYcdm4vl6mJ/+OD2GwgjJWuo=
X-Received: by 2002:a4a:2511:: with SMTP id g17mr20213317ooa.22.1614755464381; 
 Tue, 02 Mar 2021 23:11:04 -0800 (PST)
MIME-Version: 1.0
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 <YD6pKJUrra3Rt2b9@HIDDEN>
In-Reply-To: <YD6pKJUrra3Rt2b9@HIDDEN>
From: Pip Cet <pipcet@HIDDEN>
Date: Wed, 3 Mar 2021 07:10:28 +0000
Message-ID: <CAOqdjBfnTu2kW1qrHbn_scU08B2gy9rM6eHPdu2GRFPuZ=XaMA@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: Alan Third <alan@HIDDEN>, Pip Cet <pipcet@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

> My quick test on macOS by doing:
>
> rm src/*.pdmp
> time make
>
> sees it going from ~26s without patch to ~10s with patch, so a
> considerable improvement.

Thanks for testing!

> > > Patch will be attached once this has a bug number.
> >
> > And here's the patch. Testing would be very appreciated.
>
> It appears to work fine here, but I don't know if there's anything
> specific to test other than just running Emacs.

I suspect there may be problems on systems with very little memory. Do
you have an easy way to determine maximum resident size (on Debian
GNU/Linux, /bin/time works)? It would be interesting to see if that's
actually different.

Thanks again!
Pip




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 05:51:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 03 00:51:23 2021
Received: from localhost ([127.0.0.1]:54974 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHKPq-0003A2-Ul
	for submit <at> debbugs.gnu.org; Wed, 03 Mar 2021 00:51:23 -0500
Received: from eggs.gnu.org ([209.51.188.92]:41330)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1lHKPo-00039p-46
 for 46881 <at> debbugs.gnu.org; Wed, 03 Mar 2021 00:51:21 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e]:59393)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1lHKPh-0000S3-Ue; Wed, 03 Mar 2021 00:51:13 -0500
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2473
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1lHKPf-0003vJ-MQ; Wed, 03 Mar 2021 00:51:13 -0500
Date: Wed, 03 Mar 2021 07:51:05 +0200
Message-Id: <83r1kw6b06.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Pip Cet <pipcet@HIDDEN>, Daniel Colascione <dancol@HIDDEN>,
 Paul Eggert <eggert@HIDDEN>
In-Reply-To: <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
 (message from Pip Cet on Tue, 2 Mar 2021 20:45:04 +0000)
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

> From: Pip Cet <pipcet@HIDDEN>
> Date: Tue, 2 Mar 2021 20:45:04 +0000
> 
> On Tue, Mar 2, 2021 at 8:35 PM Pip Cet <pipcet@HIDDEN> wrote:
> > I've looked into the problem, and it seems easy to solve and worth it
> > in terms of debuggability and performance.
> 
> Very rough benchmarks, but this seems to be clearly worth it:
> 
> Performance:
> With patch:
> real    0m3.861s
> user    0m3.776s
> sys    0m0.085s
> 
> Without patch:
> real    0m7.001s
> user    0m4.476s
> sys    0m2.511s
> 
> Number of syscalls:
> With patch: 415442
> Without patch: 2028307
> 
> > Patch will be attached once this has a bug number.
> 
> And here's the patch. Testing would be very appreciated.
> 
> I'm unsure about the precise usage of dump_off vs ptrdiff_t here; I
> don't think it matters, but suggestions, nitpicks, and comments, on
> this or any other aspect, would be very appreciated.

> From 92ee138852b34ede2f43dd7f93f310fc746bb3bf Mon Sep 17 00:00:00 2001
> From: Pip Cet <pipcet@HIDDEN>
> Date: Tue, 2 Mar 2021 20:38:23 +0000
> Subject: [PATCH] Prepare pdumper dump file in memory, write it in one go
>  (Bug#46881)
> 
> * src/pdumper.c (struct dump_context): Add buf, buf_size, max_offset fields.
> (grow_buffer): New function.
> (dump_write): Use memcpy, not an actual emacs_write.
> (dump_seek): Keep track of maximum seen offset.
> (Fdump_emacs_portable): Write out the file contents when done.
> ---
>  src/pdumper.c | 20 ++++++++++++++++++--
>  1 file changed, 18 insertions(+), 2 deletions(-)
> 
> diff --git a/src/pdumper.c b/src/pdumper.c
> index 337742fda4ade..62ddad8ee5e34 100644
> --- a/src/pdumper.c
> +++ b/src/pdumper.c
> @@ -473,6 +473,10 @@ dump_fingerprint (char const *label,
>  {
>    /* Header we'll write to the dump file when done.  */
>    struct dump_header header;
> +  /* Data that will be written to the dump file.  */
> +  void *buf;
> +  ptrdiff_t buf_size;
> +  ptrdiff_t max_offset;
>  
>    Lisp_Object old_purify_flag;
>    Lisp_Object old_post_gc_hook;
> @@ -581,6 +585,13 @@ dump_fingerprint (char const *label,
>  
>  /* Dump file creation */
>  
> +static void dump_grow_buffer (struct dump_context *ctx)
> +{
> +  ctx->buf = xrealloc (ctx->buf, ctx->buf_size = (ctx->buf_size ?
> +						  (ctx->buf_size * 2)
> +						  : 1024 * 1024));
> +}
> +
>  static dump_off dump_object (struct dump_context *ctx, Lisp_Object object);
>  static dump_off dump_object_for_offset (struct dump_context *ctx,
>  					Lisp_Object object);
> @@ -747,8 +758,9 @@ dump_write (struct dump_context *ctx, const void *buf, dump_off nbyte)
>    eassert (nbyte == 0 || buf != NULL);
>    eassert (ctx->obj_offset == 0);
>    eassert (ctx->flags.dump_object_contents);
> -  if (emacs_write (ctx->fd, buf, nbyte) < nbyte)
> -    report_file_error ("Could not write to dump file", ctx->dump_filename);
> +  while (ctx->offset + nbyte > ctx->buf_size)
> +    dump_grow_buffer (ctx);
> +  memcpy ((char *)ctx->buf + ctx->offset, buf, nbyte);
>    ctx->offset += nbyte;
>  }
>  
> @@ -828,6 +840,8 @@ dump_tailq_pop (struct dump_tailq *tailq)
>  static void
>  dump_seek (struct dump_context *ctx, dump_off offset)
>  {
> +  if (ctx->max_offset < ctx->offset)
> +    ctx->max_offset = ctx->offset;
>    eassert (ctx->obj_offset == 0);
>    if (lseek (ctx->fd, offset, SEEK_SET) < 0)
>      report_file_error ("Setting file position",
> @@ -4159,6 +4173,8 @@ DEFUN ("dump-emacs-portable",
>    ctx->header.magic[0] = dump_magic[0];
>    dump_seek (ctx, 0);
>    dump_write (ctx, &ctx->header, sizeof (ctx->header));
> +  if (emacs_write (ctx->fd, ctx->buf, ctx->max_offset) < ctx->max_offset)
> +    report_file_error ("Could not write to dump file", ctx->dump_filename);
>  
>    dump_off
>      header_bytes = header_end - header_start,
> -- 
> 2.30.1

Thanks.

Daniel, Paul: any comments?  In particular, is it safe to allocate
large amounts of memory off the heap while dumping?  A couple of
places in pdumper.c says some parts of code should call malloc.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 2 Mar 2021 21:08:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 02 16:08:08 2021
Received: from localhost ([127.0.0.1]:54354 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHCFT-0005MA-6T
	for submit <at> debbugs.gnu.org; Tue, 02 Mar 2021 16:08:08 -0500
Received: from outbound.soverin.net ([116.202.65.218]:36931)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <alan@HIDDEN>) id 1lHCFN-0005LY-Sc
 for 46881 <at> debbugs.gnu.org; Tue, 02 Mar 2021 16:08:05 -0500
Received: from smtp.soverin.net (unknown [10.10.3.24])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (No client certificate requested)
 by outbound.soverin.net (Postfix) with ESMTPS id AC4E4600F5
 for <46881 <at> debbugs.gnu.org>; Tue,  2 Mar 2021 21:07:55 +0000 (UTC)
Received: from smtp.soverin.net (smtp.soverin.net [159.69.232.138]) by
 soverin.net
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=idiocy.org; s=soverin;
 t=1614719275; bh=Gn/fFdJQmHej7f8G4cjroZ7nBBxCt7DRt2VL9NyGNpw=;
 h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
 b=RB+7EVHCU9wyz/duzQr5DatpVzj+Wg34btNdMPbXYvAaZeEDoFrBQth3cbNnyU+eO
 DgkbitZB9XGkrk7aRol1PumAk8sOPSisVfjGveXS5Za4A6c/bnQRzrYQDQ3NZEox3E
 5nmTaaoKNG/dYYTwtggXYn7PLfy8ELkwNtqDRIXao6j2VnuHrAefqNCRQeQNdwH9q3
 QjssipDNBV6c6C+NHVZyMVhIVO6HuOxc3yTjMBW6WGeSCB74iwR5CbzQsRaCopcWNr
 JUTbfhFuFeNmPqunflg09m3EdwZ2oF55oRe5bLdBGjvEcdxq/OFvOg2/n3e4IzWpXO
 tZm5VbYky6a4A==
Received: by breton.holly.idiocy.org (Postfix, from userid 501)
 id D1914202AC44B9; Tue,  2 Mar 2021 21:07:52 +0000 (GMT)
Date: Tue, 2 Mar 2021 21:07:52 +0000
From: Alan Third <alan@HIDDEN>
To: Pip Cet <pipcet@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
Message-ID: <YD6pKJUrra3Rt2b9@HIDDEN>
Mail-Followup-To: Alan Third <alan@HIDDEN>, Pip Cet <pipcet@HIDDEN>,
 46881 <at> debbugs.gnu.org
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
 <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 46881
Cc: 46881 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

On Tue, Mar 02, 2021 at 08:45:04PM +0000, Pip Cet wrote:
> On Tue, Mar 2, 2021 at 8:35 PM Pip Cet <pipcet@HIDDEN> wrote:
> > I've looked into the problem, and it seems easy to solve and worth it
> > in terms of debuggability and performance.
> 
> Very rough benchmarks, but this seems to be clearly worth it:
> 
> Performance:
> With patch:
> real    0m3.861s
> user    0m3.776s
> sys    0m0.085s
> 
> Without patch:
> real    0m7.001s
> user    0m4.476s
> sys    0m2.511s
> 
> Number of syscalls:
> With patch: 415442
> Without patch: 2028307

My quick test on macOS by doing:

rm src/*.pdmp
time make

sees it going from ~26s without patch to ~10s with patch, so a
considerable improvement.

> > Patch will be attached once this has a bug number.
> 
> And here's the patch. Testing would be very appreciated.

It appears to work fine here, but I don't know if there's anything
specific to test other than just running Emacs.
-- 
Alan Third




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at 46881 <at> debbugs.gnu.org:


Received: (at 46881) by debbugs.gnu.org; 2 Mar 2021 20:45:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 02 15:45:48 2021
Received: from localhost ([127.0.0.1]:54341 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHBtr-0004p9-VY
	for submit <at> debbugs.gnu.org; Tue, 02 Mar 2021 15:45:48 -0500
Received: from mail-oi1-f174.google.com ([209.85.167.174]:40507)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lHBtq-0004ow-Bk
 for 46881 <at> debbugs.gnu.org; Tue, 02 Mar 2021 15:45:46 -0500
Received: by mail-oi1-f174.google.com with SMTP id 21so13828146oiq.7
 for <46881 <at> debbugs.gnu.org>; Tue, 02 Mar 2021 12:45:46 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to;
 bh=WCycNmA77HpRI9xt4Gr+dg728tCYASb74Wtzho97ICc=;
 b=pY2GEpx8PsbbHj3XYOjuSr9K6B9Ds6q+sL6N5aJECRVlNu2QxH0d3JZl8YNnqFMVp3
 8JLQu7UL74Xu4DJf+NkvURNCcOpM2xfd+y0GL8PFgHdiyL3oKfjXSd1sgVKlePlQvpJq
 YYdWCoeK9iFehPo+/Q4oiqsdM/OiQ3FKmcgpXKtuwPKphMRxQyGpesymBeao/f4omWXx
 6AigW0a6U1qC7RvR/sP41ghmOIcq1ctwEpM1muFuaUQjyD/jNqp9TxMC9wg0biEn+YiR
 Nc/frP/Pv/dMJJEB75hcY+U1BMQU97vRjDY07OlDi0DaIadYcPlwsmp1sGtL3duaWTo/
 knaA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to;
 bh=WCycNmA77HpRI9xt4Gr+dg728tCYASb74Wtzho97ICc=;
 b=LmaDAh1DRZk+bjHXa3gXNz0MZFj1d4CKeG6IF5aU91jNpIm8KC0cwu83FVXSSAl9as
 1tXwDW//hpzT0Y6t8TWMMLg4yUsYdgfBl8YHxjd0wok6siErFe+9yzjix0BilEc2mIq4
 46UoF0ChVkgaDoAZ1MwuytJjVna+7teIQF1ScCD9Qb6qdY5pQnfeqw7J7XF8R3ncN3u5
 xGw31Rj5MqnXJ7zyekJFJtgaMitQ3PTnCIVcbURxJIz5tjVKi98jYkb/ptifkrPFTScX
 jILr5IK8KJLTmrh/qtWVrvVW0K40eoQemfJV2OgBO1k7XSyGJmqcIJgUTLOZ8To81SpU
 q91w==
X-Gm-Message-State: AOAM532kk53TTkeAOBcHlkbwxycyV9/SmchxhZCXBE3vczRaYDV3yRLW
 EEut2ORCJXqQy+JJtBJQKz1L57zTTaL53U34T3K/xj9wruN5Lw==
X-Google-Smtp-Source: ABdhPJydEwKrvPLuo+xN+t+YFTvZU+QoELiX6iK00P1J+r5SwwAc0CDo08g4BfMZ3hz2TvenDR+jasyaUynu6uMXQUA=
X-Received: by 2002:aca:d905:: with SMTP id q5mr1111434oig.30.1614717940575;
 Tue, 02 Mar 2021 12:45:40 -0800 (PST)
MIME-Version: 1.0
References: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
In-Reply-To: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
From: Pip Cet <pipcet@HIDDEN>
Date: Tue, 2 Mar 2021 20:45:04 +0000
Message-ID: <CAOqdjBd-N=qN3B5v1Bvg_rgRL=yqYk+jzaZD3O1fDm+VJCGhnA@HIDDEN>
Subject: Re: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls
To: 46881 <at> debbugs.gnu.org
Content-Type: multipart/mixed; boundary="00000000000042cc0805bc93d141"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 46881
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--00000000000042cc0805bc93d141
Content-Type: text/plain; charset="UTF-8"

On Tue, Mar 2, 2021 at 8:35 PM Pip Cet <pipcet@HIDDEN> wrote:
> I've looked into the problem, and it seems easy to solve and worth it
> in terms of debuggability and performance.

Very rough benchmarks, but this seems to be clearly worth it:

Performance:
With patch:
real    0m3.861s
user    0m3.776s
sys    0m0.085s

Without patch:
real    0m7.001s
user    0m4.476s
sys    0m2.511s

Number of syscalls:
With patch: 415442
Without patch: 2028307

> Patch will be attached once this has a bug number.

And here's the patch. Testing would be very appreciated.

I'm unsure about the precise usage of dump_off vs ptrdiff_t here; I
don't think it matters, but suggestions, nitpicks, and comments, on
this or any other aspect, would be very appreciated.

Pip

--00000000000042cc0805bc93d141
Content-Type: text/x-patch; charset="US-ASCII"; 
	name="0001-Prepare-pdumper-dump-file-in-memory-write-it-in-one-.patch"
Content-Disposition: attachment; 
	filename="0001-Prepare-pdumper-dump-file-in-memory-write-it-in-one-.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_klshagd70>
X-Attachment-Id: f_klshagd70

RnJvbSA5MmVlMTM4ODUyYjM0ZWRlMmY0M2RkN2Y5M2YzMTBmYzc0NmJiM2JmIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBQaXAgQ2V0IDxwaXBjZXRAZ21haWwuY29tPgpEYXRlOiBUdWUs
IDIgTWFyIDIwMjEgMjA6Mzg6MjMgKzAwMDAKU3ViamVjdDogW1BBVENIXSBQcmVwYXJlIHBkdW1w
ZXIgZHVtcCBmaWxlIGluIG1lbW9yeSwgd3JpdGUgaXQgaW4gb25lIGdvCiAoQnVnIzQ2ODgxKQoK
KiBzcmMvcGR1bXBlci5jIChzdHJ1Y3QgZHVtcF9jb250ZXh0KTogQWRkIGJ1ZiwgYnVmX3NpemUs
IG1heF9vZmZzZXQgZmllbGRzLgooZ3Jvd19idWZmZXIpOiBOZXcgZnVuY3Rpb24uCihkdW1wX3dy
aXRlKTogVXNlIG1lbWNweSwgbm90IGFuIGFjdHVhbCBlbWFjc193cml0ZS4KKGR1bXBfc2Vlayk6
IEtlZXAgdHJhY2sgb2YgbWF4aW11bSBzZWVuIG9mZnNldC4KKEZkdW1wX2VtYWNzX3BvcnRhYmxl
KTogV3JpdGUgb3V0IHRoZSBmaWxlIGNvbnRlbnRzIHdoZW4gZG9uZS4KLS0tCiBzcmMvcGR1bXBl
ci5jIHwgMjAgKysrKysrKysrKysrKysrKysrLS0KIDEgZmlsZSBjaGFuZ2VkLCAxOCBpbnNlcnRp
b25zKCspLCAyIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3NyYy9wZHVtcGVyLmMgYi9zcmMv
cGR1bXBlci5jCmluZGV4IDMzNzc0MmZkYTRhZGUuLjYyZGRhZDhlZTVlMzQgMTAwNjQ0Ci0tLSBh
L3NyYy9wZHVtcGVyLmMKKysrIGIvc3JjL3BkdW1wZXIuYwpAQCAtNDczLDYgKzQ3MywxMCBAQCBk
dW1wX2ZpbmdlcnByaW50IChjaGFyIGNvbnN0ICpsYWJlbCwKIHsKICAgLyogSGVhZGVyIHdlJ2xs
IHdyaXRlIHRvIHRoZSBkdW1wIGZpbGUgd2hlbiBkb25lLiAgKi8KICAgc3RydWN0IGR1bXBfaGVh
ZGVyIGhlYWRlcjsKKyAgLyogRGF0YSB0aGF0IHdpbGwgYmUgd3JpdHRlbiB0byB0aGUgZHVtcCBm
aWxlLiAgKi8KKyAgdm9pZCAqYnVmOworICBwdHJkaWZmX3QgYnVmX3NpemU7CisgIHB0cmRpZmZf
dCBtYXhfb2Zmc2V0OwogCiAgIExpc3BfT2JqZWN0IG9sZF9wdXJpZnlfZmxhZzsKICAgTGlzcF9P
YmplY3Qgb2xkX3Bvc3RfZ2NfaG9vazsKQEAgLTU4MSw2ICs1ODUsMTMgQEAgZHVtcF9maW5nZXJw
cmludCAoY2hhciBjb25zdCAqbGFiZWwsCiAMCiAvKiBEdW1wIGZpbGUgY3JlYXRpb24gKi8KIAor
c3RhdGljIHZvaWQgZHVtcF9ncm93X2J1ZmZlciAoc3RydWN0IGR1bXBfY29udGV4dCAqY3R4KQor
eworICBjdHgtPmJ1ZiA9IHhyZWFsbG9jIChjdHgtPmJ1ZiwgY3R4LT5idWZfc2l6ZSA9IChjdHgt
PmJ1Zl9zaXplID8KKwkJCQkJCSAgKGN0eC0+YnVmX3NpemUgKiAyKQorCQkJCQkJICA6IDEwMjQg
KiAxMDI0KSk7Cit9CisKIHN0YXRpYyBkdW1wX29mZiBkdW1wX29iamVjdCAoc3RydWN0IGR1bXBf
Y29udGV4dCAqY3R4LCBMaXNwX09iamVjdCBvYmplY3QpOwogc3RhdGljIGR1bXBfb2ZmIGR1bXBf
b2JqZWN0X2Zvcl9vZmZzZXQgKHN0cnVjdCBkdW1wX2NvbnRleHQgKmN0eCwKIAkJCQkJTGlzcF9P
YmplY3Qgb2JqZWN0KTsKQEAgLTc0Nyw4ICs3NTgsOSBAQCBkdW1wX3dyaXRlIChzdHJ1Y3QgZHVt
cF9jb250ZXh0ICpjdHgsIGNvbnN0IHZvaWQgKmJ1ZiwgZHVtcF9vZmYgbmJ5dGUpCiAgIGVhc3Nl
cnQgKG5ieXRlID09IDAgfHwgYnVmICE9IE5VTEwpOwogICBlYXNzZXJ0IChjdHgtPm9ial9vZmZz
ZXQgPT0gMCk7CiAgIGVhc3NlcnQgKGN0eC0+ZmxhZ3MuZHVtcF9vYmplY3RfY29udGVudHMpOwot
ICBpZiAoZW1hY3Nfd3JpdGUgKGN0eC0+ZmQsIGJ1ZiwgbmJ5dGUpIDwgbmJ5dGUpCi0gICAgcmVw
b3J0X2ZpbGVfZXJyb3IgKCJDb3VsZCBub3Qgd3JpdGUgdG8gZHVtcCBmaWxlIiwgY3R4LT5kdW1w
X2ZpbGVuYW1lKTsKKyAgd2hpbGUgKGN0eC0+b2Zmc2V0ICsgbmJ5dGUgPiBjdHgtPmJ1Zl9zaXpl
KQorICAgIGR1bXBfZ3Jvd19idWZmZXIgKGN0eCk7CisgIG1lbWNweSAoKGNoYXIgKiljdHgtPmJ1
ZiArIGN0eC0+b2Zmc2V0LCBidWYsIG5ieXRlKTsKICAgY3R4LT5vZmZzZXQgKz0gbmJ5dGU7CiB9
CiAKQEAgLTgyOCw2ICs4NDAsOCBAQCBkdW1wX3RhaWxxX3BvcCAoc3RydWN0IGR1bXBfdGFpbHEg
KnRhaWxxKQogc3RhdGljIHZvaWQKIGR1bXBfc2VlayAoc3RydWN0IGR1bXBfY29udGV4dCAqY3R4
LCBkdW1wX29mZiBvZmZzZXQpCiB7CisgIGlmIChjdHgtPm1heF9vZmZzZXQgPCBjdHgtPm9mZnNl
dCkKKyAgICBjdHgtPm1heF9vZmZzZXQgPSBjdHgtPm9mZnNldDsKICAgZWFzc2VydCAoY3R4LT5v
Ympfb2Zmc2V0ID09IDApOwogICBpZiAobHNlZWsgKGN0eC0+ZmQsIG9mZnNldCwgU0VFS19TRVQp
IDwgMCkKICAgICByZXBvcnRfZmlsZV9lcnJvciAoIlNldHRpbmcgZmlsZSBwb3NpdGlvbiIsCkBA
IC00MTU5LDYgKzQxNzMsOCBAQCBERUZVTiAoImR1bXAtZW1hY3MtcG9ydGFibGUiLAogICBjdHgt
PmhlYWRlci5tYWdpY1swXSA9IGR1bXBfbWFnaWNbMF07CiAgIGR1bXBfc2VlayAoY3R4LCAwKTsK
ICAgZHVtcF93cml0ZSAoY3R4LCAmY3R4LT5oZWFkZXIsIHNpemVvZiAoY3R4LT5oZWFkZXIpKTsK
KyAgaWYgKGVtYWNzX3dyaXRlIChjdHgtPmZkLCBjdHgtPmJ1ZiwgY3R4LT5tYXhfb2Zmc2V0KSA8
IGN0eC0+bWF4X29mZnNldCkKKyAgICByZXBvcnRfZmlsZV9lcnJvciAoIkNvdWxkIG5vdCB3cml0
ZSB0byBkdW1wIGZpbGUiLCBjdHgtPmR1bXBfZmlsZW5hbWUpOwogCiAgIGR1bXBfb2ZmCiAgICAg
aGVhZGVyX2J5dGVzID0gaGVhZGVyX2VuZCAtIGhlYWRlcl9zdGFydCwKLS0gCjIuMzAuMQoK
--00000000000042cc0805bc93d141--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 2 Mar 2021 20:34:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 02 15:34:28 2021
Received: from localhost ([127.0.0.1]:54332 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lHBiu-0004Yz-I9
	for submit <at> debbugs.gnu.org; Tue, 02 Mar 2021 15:34:28 -0500
Received: from lists.gnu.org ([209.51.188.17]:54260)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pipcet@HIDDEN>) id 1lHBiq-0004Yn-Kt
 for submit <at> debbugs.gnu.org; Tue, 02 Mar 2021 15:34:27 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:42560)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <pipcet@HIDDEN>) id 1lHBiq-0006PU-Du
 for bug-gnu-emacs@HIDDEN; Tue, 02 Mar 2021 15:34:24 -0500
Received: from mail-ot1-x32f.google.com ([2607:f8b0:4864:20::32f]:37839)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <pipcet@HIDDEN>) id 1lHBil-0005Q0-Ss
 for bug-gnu-emacs@HIDDEN; Tue, 02 Mar 2021 15:34:24 -0500
Received: by mail-ot1-x32f.google.com with SMTP id g8so17697918otk.4
 for <bug-gnu-emacs@HIDDEN>; Tue, 02 Mar 2021 12:34:19 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:from:date:message-id:subject:to;
 bh=5b9IwmcF3bL0NiSEoxEpASmYQ/yzhyOmJN0qSxpQJWw=;
 b=BJfWl/Ra5v1psfnx4Oys7EkG28sHuSAeXhkjotuARlPH/u8DdiCBDefCVyCKH6C14q
 /1r0sPKDWZA7SO7qHmhqeLhwSAC/EdTwFqNU79W1C6K9d7fntEuN8280SvFJroTq+Ufh
 mKGj2fXAbRz1dqbFvHd4gsPQOtk/5sB6XBDtiqEP7FBW2KeaZEZ5X1jJ951NsMjv3+tg
 5LJEAK92qK6cgotj8G36e8X6ifD0pskYVJPMu3isMOFMK4srPigyveO9k+ro1D5h6rzx
 jDKGq0pX84AGVf5DZIuVzbQ7mndEgLilSZuw1pVd7KTOP1g37sgp/tTrZDVGr5S+x3mv
 i89Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=5b9IwmcF3bL0NiSEoxEpASmYQ/yzhyOmJN0qSxpQJWw=;
 b=N1rpogs53ZQPaSMwnYCxtSARWfMh+01gVHsaVNmf8qGAohoyk6Ze0gnGpz3Ob7rJuo
 tUDHpnkO1TduGPFqSfT1Od1JhEbX1mQX2VzFxiejAiUyXYyQbMdGQKAuHSXEwYz0aFMz
 QqEegOYYABFy56EmKZTDqEiP1pDSf2tagU5iGmuLmH0jmX+lJUvBnmD/Xy9EaJJQqbdy
 2UfpYeL5ZrP0QgdmGmebvQZLl6BLsHRu277eL+QICsBdf2pJ4Ea0Nm1fWwsJ22kX3xhp
 2IQcBLzmAuOB9YtqOijSD8ssCi4iCGtf10nLiGrXg8BR3AcgO+3V91mNQ1Zj076Sai6P
 aW0Q==
X-Gm-Message-State: AOAM531+fU3Al4eXJLePou897P2Vw3ulkOaHNNcp7I89VjpID7qDNfCV
 2qWq69Kl2o9v8BxJM7PjXwAs5xQciaZauBXxhlVpiRZAvxw=
X-Google-Smtp-Source: ABdhPJx0ANq9SgORjfvFY9p6A0UFIA+L4KtWywysakl68tnr180nyHLuZN6ottAsi2/tK8xtrFAgfiPUKjnAcH0vB40=
X-Received: by 2002:a9d:131:: with SMTP id 46mr19269769otu.287.1614717258619; 
 Tue, 02 Mar 2021 12:34:18 -0800 (PST)
MIME-Version: 1.0
From: Pip Cet <pipcet@HIDDEN>
Date: Tue, 2 Mar 2021 20:33:42 +0000
Message-ID: <CAOqdjBehCkGy+Xr4fbkVQMdMmJsEvCGqn3Kmuy00zdBbG1h2Vw@HIDDEN>
Subject: 28.0.50; pdumper dumping causes way too many syscalls
To: bug-gnu-emacs@HIDDEN
Content-Type: text/plain; charset="UTF-8"
Received-SPF: pass client-ip=2607:f8b0:4864:20::32f;
 envelope-from=pipcet@HIDDEN; helo=mail-ot1-x32f.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

Playing around with the WebAssembly port, I noticed that pdumper, in
creating the dump file, makes way too many syscalls: it uses
emacs_write(), not fwrite(), so these calls translate to actual
syscalls and context switches. On immature systems (or in special
circumstances like a device mounted synchronously), they might
actually cause a hardware write for each syscall, which would wear out
flash quickly and be generally wasteful.

I've looked into the problem, and it seems easy to solve and worth it
in terms of debuggability and performance.

Patch will be attached once this has a bug number.




Acknowledgement sent to Pip Cet <pipcet@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#46881; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Fri, 5 Mar 2021 13:15:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.