GNU bug report logs - #30626
26.0.91; Crash when traversing a `stream-of-directory-files'

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Michael Heerdegen <michael_heerdegen@HIDDEN>; dated Tue, 27 Feb 2018 09:23:01 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 4 Mar 2018 17:02:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 04 12:02:29 2018
Received: from localhost ([127.0.0.1]:44639 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1esX1p-0007kZ-It
	for submit <at> debbugs.gnu.org; Sun, 04 Mar 2018 12:02:29 -0500
Received: from eggs.gnu.org ([208.118.235.92]:33156)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1esX1n-0007kM-Vf
 for 30626 <at> debbugs.gnu.org; Sun, 04 Mar 2018 12:02:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1esX1f-0004Di-SD
 for 30626 <at> debbugs.gnu.org; Sun, 04 Mar 2018 12:02:22 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:39337)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1esX1f-0004Dc-Nq; Sun, 04 Mar 2018 12:02:19 -0500
Received: from [176.228.60.248] (port=3419 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1esX1f-00082i-3f; Sun, 04 Mar 2018 12:02:19 -0500
Date: Sun, 04 Mar 2018 19:02:03 +0200
Message-Id: <83lgf7u644.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Noam Postavsky <npostavs@HIDDEN>
In-reply-to: <87fu5f3kcr.fsf@HIDDEN> (message from Noam Postavsky on Sun,
 04 Mar 2018 10:56:36 -0500)
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN>
 <87lgfdnf86.fsf@HIDDEN> <87muzs11hk.fsf@HIDDEN>
 <87606e4lel.fsf@HIDDEN>
 <CAOj2CQQCf9TVCHC+QM0H3tgYqfGSXqnQM8r_n+xaf9xk3-Hg1g@HIDDEN>
 <87lgf83gun.fsf@HIDDEN> <87fu5f3kcr.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 30626
Cc: michael_heerdegen@HIDDEN, john.b.mastro@HIDDEN, nicolas@HIDDEN,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Noam Postavsky <npostavs@HIDDEN>
> Date: Sun, 04 Mar 2018 10:56:36 -0500
> Cc: Michael Heerdegen <michael_heerdegen@HIDDEN>,
> 	Nicolas Petton <nicolas@HIDDEN>, 30626 <at> debbugs.gnu.org
> 
> Hmm, I think it won't work by itself though, just doing
> 
>     (stream-flush (stream-range 1 1000000))
> 
> also crashes, due to the head of the stream being referenced from the C
> stack somewhere (I can get the address from gdb, but I can't figure out
> how to get to the corresponding C variable from there).

Did you try "info symbol ADDRESS"?  (I'm not sure this will work for
automatic variables, though.)

You could also try "info locals" after "set print address on" and/or
"set print symbol on".





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 4 Mar 2018 15:56:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 04 10:56:45 2018
Received: from localhost ([127.0.0.1]:44576 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1esW0D-0004El-34
	for submit <at> debbugs.gnu.org; Sun, 04 Mar 2018 10:56:45 -0500
Received: from mail-it0-f46.google.com ([209.85.214.46]:34035)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1esW0B-0004EZ-Qh
 for 30626 <at> debbugs.gnu.org; Sun, 04 Mar 2018 10:56:44 -0500
Received: by mail-it0-f46.google.com with SMTP id n128so6696261ith.1
 for <30626 <at> debbugs.gnu.org>; Sun, 04 Mar 2018 07:56:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=eyn57TeEDhnHwGD7zIvu7QkupeY0DBtiGRSMiKhHPPU=;
 b=czSEQfqZNGauRwWDRUTYpsmHXgie2hcGv+RngJFlFZgMgF9Ffj7iqzemiu6tliNMY4
 re3C5C0b/kqxL2fUKN38t5Fz4zlRARqAkJZVVeygzkelrYdzG6z9aSXRBfHLCkrHpF4M
 4Y4VTWNzfuOHta+vEsRuyYeaz4/x1/CX1qWNLL1smK8BcWuVPQS6CfFca4oxoLjOjj7q
 MjTndPe6dx/V8YoTCwYcCK7Vdc0w0wUPJ7meYCQ/SDN0TcNul03kviuZXkmHRtw9JNBw
 dnAeSUfJxVpxVu+EQ1XrF4bYWIQ1P+3yv77NlF7Jq236IRXOql6+/B0yyGof5j7nvow+
 mW1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=eyn57TeEDhnHwGD7zIvu7QkupeY0DBtiGRSMiKhHPPU=;
 b=Xqg7/cp42WIFI9k7W2tIq4G65ovTKF9/l4Fb8HKobfLy/PFC3+dIpGzblddwU1cika
 YFu4peFa1v+1WVHT1aMq2wq3xIPlybBlVnvulGAx4+o/qobSOZllez066kGMqjEqCv0J
 KbQekoeaPUG3TD0WJEjoJlv3bJrneQm7eHBzqFt5KgvY6j5n8g3ZFFr4TZYzFKvbh/J2
 YXx8+/QEbp6Tlb89A20wliZLR76k5mLt6QG+YT1PamJfzm799ALPSxNhbzO/2umDmFds
 GaWpckq3VIOra0S0TS08nyo/mfHt0hi1qDEO+UlOPUqloOgJ12sOifndf1uHtaJHnHL2
 2EDQ==
X-Gm-Message-State: AElRT7GlXAtMfyht2n9hYxCOl4rkxvqqvVnLUso28yavMkq9c3L5fSoc
 PIC/5fwKZM1kYiRfieaB8w8Q4Q==
X-Google-Smtp-Source: AG47ELspReNFHR/AoJ8KOvTds/Ho+M/xnlSWvCk+tYln4F8erP6qI8bd65mOrh7oiGDXyKB8FIurxg==
X-Received: by 10.36.121.195 with SMTP id z186mr10477649itc.148.1520178998116; 
 Sun, 04 Mar 2018 07:56:38 -0800 (PST)
Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34])
 by smtp.googlemail.com with ESMTPSA id
 h63sm3803821ioa.81.2018.03.04.07.56.37
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Sun, 04 Mar 2018 07:56:37 -0800 (PST)
From: Noam Postavsky <npostavs@HIDDEN>
To: John Mastro <john.b.mastro@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN> <837eqxyqoe.fsf@HIDDEN>
 <87lgfdnf86.fsf@HIDDEN> <87muzs11hk.fsf@HIDDEN> <87606e4lel.fsf@HIDDEN>
 <CAOj2CQQCf9TVCHC+QM0H3tgYqfGSXqnQM8r_n+xaf9xk3-Hg1g@HIDDEN>
 <87lgf83gun.fsf@HIDDEN>
Date: Sun, 04 Mar 2018 10:56:36 -0500
In-Reply-To: <87lgf83gun.fsf@HIDDEN> (Noam Postavsky's message of "Sat, 03
 Mar 2018 18:00:00 -0500")
Message-ID: <87fu5f3kcr.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: Michael Heerdegen <michael_heerdegen@HIDDEN>,
 Nicolas Petton <nicolas@HIDDEN>, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

Noam Postavsky <npostavs@HIDDEN> writes:

> John Mastro <john.b.mastro@HIDDEN> writes:
>
>> (let* ((stream0 (stream-range 1 1000000))
>>        (stream stream0))
>>   (setq stream0 nil) ;; <<< Inserted by compiler
>>   (while (not (stream-empty-p stream))
>>     (cl-callf stream-rest stream)))
>>
>> If the code does reference stream0 later, locals clearing can't help
>> you, but that's considered a "if it hurts, don't do it" situation.
>>
>> This probably isn't practical for Emacs, especially since it could only
>> work for byte-compiled code, but thought the prior art may be of
>> interest.
>
> Not sure how doable this solution is, but the fact that it works only
> for byte-compiled code seems fine to me.  The interpreted case is doomed
> to fail anyway since the interpreter doesn't prune redundant variables
> from closures.

Hmm, I think it won't work by itself though, just doing

    (stream-flush (stream-range 1 1000000))

also crashes, due to the head of the stream being referenced from the C
stack somewhere (I can get the address from gdb, but I can't figure out
how to get to the corresponding C variable from there).




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 3 Mar 2018 23:00:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 03 18:00:10 2018
Received: from localhost ([127.0.0.1]:43301 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1esG8P-00088f-Uw
	for submit <at> debbugs.gnu.org; Sat, 03 Mar 2018 18:00:10 -0500
Received: from mail-io0-f178.google.com ([209.85.223.178]:37514)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1esG8N-00087K-NX
 for 30626 <at> debbugs.gnu.org; Sat, 03 Mar 2018 18:00:08 -0500
Received: by mail-io0-f178.google.com with SMTP id d71so14256626iog.4
 for <30626 <at> debbugs.gnu.org>; Sat, 03 Mar 2018 15:00:07 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=DUfm0OkrVRdeSHZoneFuVeZ5D0A9J3+UdOZKfnY5gpc=;
 b=sv566DU+GR4oXhDGdSoEJctKszSECUMmKKBw46a9bitk2rFD6vC2dvGuBE4eKekhbb
 zhDxsM86CeZI4uIQwem3DUAZ99rwn4kKHS09y9j5R8uCi4mGJpiXFBzggpZ0X/4mhRT+
 UptKcDIz0Q2qKcM613F8U0cApIqAEX3dVXOQZjC80jQzJaaUvfqXQe4vwjjwy7x9Kqjn
 K4CTYtyCmK/QpEdUhGyrCDrB25WJ2AZrGM96nWPO8jWrmiJNA66pI0k8m72HafC4eUFY
 1Unn/zM6y5JGF9TJZuIaN8RdGyHOVZrKI8A0FrkiPQ7L9kMkY/vsZS5HtctiWkL8pHJk
 Xkbw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=DUfm0OkrVRdeSHZoneFuVeZ5D0A9J3+UdOZKfnY5gpc=;
 b=Su0nFojlQF1d7aH9fYMKuBHXbDVdwiG2R2Cq7hkmTH/udNtkSXIqnfORvU70dl0h5R
 Xys2dnQ2vwqwHEvNQCCCHJRcMb6e7eZJ4lDaOQw1f15P+G70MQ6RqPJVY7+2/Wn8XC+y
 n3CUdOxpKRAjO5J/EP5ve01jVaQnRVYRSW7outh6BY+dZHztcXp8cjNCWfm5dOVd4ZvW
 DyHJK/o7SQI3iO0b5ZdLN8qtmDFyHE23Fonmg4zcLmh78wu66fEyEZbQOY1EmXj1JN3l
 AyBSDy2/c07/zYp+bhwAk2gy/Geb0CmtfBJrUgSqcfsAtXIYuye2J2I9KH20jocxyybX
 VTuw==
X-Gm-Message-State: APf1xPAH4kj3/oGh615aTf8mB5u9rEENp4cUrAnqtKLfl9mgE1gMREeE
 8jyEADpU98PUPQALK/mu9+M=
X-Google-Smtp-Source: AG47ELuMBJP1D2ev1ea4P+3rH/O/txVTJ9XeDRE4iffYGcwTOVM/7yY2vags7yTWB4EXZEBjajTlKw==
X-Received: by 10.107.89.13 with SMTP id n13mr12302420iob.154.1520118002193;
 Sat, 03 Mar 2018 15:00:02 -0800 (PST)
Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34])
 by smtp.googlemail.com with ESMTPSA id
 33sm5973960ioj.71.2018.03.03.15.00.01
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Sat, 03 Mar 2018 15:00:01 -0800 (PST)
From: Noam Postavsky <npostavs@HIDDEN>
To: John Mastro <john.b.mastro@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN> <837eqxyqoe.fsf@HIDDEN>
 <87lgfdnf86.fsf@HIDDEN> <87muzs11hk.fsf@HIDDEN> <87606e4lel.fsf@HIDDEN>
 <CAOj2CQQCf9TVCHC+QM0H3tgYqfGSXqnQM8r_n+xaf9xk3-Hg1g@HIDDEN>
Date: Sat, 03 Mar 2018 18:00:00 -0500
In-Reply-To: <CAOj2CQQCf9TVCHC+QM0H3tgYqfGSXqnQM8r_n+xaf9xk3-Hg1g@HIDDEN> (John
 Mastro's message of "Fri, 2 Mar 2018 13:48:23 -0800")
Message-ID: <87lgf83gun.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: Michael Heerdegen <michael_heerdegen@HIDDEN>,
 Nicolas Petton <nicolas@HIDDEN>, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

John Mastro <john.b.mastro@HIDDEN> writes:

> (let* ((stream0 (stream-range 1 1000000))
>        (stream stream0))
>   (setq stream0 nil) ;; <<< Inserted by compiler
>   (while (not (stream-empty-p stream))
>     (cl-callf stream-rest stream)))
>
> If the code does reference stream0 later, locals clearing can't help
> you, but that's considered a "if it hurts, don't do it" situation.
>
> This probably isn't practical for Emacs, especially since it could only
> work for byte-compiled code, but thought the prior art may be of
> interest.

Not sure how doable this solution is, but the fact that it works only
for byte-compiled code seems fine to me.  The interpreted case is doomed
to fail anyway since the interpreter doesn't prune redundant variables
from closures.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 3 Mar 2018 08:47:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 03 03:47:53 2018
Received: from localhost ([127.0.0.1]:41812 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1es2pc-0001lC-Ti
	for submit <at> debbugs.gnu.org; Sat, 03 Mar 2018 03:47:53 -0500
Received: from petton.fr ([89.234.186.68]:36072)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <nicolas@HIDDEN>) id 1es2pa-0001kv-LV
 for 30626 <at> debbugs.gnu.org; Sat, 03 Mar 2018 03:47:51 -0500
From: Nicolas Petton <nicolas@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
In-Reply-To: <87k1utmw6d.fsf@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN> <87lgfd52by.fsf@HIDDEN>
 <87bmg91ity.fsf@HIDDEN> <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN> <87muzs11hk.fsf@HIDDEN>
 <87606e4lel.fsf@HIDDEN> <87vaeefd23.fsf@HIDDEN> <87k1utmw6d.fsf@HIDDEN>
Date: Sat, 03 Mar 2018 09:47:41 +0100
Message-ID: <87po4lfsuq.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=petton.fr; s=mail;
 t=1520066864; bh=61xDQ6GPfIPUCdgRMLK9molyPFF4tGvL/Dm5Q/1YpYM=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID:MIME-Version:Content-Type;
 b=KgugRkGp0NJEWT+iLNGEYp8jZOoonUA4ZJtvExGc9je7hO6sI9yVrlk/iq0KgQF5/2fALLpMSu4XuYewSfUjfuFchZSr9SvOzrU+LDhE0PzGuZUV6wGC3DrpKYauYg277Bfl/o4Bbh/cZ0xhsnQUDYx/pSP56Ve0uB7xdRx8rnI=
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: Noam Postavsky <npostavs@HIDDEN>, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

--=-=-=
Content-Type: text/plain

Michael Heerdegen <michael_heerdegen@HIDDEN> writes:

> Please don't do this.  The semantics of streams is worth keeping.  I
> make use of it in el-search and other stuff.

I'm not saying that I want this :)

Nico

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE1AWqLIYsVPF+7mvg6LzXhmr8+XgFAlqaYS0ACgkQ6LzXhmr8
+Xhh+wgAij2rvDtqdsxRxlgZCl+uCLAYzoJdMwkAbJPhm7ygxgULP4uBgQlXNIA3
2Ozp4IAWkcQhXPjnoGDcmoLvYyy4Wh08dTWyBpdkyI0mzdasf2LZqxVUkl0pMh0f
mEbTcnSg4pMlWK28ZRHtKv6jx2oA3YWf4iBCNoYdaty1yqvtlHdxQJWY4rWsNYLq
7zHgTYH1vdSolGoWlSHbNJb8hEf5SaJQGsb39Z5ycmUABEMjdjKWILFXGcEGMCPz
cmmKoTphsVo2sCAW/3VbiS7asvL69sAHw/k/2q6Lbs6nLnBxGwVacS5V8frXU8Gf
mp3SUGWOmmNt1lWfXUcg+EveTnCg3Q==
=6t13
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 3 Mar 2018 07:56:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 03 02:56:15 2018
Received: from localhost ([127.0.0.1]:41777 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1es21f-0000QD-MC
	for submit <at> debbugs.gnu.org; Sat, 03 Mar 2018 02:56:15 -0500
Received: from mout.web.de ([212.227.15.14]:35665)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1es21d-0000Pz-Sx
 for 30626 <at> debbugs.gnu.org; Sat, 03 Mar 2018 02:56:14 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb002
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LpOKb-1eFvNY40ub-00fCeT; Sat, 03
 Mar 2018 08:56:06 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Nicolas Petton <nicolas@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN>
 <87muzs11hk.fsf@HIDDEN> <87606e4lel.fsf@HIDDEN>
 <87vaeefd23.fsf@HIDDEN> <87sh9ifb3w.fsf@HIDDEN>
Date: Sat, 03 Mar 2018 08:56:05 +0100
In-Reply-To: <87sh9ifb3w.fsf@HIDDEN> (Nicolas Petton's message of "Fri, 02
 Mar 2018 21:58:43 +0100")
Message-ID: <87fu5hmw2y.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:Rzp4CMhQkfV47snGTbmF6fcPxYF8dM/OBBaCT/DYZVjWlNyfkKv
 I/NUzewDcnYh1WPat/vN7bSMueL2ogfShPzMBA8sbQT9Uak8qktNJdxJVQhZnUQDBqwFDw2
 QQk1b/kVclNmY73I9T/o45uky2Q6EhBMI7o546efwZdWeliW76XgBCPJsLLBjhiRbW/dt6/
 +pU7cNdRt3aXeSNS+x4Ww==
X-UI-Out-Filterresults: notjunk:1;V01:K0:+wcrWG0NSK0=:YgV8zoWeZVlKA6MEWfo+vc
 ImZDFSGueBliJbSa38VLdEaVlR2GRjJV+1GqTeg5WCCsk0sviYS+tTMgPDB/aasqkEYvprIWR
 AnUeLGhhiHfHN3chraQMoMODDgz2IOJPsw0KoEj7FOShUMC1YBum4UvZoHOImEBPWyZ84NM1y
 ZFpFh6JQs7guDDxf9Y8MFYBS4kp17QQEKMyQ0KzGNYWdZIjIEhf//WHwLZDpdfi3WelBoEI8e
 fAWz2ODibeVBTOKmFjffA6sFBk29w2QBa6lKDG0UnKaxmZHTbbhVdwiUMkE+CIUL5kAbziJEq
 i8b6y1RgIR+nNP3NAAL0p0r0oOY2nF8j/cHb61tH/6RdWA8/UGWO5blvIeZDdzp5Cf82tBwKO
 LgmBoeyQqgGccyilEgt9ZI4UcNZJnB1iNimIqBF3H6Q0zWqngXc1bg9wI/fRq9mRwBBiQ7zJN
 +rdrslWDmrn0SkJdG62NbDv02/pV/EkjMU0fzZc9DJDM/g+oBeytWJ9m/VebJxfpeKqFPsiqJ
 YBg8WjhtwsANH6m4Fkl3J8Ir7rOmutVOWaqmq1AkA3KhB1ICTN8GdsqmEDb7u5ZaClQ7bOyQM
 V56M0bH530Q3VrgwwrkW6OzPON/+T0Lc9dcVEtmoHF6KDDFpnWsAG1pkqRxSjiHsOk640SpO5
 Uuf6MYSiTns0A8uXrKTDNY1SWgLXiv9WY7LNFQ9rZ4VS22mOhy40Rbt1Z5gQ6hJMzN3U+c/Al
 JsjbDYikGmiknv1lVYyKEuyIQ8HsZUy8CgZolHqbQ1hSZg75oWaK4SWzT4axloSkXQ0vIVXIi
 rHBuPPqGs9+92exFB9IMeapdA1iUv//bOOO8dkP1n+At1kG+y4=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: Noam Postavsky <npostavs@HIDDEN>, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Nicolas Petton <nicolas@HIDDEN> writes:

> I had something like the following in mind:
>
>   (cl-defstruct nstream current next-function)
>   
>   (cl-defmethod nstream-next ((stream nstream))
>     (setf (nstream-current stream) (funcall (nstream-next-function stream)
>                                             (nstream-current stream))))
>   
>   (defun nstream-range (&optional start end step)
>     (unless start (setq start 0))
>     (unless step (setq step 1))
>     (make-nstream :current start
>                   :next-function (lambda (cur)
>                            (if (equal cur end)
>                                nil
>                              (+ cur step)))))

This is an implementation of iterators, not streams.  We already have an
implementation of iterators in Emacs.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 3 Mar 2018 07:54:13 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 03 02:54:13 2018
Received: from localhost ([127.0.0.1]:41773 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1es1zh-0000NK-9M
	for submit <at> debbugs.gnu.org; Sat, 03 Mar 2018 02:54:13 -0500
Received: from mout.web.de ([212.227.17.11]:44135)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1es1zf-0000N8-UN
 for 30626 <at> debbugs.gnu.org; Sat, 03 Mar 2018 02:54:12 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb101
 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MK1s1-1etP2M1zwD-001VNg; Sat, 03
 Mar 2018 08:54:03 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Nicolas Petton <nicolas@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN>
 <87muzs11hk.fsf@HIDDEN> <87606e4lel.fsf@HIDDEN>
 <87vaeefd23.fsf@HIDDEN>
Date: Sat, 03 Mar 2018 08:54:02 +0100
In-Reply-To: <87vaeefd23.fsf@HIDDEN> (Nicolas Petton's message of "Fri, 02
 Mar 2018 21:16:36 +0100")
Message-ID: <87k1utmw6d.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:DIMhMu9sNbyoXsZAiYoAAM2He6/zd4KOtTWvC1mYRmZgvmiEpqX
 UuPJlFkfFTuQ0OxbftkBQajNoT4SL8o1O+kNqeZxcmAewz4dTZd9PZo7Vx9yWWnQSI0pUDj
 mxOnBWM0bAryGrGryyGPJSnWWmnHnzBqQiqtHX6RUR03bf1nE3kWFGfkdnlGhiPYnnF22ha
 wxjIXmhnDN/C5AdyPnKRQ==
X-UI-Out-Filterresults: notjunk:1;V01:K0:GY/Jl4bcwaY=:C0KRnQXqd/DIMvXwe86/En
 BRrgFVsYVJ8ee4yRJMOdHZxJbjmucnmsZOukIkjCQZ+pxXdohkyENbASMBYAHT3kCKpDxKhyU
 oeBuNeOKn6I7pKMrepQdhF61pL2e3FkVo1xFcfvcMgwbpBmGHH3WmsazC4NljFiXcFHZy4eoo
 FHHRtoO8yg/uXyZG0yqdZwBTE3dI0Ww1JTk/Fbi5V5iUL554sDsok2wRyAkRrI/NnmPkAw8PH
 t/ec/O2O/BG4ms29XQ0Ba2JYVEE1j+MThzOp7plItZauPWqOVCWHRt4AgmPCgwD39CtquTG79
 0dKg4UFobxn9Ex/Mn5BLlEKXMFPv0nTaiU77NM1s2PQjJ6tPrBZDScgaqXswHTi1lZD+/ObfH
 sFVWHpxUtFBnmOLZ4TmsoRx8WsBdJpOTGgOKiqRMbL3AW/1SFhA4Px9Nu+05vQqrsL6JRY2Hu
 aiYXfOPuZYTKZ5/UMvL+iUq7W3HhNsAwpmlbB5LiXtSAY8bUja4d+Uc+1x4F8cRM4w4yQ73lF
 5zAMbIUs3h+CP1b2cLciKL8GP4WolkOMA4F1//Np8UH0cO1s0TXfaUrrukL+EiEDeCiV2FTeC
 Pi1ZU4uM6F1vmgV6MMVwXbT1vkyUafsYn3ZqKi+RfO1mtmmyyETx0CJ3PAgWnSiFPgmj1+hri
 0EvX9h3nURhfXCXKqP15T5TL2RmPyoE1e5EQD/cMLWSLwVADWy/u6UOpbeA371JDc/a360tA0
 nv5xF+ZPhOiTVw/9XO0/eBWg8euarHrdzi0O2ODhd2rwc6O4b1HIvtE6mx3yMUNaCnqWfPbyi
 Kt0FQ2F7HsBoQYQR0rAQt3rrOw2dc/jHQ2l0m5tvKq5OZhfg0Y=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: Noam Postavsky <npostavs@HIDDEN>, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Nicolas Petton <nicolas@HIDDEN> writes:

> stream.el is at its core just an implementation of lazy-cons cells, so
> not letting stream0 get access to the result would mean changing the
> core implementation (I'm not necessarily against it).

Please don't do this.  The semantics of streams is worth keeping.  I
make use of it in el-search and other stuff.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 21:48:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 16:48:51 2018
Received: from localhost ([127.0.0.1]:41569 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ersXr-0003gV-Iu
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 16:48:51 -0500
Received: from mail-qk0-f175.google.com ([209.85.220.175]:45622)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <john.b.mastro@HIDDEN>) id 1ersXq-0003gH-BP
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 16:48:50 -0500
Received: by mail-qk0-f175.google.com with SMTP id g2so13768810qkd.12
 for <30626 <at> debbugs.gnu.org>; Fri, 02 Mar 2018 13:48:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:in-reply-to:references:from:date:message-id:subject:to
 :cc; bh=EFpfmREqGYP06GiSZ4rWHh17p3ivcCfe3pTRKT5DfBI=;
 b=ZzL5yBjDsDgJ34syzAgv56Y/2OPhI2rLLHcniPjlbldqJRXOJjF7zynNRm0ywP2AyG
 6VcU0tYzN7Lvw5N8lq75JSGO34zRQN7SdlX8MtL5C7kfW1Inq5qh5WPRMjxd5IWhuT1K
 TSlJ+fBcIDUbIc9pcX7/s8hisxqigM7PmQIfK+2hsAK6yIb+DiXReG6QjHCuJwiKoVNp
 8uY/5S4KBCb9t+l0CqO/vhAC52/HAxQiayG9YDMQZc7SObRD8FVCdKepE6ZGcOgcDHmB
 cmqucT/LlbMe2RHN7N6jEwEB/AsHU82RJh/CtGZqHe4cvwgNlgkobA2uuex5DR7vuHuV
 PqBA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:in-reply-to:references:from:date
 :message-id:subject:to:cc;
 bh=EFpfmREqGYP06GiSZ4rWHh17p3ivcCfe3pTRKT5DfBI=;
 b=E49jvqKaS+VdmW08M4dNOAyw1mRHJzC/g5xqdrKE1P3Bx4KhSmMkDMK0lrVtdQqSHy
 tdjPFCYncn4LmYMxtSCfVbF7l3DRZoI/mYXIhxLGzjrI0uV9sE0TTlkZTyhHxRDzn7RO
 x6/454i1I0Ua17VZuU+D10SZ8egMu2ziKthUQBAR26CNe+PZtdJhE/eztZOIU7ELg9pR
 K66X5BQnVeoZzk2+NDDiuvu/ZE6X2/Uwzua016l7gRHIKTgw/GbWe7TvfYcmmLnFxW6f
 Ktq+x+ULY0Y2W71qIBVcXijrcEUiDlkysPEhlIAK1OKzNwcSNxw4nc2GnkaDBkMD8n1m
 Gr5w==
X-Gm-Message-State: AElRT7E7fvZUH8I+88KfisdWcVo3xhjqEZ4Gk5XSDoJm2U2I88U0OoD1
 UMAnvK77QCvI5Fc0HUS+dD9a1djZo8eJlU6WxO8KSloc
X-Google-Smtp-Source: AG47ELvC+LL7nH/Q27qTI11GnXmLnSZm1YnevYgCXL4Px/Qwec6/Eq46vGFh7o9FPLJeU6DUXC+1D8kYBKg0/5ZSEeE=
X-Received: by 10.55.78.212 with SMTP id c203mr10596248qkb.351.1520027324330; 
 Fri, 02 Mar 2018 13:48:44 -0800 (PST)
MIME-Version: 1.0
Received: by 10.200.52.16 with HTTP; Fri, 2 Mar 2018 13:48:23 -0800 (PST)
In-Reply-To: <87606e4lel.fsf@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN>
 <87fu5moe4c.fsf@HIDDEN> <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN> <83h8q1yuin.fsf@HIDDEN>
 <87po4pnl0a.fsf@HIDDEN> <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN>
 <87muzs11hk.fsf@HIDDEN> <87606e4lel.fsf@HIDDEN>
From: John Mastro <john.b.mastro@HIDDEN>
Date: Fri, 2 Mar 2018 13:48:23 -0800
Message-ID: <CAOj2CQQCf9TVCHC+QM0H3tgYqfGSXqnQM8r_n+xaf9xk3-Hg1g@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
To: 30626 <at> debbugs.gnu.org
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: Michael Heerdegen <michael_heerdegen@HIDDEN>,
 Nicolas Petton <nicolas@HIDDEN>, Noam Postavsky <npostavs@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

Noam Postavsky <npostavs@HIDDEN> wrote:
> Aha, but the following also crashes, whether compiled or not:
>
> ;; -*- lexical-binding: t -*-
>
> (require 'stream)
>
> (let* ((stream0 (stream-range 1 1000000))
>        (stream stream0))
>   (while (not (stream-empty-p stream))
>     (cl-callf stream-rest stream)))
>
> So the problem is that the initial stream0 object can reach the entire
> unfolding stream as it goes, and just holding on to that reference is
> enough to keep the whole thing in memory.
>
> Now, I can see that letting stream0 automagically get access to the
> unfolded result can be an optimization in some cases, although in this
> case it's a pessimization.  It could also affect the semantics if
> unfolding the stream has side effects, not sure if stream.el makes
> guarantees about that though.

Clojure has/had a similar issue. They describe this scenario (having a
live reference to the beginning of the stream, preventing GC from
collecting it) as "holding onto the head" of the stream (in Clojure
they're called lazy seqs).

Their solution was what they call "locals clearing". The compiler tracks
the lifetimes of local bindings and "clears" them (by setting them to
nil/null) after their point of last use, e.g.:

(let* ((stream0 (stream-range 1 1000000))
       (stream stream0))
  (setq stream0 nil) ;; <<< Inserted by compiler
  (while (not (stream-empty-p stream))
    (cl-callf stream-rest stream)))

If the code does reference stream0 later, locals clearing can't help
you, but that's considered a "if it hurts, don't do it" situation.

This probably isn't practical for Emacs, especially since it could only
work for byte-compiled code, but thought the prior art may be of
interest.

        John




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 20:58:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 15:58:56 2018
Received: from localhost ([127.0.0.1]:41563 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1errlY-0002Yn-LT
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 15:58:56 -0500
Received: from petton.fr ([89.234.186.68]:55950)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <nicolas@HIDDEN>) id 1errlW-0002Ya-R1
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 15:58:55 -0500
From: Nicolas Petton <nicolas@HIDDEN>
To: Noam Postavsky <npostavs@HIDDEN>,
 Michael Heerdegen <michael_heerdegen@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
In-Reply-To: <87vaeefd23.fsf@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN> <87lgfd52by.fsf@HIDDEN>
 <87bmg91ity.fsf@HIDDEN> <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN> <87muzs11hk.fsf@HIDDEN>
 <87606e4lel.fsf@HIDDEN> <87vaeefd23.fsf@HIDDEN>
Date: Fri, 02 Mar 2018 21:58:43 +0100
Message-ID: <87sh9ifb3w.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=petton.fr; s=mail;
 t=1520024328; bh=pCuSUfM/JR20t69Ko1sc5/gGVnBXN+Zut5DaPsxviBA=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID:MIME-Version:Content-Type;
 b=Hta5hPEjUqpNcrFoB4oYRWNJAiTkuffh45Mu9zcdc8txjqSrAAjaJg9O3osKm3hor128mWpUcH6VBpZwqzHAEBB/uqYTZXbF2xkeNrdrUjRTUnqHSk/+rgacvhVrxhTOfkNxJF2mH51AvHL8oZbTGzsei3bLFhX9iA2yUItOBhE=
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

--=-=-=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

Nicolas Petton <nicolas@HIDDEN> writes:

> If we accept that `seq-elt', and other positional functions of seq.el
> should not work on streams, then I could rewrite stream.el to make it a
> positioned stream where previous elements are discarded after each
> element generation.  However the list of supported functions from seq.el
> API would be significantly reduced.

I had something like the following in mind:

  (cl-defstruct nstream current next-function)
=20=20
  (cl-defmethod nstream-next ((stream nstream))
    (setf (nstream-current stream) (funcall (nstream-next-function stream)
                                            (nstream-current stream))))
=20=20
  (defun nstream-range (&optional start end step)
    (unless start (setq start 0))
    (unless step (setq step 1))
    (make-nstream :current start
                  :next-function (lambda (cur)
                           (if (equal cur end)
                               nil
                             (+ cur step)))))

Cheers,
Nico

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE1AWqLIYsVPF+7mvg6LzXhmr8+XgFAlqZuwMACgkQ6LzXhmr8
+Xh6vwf9GP46u+qqF2r5QY4/X4PK6TDmpvRnKBtsJ6g1Mw1x6E3ZXwrRSz10v9sK
Dn9sexvOxJb8gqK6B5jqgbAQP0H0qGkW66liRR/Hls5It3rKZ9yHAGLFEEKaMMUp
90mfKFdSQzpyaeZLoBWIAVuoO30mcxq1DgpoJN+yJ3RkYgBz1Vbp4+eS3UI9OU2+
zzYA7SXeKBU64hzfjwrK0oF89cPvz4kyeRLSLkynFCEX/oRMiQyVWy1BSegAxXHI
3oVtpsqMc703rcioWdyla/JkulVSlhlmRVwPb2Nwf6ufy9IzEz9WhIKoBZAwWNlp
Zzn0DJWfH1khIwBjr25i48YtuOG8RQ==
=DErk
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 20:16:50 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 15:16:50 2018
Received: from localhost ([127.0.0.1]:41558 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1err6o-0001b4-CX
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 15:16:50 -0500
Received: from petton.fr ([89.234.186.68]:45614)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <nicolas@HIDDEN>) id 1err6m-0001ap-Qr
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 15:16:49 -0500
From: Nicolas Petton <nicolas@HIDDEN>
To: Noam Postavsky <npostavs@HIDDEN>,
 Michael Heerdegen <michael_heerdegen@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
In-Reply-To: <87606e4lel.fsf@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN> <87lgfd52by.fsf@HIDDEN>
 <87bmg91ity.fsf@HIDDEN> <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN> <87muzs11hk.fsf@HIDDEN>
 <87606e4lel.fsf@HIDDEN>
Date: Fri, 02 Mar 2018 21:16:36 +0100
Message-ID: <87vaeefd23.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=petton.fr; s=mail;
 t=1520021801; bh=chxFAIhvtLFP0jp1QV2a47jvta88xwGqiavG0MfX0PI=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID:MIME-Version:Content-Type;
 b=bJCNNn6mvu6fkPS7ilvQ5tr9VFnMDlXt4oEqvFD3O1p5PYL/ETSAZgCzuL50ptBNP3+JsFk9OMdUUga6t4mhMhQDQld/ewo6ywCYef7C78A3Al2y6MV8620Z+GCTr6iggPXo2QGDKXkXCw6pbPBvTxSHm3d/erJ9QD34hyxUaok=
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

--=-=-=
Content-Type: text/plain

Noam Postavsky <npostavs@HIDDEN> writes:

> Now, I can see that letting stream0 automagically get access to the
> unfolded result can be an optimization in some cases, although in this
> case it's a pessimization.

stream.el is at its core just an implementation of lazy-cons cells, so
not letting stream0 get access to the result would mean changing the
core implementation (I'm not necessarily against it).

If we accept that `seq-elt', and other positional functions of seq.el
should not work on streams, then I could rewrite stream.el to make it a
positioned stream where previous elements are discarded after each
element generation.  However the list of supported functions from seq.el
API would be significantly reduced.

> It could also affect the semantics if unfolding the stream has side
> effects, not sure if stream.el makes guarantees about that though.

No, it doesn't.

Nico

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE1AWqLIYsVPF+7mvg6LzXhmr8+XgFAlqZsSQACgkQ6LzXhmr8
+XjObQf/bNEvOwmzGP7PM1qqjrIJcwBmzYi1s2FdhKp8TVjfOdtsc2Ovt+yBBnm+
7OT9F1v9H6jyiQyzakcKi8O/TdtleZmmJQqGeGoWDOfJcQ0Ijwbtb7BwA7218bdT
9epzevOcmdc7XxQtQpf79SV+KSQq7OiHbAO4i2tBTU3h8QTtZzopXAfNVwpXaQsL
BZBi4y36jpOZcHT18S81hAp9ADJRksnhAXkWSRyXBjiXzcigdB3Jf5NrSBNV/SEr
F/uBHzD8RHG4rCHFVGzbv4CtDetgRwVRfGUEkl1IbHJyV9Ot3oMCY0GQB3mcGoOT
fRf13nEhfGqlVFR1+JE6VydyVIDkQg==
=r4Sx
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 15:43:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 10:43:24 2018
Received: from localhost ([127.0.0.1]:41435 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ermqC-0003qD-Hy
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 10:43:24 -0500
Received: from eggs.gnu.org ([208.118.235.92]:42103)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1ermqA-0003pz-NB
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 10:43:23 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1ermq2-0005CB-I2
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 10:43:17 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59173)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1ermq2-0005C7-Eh; Fri, 02 Mar 2018 10:43:14 -0500
Received: from [176.228.60.248] (port=3460 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1ermq1-0007GF-Qo; Fri, 02 Mar 2018 10:43:14 -0500
Date: Fri, 02 Mar 2018 17:43:27 +0200
Message-Id: <837equwkio.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
In-reply-to: <87o9k6ms8m.fsf@HIDDEN> (message from Michael Heerdegen on Fri,
 02 Mar 2018 16:06:49 +0100)
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN>
 <87muzs11hk.fsf@HIDDEN> <87606e4lel.fsf@HIDDEN> <87o9k6ms8m.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 30626
Cc: nicolas@HIDDEN, npostavs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
> Date: Fri, 02 Mar 2018 16:06:49 +0100
> Cc: Nicolas Petton <nicolas@HIDDEN>, 30626 <at> debbugs.gnu.org
> 
> > (let* ((stream0 (stream-range 1 1000000))
> >        (stream stream0))
> >   (while (not (stream-empty-p stream))
> >     (cl-callf stream-rest stream)))
> 
> I guess you should have some awe if you write something like
> (stream-range 1 1000000) and keep in mind what happens with this
> gigantic thing.  Though, could the object `stream0' references not
> already be garbage-collected when the loop has been entered, since there
> is no lexical reference to that variable there?

It's still referenced by the let* form, I think.

But even if it didn't, you cannot rely on it being GC'ed, because
Emacs triggers GC at times which are hard or even impossible to
predict.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 15:07:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 10:07:09 2018
Received: from localhost ([127.0.0.1]:41391 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ermH7-0002yQ-IY
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 10:07:09 -0500
Received: from mout.web.de ([212.227.15.14]:50735)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1ermH6-0002y9-B9
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 10:07:08 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb003
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0MSJH1-1fG8v31vEh-00TYNT; Fri, 02
 Mar 2018 16:06:50 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Noam Postavsky <npostavs@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN>
 <87muzs11hk.fsf@HIDDEN> <87606e4lel.fsf@HIDDEN>
Date: Fri, 02 Mar 2018 16:06:49 +0100
In-Reply-To: <87606e4lel.fsf@HIDDEN> (Noam Postavsky's message of "Fri, 02
 Mar 2018 09:11:46 -0500")
Message-ID: <87o9k6ms8m.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:mXgsQUGMor4X7g8Xtu6LZ2Cw+FVkMbVGg9PG2Iri+X+D5M41h1m
 9Nud3A61rEX7eBV6V5aMTpt0gqoohzAGm2rilx373sBoJaLb4S75CGctxoCsa5E3G0j1wNL
 2dBJ7iomA2g9tKGplUy/y34b5B+Db6qEmP+xjzxLTAakyEw94Ri3z11jw6uINPqgtGsLggU
 ZZ61sT8Ds0wgKNR/TU4Ow==
X-UI-Out-Filterresults: notjunk:1;V01:K0:dEX4lT+N6XI=:zwQt9nETGEoCLYKFrDovrn
 nGoUVbeXgdJBsaNlGJokjlh9KZ2BRKgLdiUko9/mq4ezTNsbXJrfBr/KKgfeRKiNjmnGV3VBi
 seg8QbISjShgqbqj3Gxb7HfEJgzu1MrCBAWsoyILsDSEzRpVfyeQobbda6+jizwFSI4OetqwN
 1G+tFXCb6Vy9VvFgKjqK7+7gtOScxdswizVcVOXN689I5v4/D+xTHxdTbfsuFNkhpyc+o6aQb
 i/AhACfH0cPfn6pbDYiTYUUoboDbOEMd/xRgmETrQlJFGBuY8Hxh9OoqGpShtEn7F8Mg39jW8
 drNIplzncK+oMQNYMvuCw+yP4+3/WlNvIJhyoRrwZEM7Wk7rartsgZWvG5nsqTABVjob0pj1z
 VC9AK/aINdk+inVavLdfinGuDJjHYYnUVIqYlCyQ3u2DgKu1mxCd0/36dfW8DrFctkr4O34+0
 WYLZgv5xwr4nONLoGXd4zouSXzcdtxd4Ay8H8DRyU1tcU7/4lb9rLxNr3HxF/qU19bv1cKrDp
 11BHDvmv2JiHo/LnhiLWHSRkc/AvDFdG23qa4DeSDwlIGuSHtHUuVaMuGyj3tOOP5qk5UQ/eX
 f+1NjSxHNL3CaXoyekDWL5q8/ZPHuXxTlyjsyPcD7ra21+AXZUdZnn9y88f1AUtptiQbFQW7q
 2dXYENcSqJWJ/V6K/89UhTvyQYr4WLZSVilx2mleC+n3i5nrySCgazAYpzixWTL0DOZir/TPO
 083UDFdZNiL/azbbqj95/JqRun3vslsiTCWc/ZQqbB6GNSyTGF2B+n78MnIzHh+4rUCUe/y4a
 pRCWAuueOdxNe9PjsjFtaW6t2K/9xE+feFADHbzBLWXlITPlfY=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: Nicolas Petton <nicolas@HIDDEN>, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Noam Postavsky <npostavs@HIDDEN> writes:

> Aha, but the following also crashes, whether compiled or not:
>
> ;; -*- lexical-binding: t -*-
>
> (require 'stream)
>
> (let* ((stream0 (stream-range 1 1000000))
>        (stream stream0))
>   (while (not (stream-empty-p stream))
>     (cl-callf stream-rest stream)))

I guess you should have some awe if you write something like
(stream-range 1 1000000) and keep in mind what happens with this
gigantic thing.  Though, could the object `stream0' references not
already be garbage-collected when the loop has been entered, since there
is no lexical reference to that variable there?

BTW, it gets even worse if you append streams, since the original
streams are not copied and magically unfolded when you generate elements
from the concatenation.

> Now, I can see that letting stream0 automagically get access to the
> unfolded result can be an optimization in some cases, although in this
> case it's a pessimization.  It could also affect the semantics if
> unfolding the stream has side effects, not sure if stream.el makes
> guarantees about that though.

AFAICT we make no such guarantees at all.  When generating stream
elements has side effects (which is not ideal, but it's not forbidden),
then you must know what you are doing.  In my experience, side effects
often directly correlate with element generation - e.g. for a stream of
search matches, a side effect is to set a variable to the position where
to continue searching.  These kind of side effects are not problematic.

For non-trivial side-effects, dunno, never wanted something like that.
But this problem also concerns other forms of delayed evaluation,
including thunks, and generally everything you can do with closures.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 14:11:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 09:11:54 2018
Received: from localhost ([127.0.0.1]:40312 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erlPe-0001P9-Nj
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 09:11:54 -0500
Received: from mail-it0-f50.google.com ([209.85.214.50]:34644)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1erlPd-0001Ov-LI
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 09:11:53 -0500
Received: by mail-it0-f50.google.com with SMTP id n128so2142822ith.1
 for <30626 <at> debbugs.gnu.org>; Fri, 02 Mar 2018 06:11:53 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=lHPt3axmMeemcCSiyLyeUQOwJu9A0/wp1ogB44e7ADI=;
 b=IW0X5TM/sgI/QKtfdFXTIAwYtgZSDsDiH2YLm7vbJyUG4Dv5dR0GT8v3AyXcggTl3s
 hTWwv0q5zXkl0Bq7olytpv5laQPwt7AA7pxacNZherNnSfFUuhOX6bAG2kcqTjiswrtC
 8Tto1DNxMudPXBev4Loet2ADuP1grhE+5p2H2OhVhTOZkJRhS9FwSTkxYvhjcuxBKvr9
 Txa4Z/6g2Jttb3TW3GKHM3V6zTtvT0dE8yG3uZXCvYdKzYWfw2ECh/cSK8uReVN13VYr
 /MSGWPB+m+2YPzr/L/Zj5AvGg2WN7gPSlYPsnwSdg73SmGeMwkXEgpUHvf/eqGOa02Et
 beeA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=lHPt3axmMeemcCSiyLyeUQOwJu9A0/wp1ogB44e7ADI=;
 b=B4w/fMOuHWEd8xQRp5Xma7xk6xzMpq2wkx2t+9drEmK9RveUImQ98gVmi5+WFLHCGx
 BNfiTqZS28AJAmN8nh0BlA0Inka3EJtSN8+X2QJXoYAj2ggFvdnRSd5SsPL78Cna24Om
 f0hU7G518BIrfhBl7k18+I+n27Q19ImRxEDXBgxFmegb5besIvb6Cv5YHwwj0HCJ3G43
 /UgYqxvKKTDYantx9KOiF7+zlyCb7LSx8fl/ULB0lp87mlQGDu1jyEE377v5bV7wWp70
 aDgG6t43dpyW3LyK3NEPkvUTaKoA3QrJn9BisADIMJkNSQxPmybldOIcO40uM4zeULcm
 gh7g==
X-Gm-Message-State: AElRT7GJf3T7noOUBTjwgizokdKjK9UN07E87j2ZrK2YnRLdq9fk19P1
 YTqSr/nSysWh6r37u6yvuLw=
X-Google-Smtp-Source: AG47ELuKUlUGdjpbtSYPJin2WtBpXrPwFiH+KAuB7HTF5w1n1iT5engUK1YWokS+rRI6lmeOuGO+gQ==
X-Received: by 10.36.55.70 with SMTP id r67mr2688149itr.40.1519999907945;
 Fri, 02 Mar 2018 06:11:47 -0800 (PST)
Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34])
 by smtp.googlemail.com with ESMTPSA id
 12sm875293itm.1.2018.03.02.06.11.47
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Fri, 02 Mar 2018 06:11:47 -0800 (PST)
From: Noam Postavsky <npostavs@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN> <837eqxyqoe.fsf@HIDDEN>
 <87lgfdnf86.fsf@HIDDEN> <87muzs11hk.fsf@HIDDEN>
Date: Fri, 02 Mar 2018 09:11:46 -0500
In-Reply-To: <87muzs11hk.fsf@HIDDEN> (Michael Heerdegen's message of "Thu, 01
 Mar 2018 12:25:43 +0100")
Message-ID: <87606e4lel.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: Eli Zaretskii <eliz@HIDDEN>, Nicolas Petton <nicolas@HIDDEN>,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Michael Heerdegen <michael_heerdegen@HIDDEN> writes:

> #+begin_src emacs-lisp
> ;; -*- lexical-binding: t -*-
>
> (require 'stream)
>
> (let ((stream (stream-range 1 1000000)))
>   (while (not (stream-empty-p stream))
>     (cl-callf stream-rest stream)))
> #+end_src
>
> This is semantically exactly like test2.el, only the call to
> `stream-flush' has been replaced by literally writing out the
> definition.  Nonetheless, the compiled file suddenly doesn't crash Emacs
> when loaded.  Loading the uncompiled file test3.el still crashes.

Aha, but the following also crashes, whether compiled or not:

;; -*- lexical-binding: t -*-

(require 'stream)

(let* ((stream0 (stream-range 1 1000000))
       (stream stream0))
  (while (not (stream-empty-p stream))
    (cl-callf stream-rest stream)))

So the problem is that the initial stream0 object can reach the entire
unfolding stream as it goes, and just holding on to that reference is
enough to keep the whole thing in memory.

Now, I can see that letting stream0 automagically get access to the
unfolded result can be an optimization in some cases, although in this
case it's a pessimization.  It could also affect the semantics if
unfolding the stream has side effects, not sure if stream.el makes
guarantees about that though.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 13:14:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 08:14:19 2018
Received: from localhost ([127.0.0.1]:40269 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erkVv-0008TJ-Ho
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 08:14:19 -0500
Received: from mout.web.de ([212.227.15.3]:32991)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1erkVt-0008T3-5w
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 08:14:17 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb002
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LnjNH-1eDT4Z3Ll1-00htRs; Fri, 02
 Mar 2018 14:14:00 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Noam Postavsky <npostavs@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <5d5ccb32-434a-cda9-67c4-c60abeb450df@HIDDEN>
 <CAM-tV-9-btKs=gpp7+j0RZ9SfqL-4Q6VSnnPAoA9R_tgvgLEEQ@HIDDEN>
 <87muzrg2hq.fsf@HIDDEN>
 <CAM-tV-8S-TfjwuMaFvH+MfWF1TL_z6tzJGskQMgQ8vavwf=2rw@HIDDEN>
 <87fu5jeyyg.fsf@HIDDEN> <878tba4oo3.fsf@HIDDEN>
Date: Fri, 02 Mar 2018 14:13:59 +0100
In-Reply-To: <878tba4oo3.fsf@HIDDEN> (Noam Postavsky's message of "Fri, 02
 Mar 2018 08:01:16 -0500")
Message-ID: <874llyfwmg.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:bhxg7g7AFV7bzh2b+niKz0SiGo/IjU5NAPX8cOHBsl6XrgUap5c
 uJDNjM/3aEwu0G2juRGpXYapvqr50eIgTVwF5SqoMVh9GbwkLHTH8euhrUGw+VJG1K8liti
 kbPP78NleekpgXfGzSxc57Ekru2861cb9rFsnQQZYQQRNSz1X41ePTzoOcxvogXn9aJY8Ft
 n4CqAVaZ0fpo06BSnrvRg==
X-UI-Out-Filterresults: notjunk:1;V01:K0:0GJg6FnsdjM=:hrkjCuZBmuhWW7blc0dYtz
 KzSWnia32oxOGC71qfMiKnyuvLKVLwvyGlLlln8ZD4H7vV/wCLETamKo2AFpeB9nxJ06aq2ph
 7TsaRh6NWBFAKD7wnC8wZoruKrlHC+6cbVsdQTA5RUNeWBxBpxpDPB2M8ZWeQEP5YECqQFj2e
 YM7vvYD0M4PUbTgkmD7LwroN8VfE0LzDCINZ7wLUNJvWj6Qsx3t0+f4k6GtMU3mtoUONZaBOW
 nGBP1BYo49M/XMmufoRURIIqQdYSiLpFZzE3qalLMWKDa5vkPxWBQU7SZz0aotIGaxCEiTb5S
 uSuQpOtm9VYgy1xo5aBCnL4EaTRbW5vnxWtt+gfCnTliU3HhycXBhxlfkBoUFtDlbHqAxM8iZ
 4tSGfAe4Im/7JUq5Fq6oGiQ0XPTLR6b7eR2N6ZpIUeJUkHUkjkQWcqb/tZ1UDgFuBKmWLjgC0
 EPOC6sy2t71Y9FN3S1poBzZmn7VyprQDAF/+jdXIQYCEn34HXw6ArXdfC+NpxxgkuS/rYQUn5
 Pu2SHF4z+CbXpg3xqU7cPloriQUYoRyS2tvT6tqcO4qKh27VhwLvRnGMO8zrfT+TkZKRIbi3T
 OrqUGOVAFeaCtbpj0MyawCns3LBKsVQuI4CTZYF72CzZNb8JD+6Ce/YChILi4CfckWUwp4JXi
 lAVfuXLa5+hYKrksP9sU8MZtgbJe5k3ztoEkYirfgsHV/gajkc4NGOED4kFCVzC47wfonn/Nz
 FHeLzDGKvfEPU3C2pfWPxP8khOKvCORZcDVhmib0a5uN6H/uDtsXjNEnYoIheTK6nGjBDeZ+N
 TpqXIvqbxKOZ1OGNIXYlqa58gihtNo/gtS9gB7Jca/oSKMmG7E=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: Eli Zaretskii <eliz@HIDDEN>, Daniel Colascione <dancol@HIDDEN>,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Noam Postavsky <npostavs@HIDDEN> writes:

> Ah, so let me be more precise.  Iterating over (stream-range 1 n) should
> require only a constant amount of *reachable* memory at any particular
> instant.  So your example above is okay, but the following one would be
> not acceptable (I mean, it's fine if some random lisp code does that,
> but stream-range should not be creating such long lists, or equivalently
> large structures):
>
>     (let ((list nil))
>       (dotimes (i 1000000)
>         (push i list)
>         (message "%S" (car list))))

Yes, absolutely agreed.

Using streams, you can logically refer to the complete list of elements
as one object, but the programmer must ensure that the referable list of
generated elements doesn't get too large.

Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 13:05:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 08:05:03 2018
Received: from localhost ([127.0.0.1]:40232 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erkMw-0008Dn-R4
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 08:05:03 -0500
Received: from mout.web.de ([212.227.15.14]:44443)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1erkMv-0008DG-5d
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 08:05:01 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb004
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LyOuM-1egJqC1fks-015trc; Fri, 02
 Mar 2018 14:04:54 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Noam Postavsky <npostavs@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <5d5ccb32-434a-cda9-67c4-c60abeb450df@HIDDEN>
 <CAM-tV-9-btKs=gpp7+j0RZ9SfqL-4Q6VSnnPAoA9R_tgvgLEEQ@HIDDEN>
 <87muzrg2hq.fsf@HIDDEN>
 <CAM-tV-8S-TfjwuMaFvH+MfWF1TL_z6tzJGskQMgQ8vavwf=2rw@HIDDEN>
 <87fu5jeyyg.fsf@HIDDEN>
Date: Fri, 02 Mar 2018 14:04:53 +0100
In-Reply-To: <87fu5jeyyg.fsf@HIDDEN> (Michael Heerdegen's message of "Fri, 02
 Mar 2018 08:08:55 +0100")
Message-ID: <878tbafx1m.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:cWXk8iPvsSVTcDvL0N/M8rcHMg7OiCp6R3Eg+I/2VMb/FxbRuPk
 5ipAiVKAvQgYXAPnPgs6oqQ4NVF6pLOSlKQJyaUFAOOypoWP44KJ56ts5HgtA4tX1XOZUC+
 4E9gWm7mtjSm4sHNW3RfNJdxJmlSwI4EFMZnteZvxMTSWoVJXtyht3ffheHSrK615n99gDx
 DF0cYKUIIi6Ehi33J8MXQ==
X-UI-Out-Filterresults: notjunk:1;V01:K0:hd7cvNf9LOc=:wGZYuS8l5LA1aXWpkBP0vx
 wdv22GGa7Wxr0r+hm93jzJPuLgOOW+aGL+o4gbGgl0T8Vs/CFVcm/8hRDALkYfmJHDbz3/acq
 07yxuoJoBTx0pkmEZR3YaCmwX1GuVlt/hLBtJsWgiuOqEMBaKqEBidn5CT0Nk/aGqmxKKyVYG
 5wNXtykmWpRFi66D1i3102wtoMckxZqCE4hlKhNQbFZdIW0sWV2bRDXUZcHBMRr0o6KmGCWzA
 AI9uBQIaq5pd/o+QGC/3hvobKgYCgDimaOVDY+xtuEWZnXa0T+oagWXrlzKSKoHzMHYfnDICI
 B1vUuTunYdAoknkqGavdaXpAD2sLAuxx+zqJ9CoiQ507u8cznqLIfRklhWHab3Zmd/I/nbsrL
 WyxcS+EWdlWcVlTLBXr5Ujkb6Iwx1uHapLhZu1EsWX06FYQRWScFoBiXA9PEty+zRpHIsIeEb
 v/xYKL0Xu4tmhLuWzXgy1iL+yDx8XQU7GVQAGUazPsuG1RweFxqVloEg3cRf+jkJJhqMEhNto
 VbOoPmTybZGIbpAi+aMwnCZMSgqQ+LgTxfBVo5gHKg4A+nRBD5lRPXjCoPx0kJ6zV3Oi66qBv
 CnA6FRAq3xWEsr/8ibBcf3uNitK7QDQNg8HyTo9fTZVWFoB3d2a1MLtTlXWojG1QF807XFVDF
 V+MyPOMasBHmY1mBVPXICrHyYl34jFKwU8Eh97mMeOFXy849wj+dd9cYxlz6uMaUHlcbu/++P
 VdZ81CpOiBBvN17VhWt3s+/JhSDiimld1iYD0N3tKKIvaOIATtiKN/b+3ZLreBOXzAhltl2ND
 2yH/tS8/ovUgJ1Dkcz5R7vuNjoOD9ymxX0X7mBdJ5+vt5rZnho=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Michael Heerdegen <michael_heerdegen@HIDDEN> writes:

> Noam Postavsky <npostavs@HIDDEN> writes:
>
> > Maybe I've misunderstood, but is it not the case that iterating over
> > (stream-range 1 n) should require only a constant amount of memory,
> > regardless of the value of n?

But in this regard, we have a problem with how lexical-binding is
implemented for interpreted code.  Nested thunks (as implemented in
"thunk.el") accumulate useless variable bindings - e.g.

(defun test ()
  (thunk-force
   (thunk-delay
    (thunk-force
     (thunk-delay
      (thunk-force
       (thunk-delay
        (lambda () 1))))))))
(test)
==>
#1=(closure
((check)
(#:val . #1#)
(#:forced . t)
(check)
(#:val . #1#)
(#:forced . t)
(check)
(#:val . #1#)
(#:forced . t)
t)
nil 1)

The length of the variable list is equivalent to the number of thunk
wrappers.  I believe that these useless variable lists are responsible
for the crashes of the uncompiled versions of the test files I had
posted.  I think this problem is different from the gc issue.

Streams use nested thunks.  Of course does thunk.el not explicitly add
such variable lists to the result - this is how closures are built in
interpreted code.  For nested thunks these just add up.

BTW, if you byte-compile the above `test' function, then

(disassemble (test))
==>
byte code:
  args: nil
0       constant  1
1       return

and this problem is gone.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 13:01:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 08:01:27 2018
Received: from localhost ([127.0.0.1]:40228 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erkJT-0007ay-Bd
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 08:01:27 -0500
Received: from mail-io0-f176.google.com ([209.85.223.176]:34433)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1erkJS-0007Us-7L
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 08:01:26 -0500
Received: by mail-io0-f176.google.com with SMTP id e7so10552545ioj.1
 for <30626 <at> debbugs.gnu.org>; Fri, 02 Mar 2018 05:01:26 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=YmEavTgt1rdL9FdqjtV5K2D2HbFinXHIPeUDCwyhqCQ=;
 b=UStxfH5q48Z42aNUuenCp4aVPkitvVkNL7H6gVL0FiGjlrcTjLuxyVKzNyXAJ7f5rT
 Gpd2qZubzNA0SdBPypECjM57maNxFv7JRbR/8YvLUifbMVnUydNF5PIVclQF2FhNJtLR
 u9reQMWu5tvmXFvr3vPpbjxguq9wgoB3x+TdMFrA+BgvFk/pLUq5rZEMxxbTkOFAcZdf
 aLr+v0pXVLTnO/wgK87/sDc4+JwflvkUW7DIoki1dLMIwjFksBXBHh3VoBZusrEcGgwH
 D5w2i2+NV/BaVl16jd8xxDdRAuKbdfLY/jQ8mqWnE1FwmLdlZ3U/JY+NXW8sjLEoHkfS
 nU+g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=YmEavTgt1rdL9FdqjtV5K2D2HbFinXHIPeUDCwyhqCQ=;
 b=WZEDTQwjspweakbq/VDdOJDZj1XKrOvHr53hD4jAB5ZyIuSD0tSSWEieQr9y6mE4L1
 27lkIjnfrTfEzEjz0OPjw2vUUWWEMh1NgE9zgGyrqXwXX9mMzXZyFpfkh03gTQkD952H
 8e1VhTU447i58H6nRHSekked3wUrj/2KjSaF331S8yYovw92LUntNINhdD47zFwa5NZs
 ZnVgz33qCBgqzz92wXUkhoWXvi/6t8Kv5SFowv3rB7SFLCL5LObrotNPic1xPDju7BpY
 VpS4YCXxx4t1T0velvuAG7V+LOQmImLqPb59ToxuAoqZM93q8CYotAChY4e/Py9SSZi5
 Nuag==
X-Gm-Message-State: AElRT7GPGLJuL1CpO5pcp8YOEjzLS6Jht5eu46uK+5fQ1qvkAHSkwBQ0
 +eP4sagiNl8eaaIZor9qutA6kA==
X-Google-Smtp-Source: AG47ELtdyf5b4JoyzIjI+Q5HQK34ZBQs+KQIWajDhqCHv/y8mJSgiFweQcs+wFNSoES+FO5NlU5uMg==
X-Received: by 10.107.134.95 with SMTP id i92mr6332864iod.210.1519995680384;
 Fri, 02 Mar 2018 05:01:20 -0800 (PST)
Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34])
 by smtp.googlemail.com with ESMTPSA id
 c91sm4425920iod.16.2018.03.02.05.01.16
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Fri, 02 Mar 2018 05:01:18 -0800 (PST)
From: Noam Postavsky <npostavs@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <5d5ccb32-434a-cda9-67c4-c60abeb450df@HIDDEN>
 <CAM-tV-9-btKs=gpp7+j0RZ9SfqL-4Q6VSnnPAoA9R_tgvgLEEQ@HIDDEN>
 <87muzrg2hq.fsf@HIDDEN>
 <CAM-tV-8S-TfjwuMaFvH+MfWF1TL_z6tzJGskQMgQ8vavwf=2rw@HIDDEN>
 <87fu5jeyyg.fsf@HIDDEN>
Date: Fri, 02 Mar 2018 08:01:16 -0500
In-Reply-To: <87fu5jeyyg.fsf@HIDDEN> (Michael Heerdegen's message of "Fri, 02
 Mar 2018 08:08:55 +0100")
Message-ID: <878tba4oo3.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: Eli Zaretskii <eliz@HIDDEN>, Daniel Colascione <dancol@HIDDEN>,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)

Michael Heerdegen <michael_heerdegen@HIDDEN> writes:

> Noam Postavsky <npostavs@HIDDEN> writes:
>
>> Maybe I've misunderstood, but is it not the case that iterating over
>> (stream-range 1 n) should require only a constant amount of memory,
>> regardless of the value of n?
>
> Depends on your definition of `require'.  Like, for example,
>
>   (dolist (i 1000000) (message "%S" (cons i (1+ i))))
>
> each iteration step creates and discards a new cons (or a constant
> number of conses).  Not a exceptional thing in Lisp.  When iterating
> over (stream-range 1 n), the garbage collector seems to have problems
> with how the garbage is structured.

Ah, so let me be more precise.  Iterating over (stream-range 1 n) should
require only a constant amount of *reachable* memory at any particular
instant.  So your example above is okay, but the following one would be
not acceptable (I mean, it's fine if some random lisp code does that,
but stream-range should not be creating such long lists, or equivalently
large structures):

    (let ((list nil))
      (dotimes (i 1000000)
        (push i list)
        (message "%S" (car list))))




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 07:09:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 02 02:09:16 2018
Received: from localhost ([127.0.0.1]:40031 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ereoe-0004DN-MC
	for submit <at> debbugs.gnu.org; Fri, 02 Mar 2018 02:09:16 -0500
Received: from mout.web.de ([212.227.15.4]:43147)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1ereoc-0004DA-Jg
 for 30626 <at> debbugs.gnu.org; Fri, 02 Mar 2018 02:09:15 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb001
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0M9oW8-1eyDKL2iQy-00B33Q; Fri, 02
 Mar 2018 08:08:57 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Noam Postavsky <npostavs@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <5d5ccb32-434a-cda9-67c4-c60abeb450df@HIDDEN>
 <CAM-tV-9-btKs=gpp7+j0RZ9SfqL-4Q6VSnnPAoA9R_tgvgLEEQ@HIDDEN>
 <87muzrg2hq.fsf@HIDDEN>
 <CAM-tV-8S-TfjwuMaFvH+MfWF1TL_z6tzJGskQMgQ8vavwf=2rw@HIDDEN>
Date: Fri, 02 Mar 2018 08:08:55 +0100
In-Reply-To: <CAM-tV-8S-TfjwuMaFvH+MfWF1TL_z6tzJGskQMgQ8vavwf=2rw@HIDDEN>
 (Noam Postavsky's message of "Thu, 1 Mar 2018 12:15:43 -0500")
Message-ID: <87fu5jeyyg.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:TcsKnI4YUJlrmbQHDHf/m1ZQH2l6yFnjF2srPHAjjejXlVhgu9g
 By45ZiN9GuwnzYcwfkhC1qMkPk3pcmBlbw268XGIJLgQygoWn5dezhTWP6nN4PZDOASdD8s
 VmsKYzBti2SAUPTjHJFy2RJbQ0TUNotjxMtpfkOjvQA09CzRGAzKuN4/vpPJ51ABjsyqrB9
 V+xzozGaa6Ci3nJibCSiA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:BAnyNq6AhdQ=:OcBQAAh0SlVJCscBw2wX5K
 xcALWDLlkSV6xSQkRKnefwOQvAWVvwka2yD8Nkn1bVA5/Zi0/sXygyDDrl9P0InUewhq7r33n
 FbaSdXE6NfQAwqsiszp0Bz2hjrVAd4rMc8H466Ch6Q1RJRVSRZmCdg70e2al52uIhkk/xgz2b
 BxuIJ9zBXAmPyPTor0/RNQpkUfGnoi2MNcsZLYh+Unnn1ASijL1FW3jS+Et1e9YPHDHUcA0fL
 zoq46j087QwEqbFhOyCtKTo+BdwdvrQHGE3+vCzkvJTC2IYGCnPd1PPxMxb6juuwP3eQOJ+MU
 EBt/cpyBCi/uBUp9db3/uRtpHBXkY2sP8Z3d/75OfgYSRV5L1OjzyLLesIAc7alw4TIlkCwVH
 Yy8iySVYAB3mYtR4cicXfxWa66rqdDTnkUS1uRF51GkVKaUtrdOd0s/90cERD3eEJKXuVviH0
 tX08gcK6dIB5+Mg+F26kv6R/OwG7z9f0z0zkwxXUDuhoDWegXS2H06M7Lf4cetIAYNM+joa9Z
 c03hmmOEdsygQbMtAwSJu5AmzPkvPsiYW+3L+4bQAU9bW1NCYwCdh7jyZW6IWM0XS//aw0q/P
 q+zWiWaSELINRooZicjQUcfGo8SOuzbOsDEJcV1evV1+58SDDPOEAb9bYwnz24DTCa4wFgAR8
 qUlB+RxxqMkIBl25IVVlQtCLjIGqMVSXfiDToE91Gxa/FPX5qqBenZ3xU2YJpr/uba7B4Ukcf
 35XyN/M6XHRMANr799qXFA3Zar0hr3CdqF1dQ9p8Pa+Lakr2sQ06HLFTGB4cA+N2fATnisb6l
 0z6sTjBrO4BM0zyk2DTQLtBr/n90SppmJFwLvXs6dS7aeY1280=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: Eli Zaretskii <eliz@HIDDEN>, Daniel Colascione <dancol@HIDDEN>,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Noam Postavsky <npostavs@HIDDEN> writes:

> Maybe I've misunderstood, but is it not the case that iterating over
> (stream-range 1 n) should require only a constant amount of memory,
> regardless of the value of n?

Depends on your definition of `require'.  Like, for example,

  (dolist (i 1000000) (message "%S" (cons i (1+ i))))

each iteration step creates and discards a new cons (or a constant
number of conses).  Not a exceptional thing in Lisp.  When iterating
over (stream-range 1 n), the garbage collector seems to have problems
with how the garbage is structured.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 17:15:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 01 12:15:51 2018
Received: from localhost ([127.0.0.1]:39571 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erRo7-00071d-Cr
	for submit <at> debbugs.gnu.org; Thu, 01 Mar 2018 12:15:51 -0500
Received: from mail-ot0-f181.google.com ([74.125.82.181]:46008)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1erRo6-00071O-6T
 for 30626 <at> debbugs.gnu.org; Thu, 01 Mar 2018 12:15:50 -0500
Received: by mail-ot0-f181.google.com with SMTP id f11so6227625otj.12
 for <30626 <at> debbugs.gnu.org>; Thu, 01 Mar 2018 09:15:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:in-reply-to:references:from:date:message-id:subject:to
 :cc; bh=KOLq+21D6dCWMJ5tcp6BoMnKTCUue4G7LPcxu//TEjk=;
 b=U7euIbGT8ezO1Hp7mWWqDElrZ4JrbNPShyiayDoDvtdY9fiFcmmDNLetlm/eMUVml2
 CD5i0OWXtP9P7jOfi+tWM9G416OHU1IvRAaB8LdB2mmCx5xEEuEuSUl4oygpr1TsKGNJ
 qbO87DIASh6Bz/aFTgyRNiRb/l3k3scT6MjS+fKBOGF0dhyrv1N03ZaiaHz2bn7lJUyR
 IoT/UdYxwV+pnv5D6MwZs3ZM8wZpMpOU2NWJlGiKaGrdfudeliw813v7ATXEZ5cTGqK/
 bqSCYlGWol9AQ2D+sb0TiUscjP4sorCwzP5854vRS38wzI5e8X9sJKGBO4aa70vGS/lf
 QbHQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:in-reply-to:references:from:date
 :message-id:subject:to:cc;
 bh=KOLq+21D6dCWMJ5tcp6BoMnKTCUue4G7LPcxu//TEjk=;
 b=T6oq6IEsY0XfACJY8pWSf3uNwEMiXYA1CtKl140iQOTjB8DScjool04IhpbIIdtbPd
 BaX5R9dit4z+wQvOTbLB934Vphb5I/DCFjSz/+4zQ/uYCfFbvy5o3q04Sm7oDMcPvSBJ
 xM+Jgl5M/Dlv/ELn21fxFeEBAuhY7pSEHFZR2LJojKORKx7MUK7OihipE+zzXLX6FRtc
 kQoATXNNDHzMlju/Jvqx06k+rBypm1nGsHcPhugFZzAOSQFoTswg3qxIN6xDfTT3ngW2
 z4Yk6gjtf6fzqrn2tXFCmzIrbM0RUWJASp8wnSvrQ53N3okXRMFB0pNeANFS8MgR4Cun
 1CoQ==
X-Gm-Message-State: AElRT7FdHpPWT9wHSTLkKVWaxhvCgZ6xRqvg+ebD+2BYq1VjHawPIc24
 iHSGP3lNsk72Zgs9zB6EfBaz9qLosB690TErJUU=
X-Google-Smtp-Source: AG47ELtmQ34KbyS6Z0HTqfKg8PgSEYjQTxHOXB33weUD2Z7PvaujV14gMxvpt6QB2zoUtVQ+UcT5enEUlCtHEv73/lM=
X-Received: by 10.157.29.218 with SMTP id w26mr1813973otw.24.1519924544545;
 Thu, 01 Mar 2018 09:15:44 -0800 (PST)
MIME-Version: 1.0
Received: by 10.74.59.65 with HTTP; Thu, 1 Mar 2018 09:15:43 -0800 (PST)
In-Reply-To: <87muzrg2hq.fsf@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN>
 <87fu5moe4c.fsf@HIDDEN> <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <5d5ccb32-434a-cda9-67c4-c60abeb450df@HIDDEN>
 <CAM-tV-9-btKs=gpp7+j0RZ9SfqL-4Q6VSnnPAoA9R_tgvgLEEQ@HIDDEN>
 <87muzrg2hq.fsf@HIDDEN>
From: Noam Postavsky <npostavs@HIDDEN>
Date: Thu, 1 Mar 2018 12:15:43 -0500
Message-ID: <CAM-tV-8S-TfjwuMaFvH+MfWF1TL_z6tzJGskQMgQ8vavwf=2rw@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: Eli Zaretskii <eliz@HIDDEN>, Daniel Colascione <dancol@HIDDEN>,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

On Thu, Mar 1, 2018 at 11:54 AM, Michael Heerdegen
<michael_heerdegen@HIDDEN> wrote:
> Noam Postavsky <npostavs@HIDDEN> writes:
>
>> Agreed, but could you please open a new thread for it? AFAICT, this
>> bug thread is about streams.el functions producing structures
>> proportional in size to the length of the entire stream, which is a
>> bug in itself, regardless of whether or not the GC can handle the
>> result.
>
> No, it is a feature: we want streams to produce these structures,
> streams are delayed lists, not just iterators.  Lists are also
> potentially unlimited nested conses, and that's not a bug.  Streams are
> the same realized with delayed conses.

Maybe I've misunderstood, but is it not the case that iterating over
(stream-range 1 n) should require only a constant amount of memory,
regardless of the value of n?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 16:55:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 01 11:55:18 2018
Received: from localhost ([127.0.0.1]:39561 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erRUE-0006To-6K
	for submit <at> debbugs.gnu.org; Thu, 01 Mar 2018 11:55:18 -0500
Received: from mout.web.de ([212.227.17.12]:46719)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1erRUC-0006Ta-QA
 for 30626 <at> debbugs.gnu.org; Thu, 01 Mar 2018 11:55:17 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103
 [213.165.67.124]) with ESMTPSA (Nemesis) id 0M5wzV-1eTiZO0FyL-00xqC3; Thu, 01
 Mar 2018 17:54:59 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Noam Postavsky <npostavs@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <5d5ccb32-434a-cda9-67c4-c60abeb450df@HIDDEN>
 <CAM-tV-9-btKs=gpp7+j0RZ9SfqL-4Q6VSnnPAoA9R_tgvgLEEQ@HIDDEN>
Date: Thu, 01 Mar 2018 17:54:57 +0100
In-Reply-To: <CAM-tV-9-btKs=gpp7+j0RZ9SfqL-4Q6VSnnPAoA9R_tgvgLEEQ@HIDDEN>
 (Noam Postavsky's message of "Thu, 1 Mar 2018 10:51:31 -0500")
Message-ID: <87muzrg2hq.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:yPftk7d2EJsspEQ1VzbZAiK8W3WYDB0tMe0SjnK7mAx8VApwXk2
 Yy6IY5XbyrPsDeaVZ8ZQs9AXskrPkdqI8hWzu8fivqJq/1/IWWg16hZIsmq8AVf/bj8nC91
 QaEYsxKN+HOYHpZ+tH4/uoBjbZz2AMZSnmWMHszPIU9XjoDvAodis6U/Gnq6mLU1NM10pfV
 kjC8oZYUwOdJl4TGw/hiw==
X-UI-Out-Filterresults: notjunk:1;V01:K0:if2sPxsvh1Y=:jmJUn1S5k8cJ3VsZfiISA5
 LBPjdqqPmNHQxBxxCqhu41CzIiuEYeY2508ncofIOrlifpxyiGlFpVsPoK8evUpzjHYoJgU+5
 AFHUU1jBibr2humfYinJydnOMVCAfftSPOUcyuAxxw9JW8sVEb8cMI6MTv+HFaE/kRjlLiwji
 ar1SNevn1n3T5wnVnFhQMsWqafc13E0Zac+0KJdBqSRNWrDBn4hRs2cA5NvlJIzcwYNetC/HW
 WYBquIW4FG5DF31lOMCwOlmB5hIXcT5eZRs1U3zcjuWYy3wX169TXvdwZCg1R9H8h0+KL/QGP
 pFfnBp3LO20Ofx7/YOXjWQfaKFMwyTOzMu0E2cJmb1DlKNGj5fJ3WqlCGn14FvNAQ5h5G+GFw
 xq3GEhw7c2w7roYJffcWKh/qtaRwqGDcIKTe7LcF77+7gRYlTynEbBiPjk07SyPJfe3Ij9SvD
 gsGOCOwSMGk7QczLJznXNMj5xJ9bMLxdgYxekcCUZqTLAevY7sFUcObQG9wAM3UVFTmFsmt2I
 RZADnY0dnGB0MeY9PBFvo5tyHx8TuE2R50FsiEtFnfQLR8yFqwfFzXoNFhuXHRdZfxc45i+NZ
 lB64z7SCi0pL9+dxuDdoMjWeo5jOS8QYZXSCM84JjPSHmppXTHmXMWmXZZ4orejPJnkY2O02C
 /gfAtSBYkJfja2OVJPMLRzelXu5JgoHHwpD2Br/j6DBwAgnyIMrUaZ2LOAwHTtPvTqIO/A+gI
 x9R9c/Knf/afOnLuLq1/tSfG6dZvbilapz5ZbbKkpc6Ps3WnFtwReYh2T5dMNppvAaTCJrAmO
 70POWhd1CrKoOO2Uii/ixYO4MO5djxAaXmtk5eagTLpv+08sdc=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: Eli Zaretskii <eliz@HIDDEN>, Daniel Colascione <dancol@HIDDEN>,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Noam Postavsky <npostavs@HIDDEN> writes:

> Agreed, but could you please open a new thread for it? AFAICT, this
> bug thread is about streams.el functions producing structures
> proportional in size to the length of the entire stream, which is a
> bug in itself, regardless of whether or not the GC can handle the
> result.

No, it is a feature: we want streams to produce these structures,
streams are delayed lists, not just iterators.  Lists are also
potentially unlimited nested conses, and that's not a bug.  Streams are
the same realized with delayed conses.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 15:51:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 01 10:51:39 2018
Received: from localhost ([127.0.0.1]:39492 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erQUd-00031u-Cu
	for submit <at> debbugs.gnu.org; Thu, 01 Mar 2018 10:51:39 -0500
Received: from mail-ot0-f182.google.com ([74.125.82.182]:45940)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1erQUb-00031g-8D
 for 30626 <at> debbugs.gnu.org; Thu, 01 Mar 2018 10:51:37 -0500
Received: by mail-ot0-f182.google.com with SMTP id f11so5952623otj.12
 for <30626 <at> debbugs.gnu.org>; Thu, 01 Mar 2018 07:51:37 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:in-reply-to:references:from:date:message-id:subject:to
 :cc; bh=TRJxXkM1tXJ+dLfVZFaZMeVWzYo5CKqeawx7lr1svJw=;
 b=jaRWJrFFxbCIh+Z4GLspNH7BN80O79E0Qva3svqwW9WjdTJUGetp6Xn2mkXJ8PVMgB
 d91BQytRUsCsQ9VNvcF6SaWArTF4wIpNFgClo6ayfITufoR+cFv2sDAREy5jB2AmuX5A
 fosfppQ9VZhkFyzi0auheExhOqGuB8w52h6BS4CGCZ6HTwVqaj1NcdkHJiXCiA5BiueS
 KvUi2aeJKEnEEC63YwVD3l5hr3M9s6yrLRWt291jVIXJ/XbGJXLkzhgj3vUKjcEC+qyA
 0xRCHCIifB5anMfnE+UTi+Fri87kuGmkleNkeQUdsFkv4TRmigFxsAzNQXZtTC3M+dOE
 2rHQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:in-reply-to:references:from:date
 :message-id:subject:to:cc;
 bh=TRJxXkM1tXJ+dLfVZFaZMeVWzYo5CKqeawx7lr1svJw=;
 b=NupoKZEa8ncUvm4FX53xlnYthM6uazjB3DiMMD+xEuY9bdaB0uvfM6LLPgBEcMfEOr
 sjlR+SwpMLjWCxZczBbfQ6He8pVDwfS2hOZpfTwIw/osDFJ/nT9R2o6qD4dMi6M6oPd2
 2oHsUdR20KW6GYkkqeyAXA8zmCKNB5fXXKmYpNpPDB0dpAy1xVIa01Di8wCvSos84QTa
 TpONcK/B+Qecm9cc+1Y+1z0NGf0x+nioT1rBVsrLK9qNMXV33db/RePqEqY4bApkQU01
 zhkwyYdkfwqg6qws5+O05X9dKClXbfYD7shw+RmDYeMrDEJRsghYlCeu5NHfnsabi/7q
 h/RQ==
X-Gm-Message-State: AElRT7E0is54+D2yYiyhHbmYZZfQR7LX1XsPl2wL9d97/zur/CcuM9sm
 BNFoQY1+1YBaJmwZjz0C96vqM4sWh9ySMNv16mc=
X-Google-Smtp-Source: AG47ELt6wXBqzVdgeIcgQ8zHTN/aDmwqXD4B3/UkxQl8iDguLmcebvGrORNfpH4Y89SfEcSy+Fn+ALwuNcpgQgn2UcI=
X-Received: by 10.157.47.178 with SMTP id r47mr1569664otb.335.1519919491455;
 Thu, 01 Mar 2018 07:51:31 -0800 (PST)
MIME-Version: 1.0
Received: by 10.74.59.65 with HTTP; Thu, 1 Mar 2018 07:51:31 -0800 (PST)
In-Reply-To: <5d5ccb32-434a-cda9-67c4-c60abeb450df@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN>
 <87fu5moe4c.fsf@HIDDEN> <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <5d5ccb32-434a-cda9-67c4-c60abeb450df@HIDDEN>
From: Noam Postavsky <npostavs@HIDDEN>
Date: Thu, 1 Mar 2018 10:51:31 -0500
Message-ID: <CAM-tV-9-btKs=gpp7+j0RZ9SfqL-4Q6VSnnPAoA9R_tgvgLEEQ@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
To: Daniel Colascione <dancol@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: Michael Heerdegen <michael_heerdegen@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

On Thu, Mar 1, 2018 at 5:44 AM, Daniel Colascione <dancol@HIDDEN> wrote:

> We need to fix GC being deeply recursive once and for all. Tweaking stack
> sizes on various platforms and trying to spot-fix GC for the occasional
> deeply recursive structure is annoying.

Agreed, but could you please open a new thread for it? AFAICT, this
bug thread is about streams.el functions producing structures
proportional in size to the length of the entire stream, which is a
bug in itself, regardless of whether or not the GC can handle the
result.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 15:00:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 01 10:00:37 2018
Received: from localhost ([127.0.0.1]:39444 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erPhE-0001rB-IG
	for submit <at> debbugs.gnu.org; Thu, 01 Mar 2018 10:00:36 -0500
Received: from eggs.gnu.org ([208.118.235.92]:50466)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1erPhA-0001qw-H5
 for 30626 <at> debbugs.gnu.org; Thu, 01 Mar 2018 10:00:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1erPh1-0000lH-A8
 for 30626 <at> debbugs.gnu.org; Thu, 01 Mar 2018 10:00:27 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:54060)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1erPh1-0000ku-6Y; Thu, 01 Mar 2018 10:00:23 -0500
Received: from [176.228.60.248] (port=4586 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1erPh0-0004G8-JZ; Thu, 01 Mar 2018 10:00:23 -0500
Date: Thu, 01 Mar 2018 17:00:33 +0200
Message-Id: <83k1uvyh66.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
In-reply-to: <87muzs11hk.fsf@HIDDEN> (message from Michael Heerdegen on Thu,
 01 Mar 2018 12:25:43 +0100)
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN> <87muzs11hk.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 30626
Cc: npostavs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
> Cc: npostavs@HIDDEN,  30626 <at> debbugs.gnu.org
> Date: Thu, 01 Mar 2018 12:25:43 +0100
> 
> Seems that whether we get a crash or not depends on details in the
> implementation of lexical-binding.

Byte compilation doesn't just produce byte code, it also changes the
code into an equivalent one, but "equivalence" in this context doesn't
include side effects like stack usage.  As an extreme example,
consider a tail-recursive program that the byte compiler converts (or
at least might convert theoretically) into a loop.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 11:25:54 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 01 06:25:54 2018
Received: from localhost ([127.0.0.1]:38262 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erMLS-00013F-48
	for submit <at> debbugs.gnu.org; Thu, 01 Mar 2018 06:25:54 -0500
Received: from mout.web.de ([212.227.17.12]:39465)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1erMLP-000131-R4
 for 30626 <at> debbugs.gnu.org; Thu, 01 Mar 2018 06:25:52 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103
 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MEmgA-1exOjq0v1O-00G584; Thu, 01
 Mar 2018 12:25:44 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN> <87lgfdnf86.fsf@HIDDEN>
Date: Thu, 01 Mar 2018 12:25:43 +0100
In-Reply-To: <87lgfdnf86.fsf@HIDDEN> (Michael Heerdegen's message of "Wed, 28
 Feb 2018 19:25:45 +0100")
Message-ID: <87muzs11hk.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:ZA3UyJrqrPYXyGGH6nkUim5onRakIxD7uo7ZlAFiMl5lZVvqLkI
 gwNnEZ4gzk4nNZ2vIkiD56E+Xe68ymt5YphYli7fEbLmX1b/LpO9MLG7FnVl2Mdi68m1OQ5
 6jLVq+/3B36Xt0SPyfke/8l0mQMQJn6T8zs9XM+dnkflUKuEQ5Do3sqkdYT/dru9WEjn2PT
 ZDDnVQu/ll6NbwdoW0bog==
X-UI-Out-Filterresults: notjunk:1;V01:K0:pixoC40cy78=:1khWpvoXDTEKOgOQZBuVQr
 3XbXhZNRqFzyTfb4NwgsGwJSF1zaf+YPHgR1lD16ex7Vo1FWY2k2kQ7QPq33oKszVngVYE/jQ
 PPMtQU2FYt+5T/x7et5E+XSB2rc1hhw4Ky9/n0Lo1AItd1LZ15kQOF9dLiPAfzMXV3f3ZFYf+
 1Hy8nN5MPRlnyA8iSrQwBLUtWWVGBmhLHWAyE/A4fQQp09fW9NhtWpLNwP6w5q2sy7TLxAE9B
 gTVYZQOftPJjoKM2xkku8+OX3osXmwMcuZjDnVJFWfOuYkpSCE8HhPIZkQBEVyhSs4s6H+toV
 5zC8jjjj34ny5PqaHn0zZ5AZXSRLushH8jXnw30QiH8+0/G1rgVj2GkTktNlC1zKMGRz3O22Y
 WlzT9RKR0Y8n3Fqey/DHwwbEB4JT7zoYxqMUTE1Zn9Q4q44SDHHoHmg9sBKzl1Nw918139lrP
 nW+z9vMucDSW7fO5mH69JG48rK1T1dtZZ5gklnF9wL5ZUNAujLskh3GKzDQyVsfyiFqweyu1Z
 iU5dwaElCznD4EFWA9I7ER14IrSKPzzaUKdVitBvO21iW80zdxEksTYaVUeYkVB6jETJrjf69
 NXYeLCFuMhqTL6FPzNf1pXgZSf5ttBDC943/K6wJAJ31vbVvJlPkUevW3g9lL0fQZ32KSFzbi
 aKx3VllAVLUbcV2OsfI8yj8PXY4FTl3UX+CprFbhwEb3wyn70MU1qpkByskiHwZ4jSZRvh/sI
 9JPoACVNoYpm/6mztu+BMiXRuwxw0iN4j+s7rc7lodof6U8bG7mA7irBt2p6/zFwgvwc5ZeNk
 ZDmWEH1bSoPvCtJQCSd8JUWxs+6sofIaVlY7VNcjC+3+q+7rlY=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: npostavs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Michael Heerdegen <michael_heerdegen@HIDDEN> writes:

> The inner `stream-range' call results in a closure, and I guess that
> this closure includes a reference to the outside VAL, which is the
> stream from one step back (though there isn't a lexical reference to the
> variable...does that make sense?)

This is probably only a part of the puzzle.  Some examples:

test1.el: This is more or less the `stream-range' example reimplemented
without dependencies so that it works in emacs -Q:

#+begin_src emacs-lisp
;; -*- lexical-binding: t -*-

(defun my-range (start)
  (let (forced val)
    (lambda ()
      (unless forced
        (setq val (cons start (my-range (1+ start)))
              forced t))
      val)))

(defun my-test ()
  (let ((range-object (my-range 1))
        current-cons)
    (while (< (car (setq current-cons (funcall range-object))) (* 1000 1000))
      (message "%d" (car current-cons))
      (setq range-object (cdr current-cons)))))

(my-test)
#+end_src

Loading this file test1.el crashes Emacs - but if you compile it,
loading the compiled file doesn't crash.  This is what I expected from
my previous thoughts, because only the uncompiled closures include a
reference to the outer VALs.


test2.el:

#+begin_src emacs-lisp
;; -*- lexical-binding: t -*-

(require 'stream)

(let ((stream (stream-range 1 1000000)))
  (stream-flush stream))
#+end_src

This traverses the whole stream ignoring the elements.  Loading this
file crashes Emacs, no matter if compiled or not.  I'm surprised it
doesn't work even when compiled.


test3.el:

#+begin_src emacs-lisp
;; -*- lexical-binding: t -*-

(require 'stream)

(let ((stream (stream-range 1 1000000)))
  (while (not (stream-empty-p stream))
    (cl-callf stream-rest stream)))
#+end_src

This is semantically exactly like test2.el, only the call to
`stream-flush' has been replaced by literally writing out the
definition.  Nonetheless, the compiled file suddenly doesn't crash Emacs
when loaded.  Loading the uncompiled file test3.el still crashes.

Seems that whether we get a crash or not depends on details in the
implementation of lexical-binding.

Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 10:45:06 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 01 05:45:06 2018
Received: from localhost ([127.0.0.1]:38255 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1erLhw-00005r-T4
	for submit <at> debbugs.gnu.org; Thu, 01 Mar 2018 05:45:05 -0500
Received: from dancol.org ([96.126.100.184]:50764)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dancol@HIDDEN>) id 1erLhu-00005j-Sg
 for 30626 <at> debbugs.gnu.org; Thu, 01 Mar 2018 05:45:03 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org;
 s=x; 
 h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
 bh=47QOjUHy9W4RZVnnFpob9g/wL0IisNhoraMPdnKwYx8=; 
 b=EKGrQ9UcEqHk9Pr+BMTmoF9v/n+/nqogSBWcd186s4GxOMGPZugy0ajJsTQQ+s8D2D2p9pEHWrZ/PT6uMInuzzrh4ydbTnQAsoe5rbu32zLi8QIyXspqezqB3//dY76JiDqhaLjJiuLb4LCfE2p+0ntEYh8LPHMSjQwEqNs3mzYTisP4uHK4dLrcuUVTA9WweHL+WGBa9tVW5BiceBh4ON5MsElyfx6KoiT5byfnpwewg4I6UFjwpk7sVOyg1POGtGwRHZvpv+/djkEaNjkuBR704RTZq3kqZ34mVFfpbMprEgqq5tD3xEHsIEJ8Zk/RcfBH0QafHaEmjpfCXLRjlw==;
Received: from [172.92.145.124] (helo=[192.168.86.27])
 by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.84_2) (envelope-from <dancol@HIDDEN>)
 id 1erLht-00015c-7W; Thu, 01 Mar 2018 02:45:01 -0800
Subject: Re: bug#30626: 26.0.91; Crash when traversing a
 `stream-of-directory-files'
To: Eli Zaretskii <eliz@HIDDEN>, Michael Heerdegen <michael_heerdegen@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
From: Daniel Colascione <dancol@HIDDEN>
Message-ID: <5d5ccb32-434a-cda9-67c4-c60abeb450df@HIDDEN>
Date: Thu, 1 Mar 2018 02:44:54 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <83zi3uz4nb.fsf@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

On 02/27/2018 10:08 AM, Eli Zaretskii wrote:
>> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
>> Cc: bug-gnu-emacs@HIDDEN,  30626 <at> debbugs.gnu.org
>> Date: Tue, 27 Feb 2018 13:08:59 +0100
>>
>> #+begin_src emacs-lisp
>> (seq-doseq (_ (stream-range 1 1000000)) nil)
>> #+end_src
>>
>> Note that this is executed as a loop due how to streams are implemented,
>> although the definition of `seq-doseq' looks recursive.  But it seems
>> that gc has a problem with the large number of conses created when
>> processing that.
> 
> What can we do instead in such cases?  Stack-overflow protection
> cannot work in GC, so you are shooting yourself in the foot by
> creating such large recursive structures.  By the time we get to GC,
> where the problem will happen, it's too late, because the memory was
> already allocated.
> 
> Does anyone has a reasonable idea for avoiding the crash in such
> programs?

We need to fix GC being deeply recursive once and for all. Tweaking 
stack sizes on various platforms and trying to spot-fix GC for the 
occasional deeply recursive structure is annoying. Here's my proposal:

Turn garbage_collect_1 into a queue-draining loop, initializing the 
object queue with the GC roots before draining it. We'll make 
mark_object put an object on this queue, turning the existing 
mark_object code into a mark_queued_object function.

garbage_collect_1 will just call mark_queued_object in a loop; 
mark_queued_object can call mark_object, but since mark_object just 
enqueues an object and doesn't recurse, we can't exhaust the stack with 
deep object graphs. (We'll repurpose the mark bit to mean that the 
object is on the to-mark queue; by the time we fully drain the queue, 
just before we sweep, the mark bit will have the same meaning it does now.)

We can't allocate memory to hold the queue during GC, so we'll have to 
pre-allocate it. We can implement the queue as a list of queue blocks, 
where each queue block is an array of 16k or so Lisp_Objects. During 
allocation, we'll just make sure we have one Lisp_Object queue-block 
slot for every non-self-representing Lisp object we allocate.

Since we know that we'll have enough queue blocks for the worst GC case, 
we can have mark_object pull queue blocks from a free list, aborting if 
for some reason it ever runs out of queue blocks. (The previous 
paragraph guarantees we won't.) garbage_collect_1 will churn through 
these heap blocks and place each back on the free list after it's called 
mark_queued_object on every Lisp_Object in the queue block.

In this way, in non-pathological cases of GC, we'll end up using the 
same few queue blocks over and over. That's a nice optimization, because 
we can MADV_DONTNEED unused queue blocks so the OS doesn't actually have 
to remember their contents.

In this way, I think we can make the current GC model recursion-proof 
without drastically changing how we allocate Lisp objects. The 
additional memory requirements should be modest: it's basically one 
Lisp_Object per Lisp object allocated.

The naive version of this scheme needs about 4.6MB of overhead on my 
current 20MB Emacs heap, but it should be possible to reduce the 
overhead significantly by taking advantage of the block allocation we do 
for conses and other types --- we can put whole blocks on the queue 
instead of pointers to individual block parts, so we can get away with a 
much smaller queue. Under this approach, the reserved-queue-block scheme 
would impose an overhead of somewhere around 1MB on the same heap. This 
amount of overhead seems reasonable. We may end up actually using less 
memory that we would for recursive mark_object stack invocation.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 18:25:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 28 13:25:55 2018
Received: from localhost ([127.0.0.1]:37823 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1er6QM-00075X-Ta
	for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 13:25:55 -0500
Received: from mout.web.de ([212.227.17.11]:47525)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1er6QL-00075H-A1
 for 30626 <at> debbugs.gnu.org; Wed, 28 Feb 2018 13:25:53 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103
 [213.165.67.124]) with ESMTPSA (Nemesis) id 0LcPWs-1eOsfc0tz4-00jt5s; Wed, 28
 Feb 2018 19:25:46 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
 <837eqxyqoe.fsf@HIDDEN>
Date: Wed, 28 Feb 2018 19:25:45 +0100
In-Reply-To: <837eqxyqoe.fsf@HIDDEN> (Eli Zaretskii's message of "Wed, 28 Feb
 2018 19:22:57 +0200")
Message-ID: <87lgfdnf86.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:FQOoquDeG4pc/xAgxhTQ+VVq2u7gGBHnFkClfxYgWmEVuFFO/R9
 qGgJaD9VT2yFVYHaSGE6mFKZf51li+x5GsGLS7opNr5LRUAIsmofd6IZqlOFWVnaMS5iXuo
 CnN9eeauaCB3yR4NNMDTSs4kmWwOydqBoY5+rbEWOtc07wAvqvO0ESSubip1nwXyK41Emm+
 lgh5fv7zsVziXkDUiM3sw==
X-UI-Out-Filterresults: notjunk:1;V01:K0:wbTGeAldK6k=:TN5Noghk3nCdfVvZxzU2nY
 +44piVPayzpMlqAZ+NnR52oUGfqzszrFNI2h4y1lV7Uq6dlEA1OErERe4vkHENsCWIX0xK+s/
 ZPVN1noaU4PzKSzLmMAp9bZmSkDKXMxGYf1DGJi0WI2ZNg7n0Yu4+rvPNCHuQY890ZTxQk4EA
 etCvceIpq4/oQgZtscjFFakO89ct31MNj+ytwSQ/laRYcBpkzJBApVjuuF0EINUGOtfks/BaW
 C2SZsNZIPon1RZXbxmlE+mmVWmLMu04xLXHoYsOhNaM2kZjgRaourf1AQJMan8DS/mLaf0vii
 p2H3Ya1BVdPmENYeUPevv+f9GqPeau4BhPZGC79+ECt0axPLNOOcl2+16Mb/9DsKuynT1m9eO
 Yxeeqyg7x9GAMZIjdTDUQdsCVFcaPR3+TKUpZaK5MURCQuiE+cjrqYrqeNXuY3oEcTWJsp/Ln
 hHGuVsVRzKgLyiqJx/T7x09yd8p4A0+ER4v/T3XYkaq2o/V7Sl44fWpQvYQwUWGusAMlql8JJ
 g0Mn6SrIMfzH/9yvaMYMA0M4HiSU8a3RBl01LBzP409YDEjuMWM7rK7VQWsMKxU6t0a4DFTR1
 j41CZAPXv1AeKsaU9N9hv1ItFCK4B9mS4+IfObH249nM/e09fIrBBXugFIcDuMqB+rNvlsYjQ
 T1gW9/0uRYzhaYhOymYGwNVdvJc8HAxhobDsC4tsDj0xc+9vm5oI0AEDfcsEccMqQClISGGeg
 MPuc7V/yhjAgN1jEl9L6vbBbMIs3BpdbsZEkJBxpyxMFNqqfK+Hoz/GM/7xbryTLAbWlhUcz0
 2fgpTsb6nIvgh2u1rRxyFlQ1C/QRBScgDqo2vZzA2R67ZivJh4=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: npostavs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

> I don't think it's thrown away from the POV of GC.  But you can easily
> see what is going on if you trace the GC on the C level.  You should
> be able to see which object causes recursion in mark_object.  I didn't
> look long enough, but what I did see looks very much like the entire
> unwound stream.

I have an idea what could be going on.  In the stream-range example,
this is how the stream is build:

#+begin_src emacs-lisp
(list stream--identifier
      (let
          (forced val)
        (lambda
          (&optional check)
          (if
              check
              forced
            (unless
                forced
              (setf
               val
               (progn
                 (cons
                  start
                  (stream-range (+ start step) end step))))
              (setf forced t))
            val))))
#+end_src

The inner `stream-range' call results in a closure, and I guess that
this closure includes a reference to the outside VAL, which is the
stream from one step back (though there isn't a lexical reference to the
variable...does that make sense?)

So there could be a chain of references via closure variables back to
the first cons.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 17:23:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 28 12:23:01 2018
Received: from localhost ([127.0.0.1]:37745 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1er5RV-0003bb-BV
	for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 12:23:01 -0500
Received: from eggs.gnu.org ([208.118.235.92]:34823)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1er5RT-0003bK-Fp
 for 30626 <at> debbugs.gnu.org; Wed, 28 Feb 2018 12:22:59 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1er5RK-000087-8e
 for 30626 <at> debbugs.gnu.org; Wed, 28 Feb 2018 12:22:54 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33348)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1er5RK-00007y-52; Wed, 28 Feb 2018 12:22:50 -0500
Received: from [176.228.60.248] (port=3566 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1er5RJ-0004Mq-Fm; Wed, 28 Feb 2018 12:22:49 -0500
Date: Wed, 28 Feb 2018 19:22:57 +0200
Message-Id: <837eqxyqoe.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
In-reply-to: <87po4pnl0a.fsf@HIDDEN> (message from Michael Heerdegen on Wed,
 28 Feb 2018 17:20:53 +0100)
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN> <87po4pnl0a.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 30626
Cc: npostavs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
> Cc: npostavs@HIDDEN,  30626 <at> debbugs.gnu.org
> Date: Wed, 28 Feb 2018 17:20:53 +0100
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> > How do you see that the car is immediately thrown away?
> 
> After expansion and dispatch, this is what gets executed:
> 
> #+begin_src emacs-lisp
> (let ((stream (stream-range 1 1000000)))
>   (while (not (stream-empty-p stream))
>     (funcall #'ignore (stream-first stream))
>     (setq stream (stream-rest stream))))
> #+end_src
> 
> Creating an element and throwing away the outermost cons alternate.

I don't think it's thrown away from the POV of GC.  But you can easily
see what is going on if you trace the GC on the C level.  You should
be able to see which object causes recursion in mark_object.  I didn't
look long enough, but what I did see looks very much like the entire
unwound stream.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 16:21:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 28 11:21:03 2018
Received: from localhost ([127.0.0.1]:37713 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1er4TX-0002B1-3V
	for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 11:21:03 -0500
Received: from mout.web.de ([212.227.17.12]:47611)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1er4TV-0002AA-2w
 for 30626 <at> debbugs.gnu.org; Wed, 28 Feb 2018 11:21:01 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103
 [213.165.67.124]) with ESMTPSA (Nemesis) id 0Lh6M7-1eKlv70MxL-00oa3f; Wed, 28
 Feb 2018 17:20:54 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
 <83h8q1yuin.fsf@HIDDEN>
Date: Wed, 28 Feb 2018 17:20:53 +0100
In-Reply-To: <83h8q1yuin.fsf@HIDDEN> (Eli Zaretskii's message of "Wed, 28 Feb
 2018 18:00:00 +0200")
Message-ID: <87po4pnl0a.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:Ohahmb63bDqc7lJ9F6Fsbm80aC/ngB0OcuC0WGRBvRADeWZDWfP
 InjDCSJwUoKwxkUrUTMLiZxgklaH3wrPyOz1qieA86r1EqYXlrcAlVrHSVj0xwUOQRfTpjm
 w1x0ai9u5caXOg6nrcekOn8Nve3yim4LtTHC5QhdRsulyx5s7FPuN80r15ng9mFeiEMvAoI
 gkg3rDcHrAvWKhzw7yt0A==
X-UI-Out-Filterresults: notjunk:1;V01:K0:O7gTCDHOFlw=:O8LtxidCs/Yr/pJOSMYZ+S
 f+5g/57N+32P6GyqmX5CZXiLbTWSAQVTdxiPwN5YvD3BQEuZSbq0P58wxxQ0DiiKGU0ixh63y
 TUEGWba6hr8U012vMXWZKxMemREPwUBomY/J5TjV1nBZhpg/6hVnr5Q2F6U+oU2OUeInj+lHS
 cZYdS4JAnhmxA8vOxgHNCch2W3IGCTEhDnbnuCxjYK22c7xnxBHh4y7yMCPFQxUSj4avzd+ju
 283gf+NGVhb5ZMFSy/WRTHOkmeL2q1dQiYjD++XeBFNrugVs5nNpPyJuRqscfbFdZ1q4il+lb
 /jfFdtDAYzsqXY0sz6mVd8RTKK5Cd9/xqC1AZEOFhkObY12zJAnvQqkKNTUia8rTwgdwFEWqh
 eIvxhHJk5veBXr5U7J8zjbc5+a0Yanw9hS/5r1McUgYBwbEus18vNZOoGe0sNemaDyF7LBklS
 cqNKrGKE1vaHKJrCpNuy9AZmHWdeXXG/ncOSvQ+vv8qJ7G9enumcgiFmVscxZcKBJOyHGiIJo
 CvQgauXzGSHH3vr2hQBhWta/t22QHZkRcbd32EDkYHjfxjZ1HzVgsxSnHK18zO131oP7Edeq1
 7YbfsNuHpGhfAlVDKiRf/P7LM8Tt+EopYti2oXy1XWw9UDHxTg1ppXuXGCNksSm4UTYA+SrVL
 QGwbVRVGIEYuvrS5teomlf7r54wWiq5UPyEIgyOQdwz0AutD+zC2wloddjBChLtCY+YrnHusX
 Io3TA3TKo8TD/IMSK27S/8OGoBgTMvot7/S/oNfhlpErb11vaG4UsTA6BHyW4EMjewO0Fd0jD
 FPVw/4UCdSuiJIG+5YDXYVtpJ8jTLE3plpRtFJGAQxY2rQrbKM=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: npostavs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

> How do you see that the car is immediately thrown away?

After expansion and dispatch, this is what gets executed:

#+begin_src emacs-lisp
(let ((stream (stream-range 1 1000000)))
  (while (not (stream-empty-p stream))
    (funcall #'ignore (stream-first stream))
    (setq stream (stream-rest stream))))
#+end_src

Creating an element and throwing away the outermost cons alternate.

> > AFAIU the problem is that the cons1 still exists in memory until
> > garbage collection kicks in.  When that happens, the cons1 points to
> > a largely recursive cons structure, though this structure is never
> > referenced from Lisp in this form.

> What is "cons1" in this context?

I said it some lines above: I defined it as name of the "first cons",
i.e. the original stream.  The return value of `stream-range' in the
above example.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 16:00:06 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 28 11:00:06 2018
Received: from localhost ([127.0.0.1]:37686 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1er49G-0001eb-8T
	for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 11:00:06 -0500
Received: from eggs.gnu.org ([208.118.235.92]:42348)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1er49C-0001d2-Rw
 for 30626 <at> debbugs.gnu.org; Wed, 28 Feb 2018 11:00:04 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1er492-0006dq-OJ
 for 30626 <at> debbugs.gnu.org; Wed, 28 Feb 2018 10:59:57 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59666)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1er492-0006dh-LR; Wed, 28 Feb 2018 10:59:52 -0500
Received: from [176.228.60.248] (port=3444 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1er491-0007wm-Vu; Wed, 28 Feb 2018 10:59:52 -0500
Date: Wed, 28 Feb 2018 18:00:00 +0200
Message-Id: <83h8q1yuin.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
In-reply-to: <87bmg91ity.fsf@HIDDEN> (message from Michael Heerdegen on Wed,
 28 Feb 2018 11:58:49 +0100)
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN> <87bmg91ity.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 30626
Cc: npostavs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
> Cc: Eli Zaretskii <eliz@HIDDEN>,  30626 <at> debbugs.gnu.org
> Date: Wed, 28 Feb 2018 11:58:49 +0100
> 
> >     (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST)
> 
> Yes, that's exactly how it's implemented.  When requesting more elements
> from the stream, that becomes
> 
>       (FIRST-VALUE .
>         (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST))
> 
> When we loop over the string, the cons whose car is the FIRST-VALUE,
> let's call it cons1, is immediately thrown away, and we continue with
> 
>       (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST)
> 
> etc.

How do you see that the car is immediately thrown away?

> AFAIU the problem is that the cons1 still exists in memory until garbage
> collection kicks in.  When that happens, the cons1 points to a largely
> recursive cons structure, though this structure is never referenced from
> Lisp in this form.

What is "cons1" in this context?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 28 Feb 2018 13:21:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 28 08:21:08 2018
Received: from localhost ([127.0.0.1]:36598 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1er1fQ-00069z-Jj
	for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 08:21:08 -0500
Received: from eggs.gnu.org ([208.118.235.92]:56218)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <nicolas@HIDDEN>) id 1er1fO-00069F-V4
 for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 08:21:07 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <nicolas@HIDDEN>) id 1er1fE-0004Ch-5v
 for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 08:21:01 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:41085)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <nicolas@HIDDEN>) id 1er1fE-0004CO-2d
 for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 08:20:56 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47148)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <nicolas@HIDDEN>) id 1er1fD-0003x1-5r
 for bug-gnu-emacs@HIDDEN; Wed, 28 Feb 2018 08:20:55 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <nicolas@HIDDEN>) id 1er1f9-00049z-6m
 for bug-gnu-emacs@HIDDEN; Wed, 28 Feb 2018 08:20:55 -0500
Received: from petton.fr ([89.234.186.68]:52614)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <nicolas@HIDDEN>)
 id 1er1f8-00048i-S9; Wed, 28 Feb 2018 08:20:51 -0500
From: Nicolas Petton <nicolas@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
In-Reply-To: <877eqx1ije.fsf@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN> <877eqx1ije.fsf@HIDDEN>
Date: Wed, 28 Feb 2018 14:20:42 +0100
Message-ID: <87lgfdgsid.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=petton.fr; s=mail;
 t=1519824047; bh=QVJE2Hb70cRVPaubh5SkfPcFC94tDbC9dN4Qi21ophg=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID:MIME-Version:Content-Type;
 b=WnmSe/ZLvi1EdJVngoRzUZ24AtXAiJN+60+4b8f/9i8ksB3WnyEqG8qT0DsTvu5jWSCpB8mUB+9dB9J52VEoBbGfyZso6/68guFTdz9Qvj5WcO84IY4iLIPE/AmHJpL3rJ0Lb3daUEwI2QrEOSR5pZ04rPdi0CDOYYMcTjgjhMk=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
 [fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.1 (----)
X-Debbugs-Envelope-To: submit
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.1 (----)

--=-=-=
Content-Type: text/plain

Michael Heerdegen <michael_heerdegen@HIDDEN> writes:

> Hello,

Hi,

> I had written that
>
>> > (seq-doseq (_ (stream-range 1 1000000)) nil)
>
> crashes.  CC'ing Nicolas, the author of stream.el.

Thanks, I'll look into it.

Cheers,
Nico

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE1AWqLIYsVPF+7mvg6LzXhmr8+XgFAlqWrKoACgkQ6LzXhmr8
+XgcqQgAzgY9bRiXsLRNLB+SxUyuC/b4+hHlfkaYzRq6M9gw5X1slBHNaiTJErpM
Hf9NWnw8HCQ2qCrcyLQrrvYnQ9OgPJic1nxf4znNWHhH/y+ryKpNhKloCwyuzRVW
44z6TFIM5tsowpYoYeY9yjSNlGu/VOxAu55wPX6gMPnNwsxyZoonzxOsgwehej/F
snJTG2EUlUTwVp0FRRl1qES+OUvZ3QGTPRX0r9kMNfjFj36UH0jCaETRTvh76uMR
3WM949/8AuPNfH81V4zmpjgH3z+1ODwoXmBtpBJcV09hqs457502BnlENCOE2pK5
z+P6t3oSy3KMSTZGyFfzBWVaycN/nA==
=mII8
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 13:20:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 28 08:20:56 2018
Received: from localhost ([127.0.0.1]:36594 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1er1fE-00068q-Bs
	for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 08:20:56 -0500
Received: from petton.fr ([89.234.186.68]:54004)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <nicolas@HIDDEN>) id 1er1fC-00068a-0n
 for 30626 <at> debbugs.gnu.org; Wed, 28 Feb 2018 08:20:54 -0500
From: Nicolas Petton <nicolas@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
In-Reply-To: <877eqx1ije.fsf@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN> <877eqx1ije.fsf@HIDDEN>
Date: Wed, 28 Feb 2018 14:20:42 +0100
Message-ID: <87lgfdgsid.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=petton.fr; s=mail;
 t=1519824047; bh=QVJE2Hb70cRVPaubh5SkfPcFC94tDbC9dN4Qi21ophg=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID:MIME-Version:Content-Type;
 b=WnmSe/ZLvi1EdJVngoRzUZ24AtXAiJN+60+4b8f/9i8ksB3WnyEqG8qT0DsTvu5jWSCpB8mUB+9dB9J52VEoBbGfyZso6/68guFTdz9Qvj5WcO84IY4iLIPE/AmHJpL3rJ0Lb3daUEwI2QrEOSR5pZ04rPdi0CDOYYMcTjgjhMk=
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

--=-=-=
Content-Type: text/plain

Michael Heerdegen <michael_heerdegen@HIDDEN> writes:

> Hello,

Hi,

> I had written that
>
>> > (seq-doseq (_ (stream-range 1 1000000)) nil)
>
> crashes.  CC'ing Nicolas, the author of stream.el.

Thanks, I'll look into it.

Cheers,
Nico

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE1AWqLIYsVPF+7mvg6LzXhmr8+XgFAlqWrKoACgkQ6LzXhmr8
+XgcqQgAzgY9bRiXsLRNLB+SxUyuC/b4+hHlfkaYzRq6M9gw5X1slBHNaiTJErpM
Hf9NWnw8HCQ2qCrcyLQrrvYnQ9OgPJic1nxf4znNWHhH/y+ryKpNhKloCwyuzRVW
44z6TFIM5tsowpYoYeY9yjSNlGu/VOxAu55wPX6gMPnNwsxyZoonzxOsgwehej/F
snJTG2EUlUTwVp0FRRl1qES+OUvZ3QGTPRX0r9kMNfjFj36UH0jCaETRTvh76uMR
3WM949/8AuPNfH81V4zmpjgH3z+1ODwoXmBtpBJcV09hqs457502BnlENCOE2pK5
z+P6t3oSy3KMSTZGyFfzBWVaycN/nA==
=mII8
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 11:05:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 28 06:05:19 2018
Received: from localhost ([127.0.0.1]:36505 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqzXz-0000nf-Jp
	for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 06:05:19 -0500
Received: from mout.web.de ([212.227.15.4]:36211)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqzXy-0000nH-7v
 for 30626 <at> debbugs.gnu.org; Wed, 28 Feb 2018 06:05:18 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb003
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LsyOA-1ejnga1o8w-012ajU; Wed, 28
 Feb 2018 12:05:10 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
Date: Wed, 28 Feb 2018 12:05:09 +0100
In-Reply-To: <83zi3uz4nb.fsf@HIDDEN> (Eli Zaretskii's message of "Tue, 27 Feb
 2018 20:08:56 +0200")
Message-ID: <877eqx1ije.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:JP1ZoJ4r0VJOg7NYJUFhT1Xf+OPMlaSkBQNECuVUQ9QcYHn46TX
 vNsQ6+d0lwpOVExciCBiMClF34B3vE67VDPKVup+A/6cSp1gfo83DZtLBnl1UA13EFZDL38
 cdGzLgtZWLaXa+88kPoRdM7iTn67tCBrFqJIaU0w5p30ULMXPzkqteX6gj9q6/1ULfpnw1n
 vWJbEE96Rqy/yJCwT50nA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:rR/WE+bKzkM=:9oRVS4zbtmFjh/aqHtnhxe
 z+CPZGipHXLDaxBenf/In8FJ3d0oicNdPy6Sl/CosY09ppS3xkszYrTWtdL5UTxxcFouN2J0v
 MBpNtKmP+Zk3nQKTg8a/JnVFHFDclxOtwrdDE0awnjI6eXsoqia2WvoPJA2j5SHRjvyFJxop9
 rblnf275a2ZuLgK6R+5wcnANktKgNcTMh0+PD6QQGYbbp8NlDBo4ExfQa2E69EPoq77rbYsIT
 40U3Apqkuylp8D1v0ckRMqLbZoiF/1tL9eTEFJzuM5vNEdLpW6HltPuAL6qqlnrxVDgvVYFxu
 mEkeDIyfgd1de1oI6bPKeZ4TMfRL+EQJBg4Tj8asl3SNWcM+TZc4fud+7HhajVPl7IC46CR+y
 7XjuhmAiuyVfIVTB8WUuJVPs27Z8ZaDLeW0DVxQyyCXI5c4GUNgBRGyMhQNDzOPNG0fShceVn
 Rdraxh8izw3M4A6DTo580oiSwO39F9b47iY5Gqdpl865FQA+Xvrbzi2+L7hgw79X/Y3iKPzuG
 4TxbtNb9Ej5RksyhHJfScPgVe7Dv7Fo/p7ghcTnTjDD1N+Rb77u82xDJ4lIkIVBN7uV0PiZZO
 BgcLAbyD/mRNqLeKfHZtVtk03ApUlo4KwSsnYhLskEU8M71UTYRIsh8z/dmlbdQIU4ffP1Lo5
 x3t48iszmHtRsl8+VJAnHOwCCHH0yhNZy1if4rCfdo2c9CQoa/p7KR84WbIU5TG/kZ/GZC9rH
 p9lxo702COHPfyNaE1FLjLkCRWNXSwMSm3joGAA6uAaBLBY/e40tRJC5y2rm8Gueaxkt7/Vno
 xfEUwMqVCbBy5o/0wHNgkkC08PyPldD6eyGM1R+1RxQRZXNMC8=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: bug-gnu-emacs@HIDDEN, Nicolas Petton <nicolas@HIDDEN>,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Hello,

I had written that

> > (seq-doseq (_ (stream-range 1 1000000)) nil)

crashes.  CC'ing Nicolas, the author of stream.el.

> What can we do instead in such cases?  Stack-overflow protection
> cannot work in GC, so you are shooting yourself in the foot by
> creating such large recursive structures.  By the time we get to GC,
> where the problem will happen, it's too late, because the memory was
> already allocated.
>
> Does anyone has a reasonable idea for avoiding the crash in such
> programs?

I would appreciate any effort to fix that, because it seems that
currently streams are broken by design, and there is no way to fix that
from the Lisp implementation.

Yes, we could implement iterators instead of streams - that's what we
get when we avoid the consing.  But it's something different and not
always an alternative, depending on what you want to do.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 28 Feb 2018 11:05:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 28 06:05:38 2018
Received: from localhost ([127.0.0.1]:36508 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqzYH-0000oE-Rf
	for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 06:05:37 -0500
Received: from eggs.gnu.org ([208.118.235.92]:49897)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqzYG-0000o1-Gi
 for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 06:05:36 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqzY9-0000b1-5n
 for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 06:05:31 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:54930)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <michael_heerdegen@HIDDEN>)
 id 1eqzY9-0000au-2B
 for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 06:05:29 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:40861)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqzY7-0006Fk-Sn
 for bug-gnu-emacs@HIDDEN; Wed, 28 Feb 2018 06:05:28 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqzY6-0000aN-Vu
 for bug-gnu-emacs@HIDDEN; Wed, 28 Feb 2018 06:05:27 -0500
Received: from mout.web.de ([212.227.15.4]:56337)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <michael_heerdegen@HIDDEN>)
 id 1eqzY0-0000Vg-5X; Wed, 28 Feb 2018 06:05:20 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb003
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LsyOA-1ejnga1o8w-012ajU; Wed, 28
 Feb 2018 12:05:10 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
Date: Wed, 28 Feb 2018 12:05:09 +0100
In-Reply-To: <83zi3uz4nb.fsf@HIDDEN> (Eli Zaretskii's message of "Tue, 27 Feb
 2018 20:08:56 +0200")
Message-ID: <877eqx1ije.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:JP1ZoJ4r0VJOg7NYJUFhT1Xf+OPMlaSkBQNECuVUQ9QcYHn46TX
 vNsQ6+d0lwpOVExciCBiMClF34B3vE67VDPKVup+A/6cSp1gfo83DZtLBnl1UA13EFZDL38
 cdGzLgtZWLaXa+88kPoRdM7iTn67tCBrFqJIaU0w5p30ULMXPzkqteX6gj9q6/1ULfpnw1n
 vWJbEE96Rqy/yJCwT50nA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:rR/WE+bKzkM=:9oRVS4zbtmFjh/aqHtnhxe
 z+CPZGipHXLDaxBenf/In8FJ3d0oicNdPy6Sl/CosY09ppS3xkszYrTWtdL5UTxxcFouN2J0v
 MBpNtKmP+Zk3nQKTg8a/JnVFHFDclxOtwrdDE0awnjI6eXsoqia2WvoPJA2j5SHRjvyFJxop9
 rblnf275a2ZuLgK6R+5wcnANktKgNcTMh0+PD6QQGYbbp8NlDBo4ExfQa2E69EPoq77rbYsIT
 40U3Apqkuylp8D1v0ckRMqLbZoiF/1tL9eTEFJzuM5vNEdLpW6HltPuAL6qqlnrxVDgvVYFxu
 mEkeDIyfgd1de1oI6bPKeZ4TMfRL+EQJBg4Tj8asl3SNWcM+TZc4fud+7HhajVPl7IC46CR+y
 7XjuhmAiuyVfIVTB8WUuJVPs27Z8ZaDLeW0DVxQyyCXI5c4GUNgBRGyMhQNDzOPNG0fShceVn
 Rdraxh8izw3M4A6DTo580oiSwO39F9b47iY5Gqdpl865FQA+Xvrbzi2+L7hgw79X/Y3iKPzuG
 4TxbtNb9Ej5RksyhHJfScPgVe7Dv7Fo/p7ghcTnTjDD1N+Rb77u82xDJ4lIkIVBN7uV0PiZZO
 BgcLAbyD/mRNqLeKfHZtVtk03ApUlo4KwSsnYhLskEU8M71UTYRIsh8z/dmlbdQIU4ffP1Lo5
 x3t48iszmHtRsl8+VJAnHOwCCHH0yhNZy1if4rCfdo2c9CQoa/p7KR84WbIU5TG/kZ/GZC9rH
 p9lxo702COHPfyNaE1FLjLkCRWNXSwMSm3joGAA6uAaBLBY/e40tRJC5y2rm8Gueaxkt7/Vno
 xfEUwMqVCbBy5o/0wHNgkkC08PyPldD6eyGM1R+1RxQRZXNMC8=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
 [fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.1 (----)
X-Debbugs-Envelope-To: submit
Cc: bug-gnu-emacs@HIDDEN, Nicolas Petton <nicolas@HIDDEN>,
 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.1 (----)

Hello,

I had written that

> > (seq-doseq (_ (stream-range 1 1000000)) nil)

crashes.  CC'ing Nicolas, the author of stream.el.

> What can we do instead in such cases?  Stack-overflow protection
> cannot work in GC, so you are shooting yourself in the foot by
> creating such large recursive structures.  By the time we get to GC,
> where the problem will happen, it's too late, because the memory was
> already allocated.
>
> Does anyone has a reasonable idea for avoiding the crash in such
> programs?

I would appreciate any effort to fix that, because it seems that
currently streams are broken by design, and there is no way to fix that
from the Lisp implementation.

Yes, we could implement iterators instead of streams - that's what we
get when we avoid the consing.  But it's something different and not
always an alternative, depending on what you want to do.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 10:58:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Feb 28 05:58:59 2018
Received: from localhost ([127.0.0.1]:36478 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqzRr-0000cM-Cb
	for submit <at> debbugs.gnu.org; Wed, 28 Feb 2018 05:58:59 -0500
Received: from mout.web.de ([212.227.17.12]:50553)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqzRp-0000c7-1h
 for 30626 <at> debbugs.gnu.org; Wed, 28 Feb 2018 05:58:57 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103
 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MOzoR-1elEYY3buB-006PZ8; Wed, 28
 Feb 2018 11:58:50 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Noam Postavsky <npostavs@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
 <87lgfd52by.fsf@HIDDEN>
Date: Wed, 28 Feb 2018 11:58:49 +0100
In-Reply-To: <87lgfd52by.fsf@HIDDEN> (Noam Postavsky's message of "Tue, 27
 Feb 2018 20:29:21 -0500")
Message-ID: <87bmg91ity.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:XkQ6Dh1kC+Kyw/7Jvu7xd3wRLLW9ULo+7fVx1kw4mvGWdh2aLb1
 q3PukxVyQ9NfQcU2PS0sYCxUiVLO9mY+OND2o38Vdc8M3FAdlyD/KrLFtwWUzNquEShTK8q
 hREyRTtNdfzP4lXRz2Fsk3e6s/xqYgbUzHQEXHJ+QnkCWPixefVAKI3DNdczItkD7RBtaqV
 2Q1/uUGfFUtVi4rB+jzaw==
X-UI-Out-Filterresults: notjunk:1;V01:K0:MSX261JF4Nk=:9myiQqNSVT3vSAAAi0VHVA
 tYUYEh9PtaTrt7p9roq3PwsyHA6JZYsa4u+ebygitFxrQiCeTMk0xnTdVAs0wy1buaWaGzrWr
 ieNaGej1jwYjoAqqg6aOT0KnUfT8jtkywqLB0ozCGMX7OUpOEgELMGzfKclB35AixyyR3VFk7
 YiRjB43ZUBo22pPMQ+Xv4LV+Kd7N3DgQ1Aiz3BfC2mNMKV8VfKJuuLnfbLAvKKYlaOhc5a+TB
 drcEEImqQ7p7c/tto1AAXWHW4e2D8TNS9u7++Z/eD9xWiZnAYZ9jXSzfrwpcQGYuT1tbHzZWw
 G/GVa6uIWS62KC7kxSOMSUYPSLIPzV1QwgNzfsFg1ajRvVrhXwJcMWwBJpeP/kPxZl/2jZ7UC
 Jk+X8fxCOKEKmHS3vr+2FmRX8uk6xj8CXSf8XLeMA75vJfRuF+A5ILVFTGKvzIB5/VFd/AllF
 WuVZLXEqtq63ao3CxCe0haMz/qLx82DaXmKLvrn3cE7dJ5PeQOcNK9bR16qOW7ycV+RQ93K6C
 sYf7MBypSjQ78RXSxlbR81/qxRIeBVJ6rJsm8ytUp7R+FR7xZs2i+tSdEiOZVS7FBOG/gJZjo
 ZxlnAmCk4xdYm6n4kthT2YZsrqMzepq4I+2uKUSZDtMu+TY7+MJvymrXHEkBGAqbZokmUhzDG
 08KSOlsJeEwxM/3/tc2j1NYMg502+KiWW9qaUunZ3k856/5HSuqZBS1NiHJZI+TOfUFhDws2k
 fG85Wda9azYAA0gsETqlzOzActd6tOjMkEtaqLH73DEtOllCYqLrxauV4ZDaw93bLIAn2t2Y0
 yM9oT7bdcTarpRSkNQP4lLZbUDFMFsCaQ8yFYQAWE5vsI862sI=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: Eli Zaretskii <eliz@HIDDEN>, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Noam Postavsky <npostavs@HIDDEN> writes:

> >> #+begin_src emacs-lisp
> >> (seq-doseq (_ (stream-range 1 1000000)) nil)
> >> #+end_src
> >> 
> >> Note that this is executed as a loop due how to streams are
> >> implemented, although the definition of `seq-doseq' looks
> >> recursive.
>
> Doesn't look recursive to me, it expands to a call to seq-do, which uses
> a simple loop.

I was imprecise, I meant that the streams are defined recursively (most
of the time).  Though, it's a delayed type of recursion.  Anyway, I
think that this doesn't matter here.

> > Does anyone has a reasonable idea for avoiding the crash in such
> > programs?
>
> I don't have a quick answer for the general case, but I think it's a bug
> in stream.el that it's creating such large structures in the first
> place.  As far as I understand it, the point of streams is to handle
> long lists by encoding them as
>
>     (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST)

Yes, that's exactly how it's implemented.  When requesting more elements
from the stream, that becomes

      (FIRST-VALUE .
        (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST))

When we loop over the string, the cons whose car is the FIRST-VALUE,
let's call it cons1, is immediately thrown away, and we continue with

      (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST)

etc.

AFAIU the problem is that the cons1 still exists in memory until garbage
collection kicks in.  When that happens, the cons1 points to a largely
recursive cons structure, though this structure is never referenced from
Lisp in this form.
  
> so as to avoid allocating large amounts of memory.  Is there an easy way
> to find out what the large structures are, and where they are coming
> from?

I think I've answered that.  At least, I think so.  What I don't
understand is that when I force the `seq-doseq' to call
`garbace-collect' explicitly every 1000 cycles, or so, it doesn't help:
the crash still happens after generating ~ 70 000 elements, or some
more, but I can't avoid the crash, no matter how often I call gc.  So
I'm not sure whether these long lists are the problem or something else.
The FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST looks harmless when I print
it, even after thousands of iterations, so I would not understand why
that could be problematic.  streams.el implements things in a way that
these rest functions are not deeply nested lambdas.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 01:29:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 20:29:29 2018
Received: from localhost ([127.0.0.1]:36267 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqqYj-00022v-Kw
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 20:29:29 -0500
Received: from mail-it0-f49.google.com ([209.85.214.49]:40704)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1eqqYi-00022i-0I
 for 30626 <at> debbugs.gnu.org; Tue, 27 Feb 2018 20:29:28 -0500
Received: by mail-it0-f49.google.com with SMTP id v186so1509606itc.5
 for <30626 <at> debbugs.gnu.org>; Tue, 27 Feb 2018 17:29:27 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=jU8JUjE6W55VY6I37oKWn09TjMLw2bqetg/bhOTki3g=;
 b=b1sFUlg2c/0p86qMHVI0A6G4Cuef0V0lN92r+W1JoLkvH50AEj+HUz32gBIuq0I0IU
 lBG5i1MJV19FATD6xjV6VeXoVgG8g8gPQDxdPeoDWXmX+il7V1uYjNxaCtmtIOgUA1sZ
 atyOtXm0D5rCBufkwp0Hh9iCaxHHPfUI9QwbLYmUoTZynNhjkeqIAv92HCcgwui2mDWw
 PrZ7agsTYoN98tf9jSqu4OsLpdlDOs5j+vE+ugmXUeyPtjqlA7V93p76z0o1O+c75SCq
 37meonZOa+8XSKN5ajzIgISwh0GhCBMoPesoltFYyQNDxg+a+/okl4IxkL6h1TZBs4xk
 NQyA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=jU8JUjE6W55VY6I37oKWn09TjMLw2bqetg/bhOTki3g=;
 b=Uijut9rvzW1/hMqIIgycpcaVSg7/oBu0WdRxzKn2jpt848bwe7xFpADkhIuSjIfJ9x
 LPjj0e/5EA2zjrdw/BBcXKzKMyLJ8pPY4/mghcld1UYs0ZoQpY72MEkm5XziHXWxQzf3
 XWLu5G8CDa9injIwlLT9gKLmmLZsb1d2Zp4gOrloRQOmOHGG/tpHBE88xSQqxtbCLHeS
 VeAUir7+XrBznZhAnl2dnYggac22/x3DG0ywc54svKe3uxyo8dXb4W6C0nIe4H2f2QUw
 03UwY7XeuiY03UE/sHOXlFB+imbz2kgp2oQZNvAnjNj2Dzxno1FsOwIOhm4glEb8OHMa
 FlWg==
X-Gm-Message-State: APf1xPBmX0JPUyobkCYKRubPmLFhpeFFzsF8ju62DUT6VALi64FIMVZ5
 CEqjxH5S4s3dHAQzIsl9j6/2HQ==
X-Google-Smtp-Source: AG47ELtuM8/FMdTKLAc+W7C0nSbU3hvIe8XbADqTWfqrkJJM6x1QT7EIXEhSndLFCCFGyq5ZCnSrfw==
X-Received: by 10.36.144.7 with SMTP id x7mr7071226itd.128.1519781362550;
 Tue, 27 Feb 2018 17:29:22 -0800 (PST)
Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34])
 by smtp.googlemail.com with ESMTPSA id
 m32sm9372369iti.3.2018.02.27.17.29.21
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Tue, 27 Feb 2018 17:29:22 -0800 (PST)
From: Noam Postavsky <npostavs@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN> <83zi3uz4nb.fsf@HIDDEN>
Date: Tue, 27 Feb 2018 20:29:21 -0500
In-Reply-To: <83zi3uz4nb.fsf@HIDDEN> (Eli Zaretskii's message of "Tue, 27 Feb
 2018 20:08:56 +0200")
Message-ID: <87lgfd52by.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 30626
Cc: Michael Heerdegen <michael_heerdegen@HIDDEN>, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

>> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
>> Cc: bug-gnu-emacs@HIDDEN,  30626 <at> debbugs.gnu.org
>> Date: Tue, 27 Feb 2018 13:08:59 +0100
>> 
>> #+begin_src emacs-lisp
>> (seq-doseq (_ (stream-range 1 1000000)) nil)
>> #+end_src
>> 
>> Note that this is executed as a loop due how to streams are implemented,
>> although the definition of `seq-doseq' looks recursive.

Doesn't look recursive to me, it expands to a call to seq-do, which uses
a simple loop.

>> But it seems that gc has a problem with the large number of conses
>> created when processing that.
>
> What can we do instead in such cases?  Stack-overflow protection
> cannot work in GC, so you are shooting yourself in the foot by
> creating such large recursive structures.  By the time we get to GC,
> where the problem will happen, it's too late, because the memory was
> already allocated.
>
> Does anyone has a reasonable idea for avoiding the crash in such
> programs?

I don't have a quick answer for the general case, but I think it's a bug
in stream.el that it's creating such large structures in the first
place.  As far as I understand it, the point of streams is to handle
long lists by encoding them as

    (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST)
 
so as to avoid allocating large amounts of memory.  Is there an easy way
to find out what the large structures are, and where they are coming
from?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 27 Feb 2018 18:09:04 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 13:09:04 2018
Received: from localhost ([127.0.0.1]:35919 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqjgW-0002rJ-FA
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:09:04 -0500
Received: from eggs.gnu.org ([208.118.235.92]:50087)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1eqjgU-0002qY-G0
 for 30626 <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:09:02 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqjgK-0003F8-GJ
 for 30626 <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:08:57 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42715)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1eqjgK-0003F4-CH; Tue, 27 Feb 2018 13:08:52 -0500
Received: from [176.228.60.248] (port=2496 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1eqjgI-0008NF-UI; Tue, 27 Feb 2018 13:08:51 -0500
Date: Tue, 27 Feb 2018 20:08:56 +0200
Message-Id: <83zi3uz4nb.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
In-reply-to: <877eqyocro.fsf@HIDDEN> (message from Michael Heerdegen on Tue,
 27 Feb 2018 13:08:59 +0100)
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 30626
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
> Cc: bug-gnu-emacs@HIDDEN,  30626 <at> debbugs.gnu.org
> Date: Tue, 27 Feb 2018 13:08:59 +0100
> 
> #+begin_src emacs-lisp
> (seq-doseq (_ (stream-range 1 1000000)) nil)
> #+end_src
> 
> Note that this is executed as a loop due how to streams are implemented,
> although the definition of `seq-doseq' looks recursive.  But it seems
> that gc has a problem with the large number of conses created when
> processing that.

What can we do instead in such cases?  Stack-overflow protection
cannot work in GC, so you are shooting yourself in the foot by
creating such large recursive structures.  By the time we get to GC,
where the problem will happen, it's too late, because the memory was
already allocated.

Does anyone has a reasonable idea for avoiding the crash in such
programs?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 18:09:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 13:09:16 2018
Received: from localhost ([127.0.0.1]:35922 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqjgi-0002ri-NF
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:09:16 -0500
Received: from eggs.gnu.org ([208.118.235.92]:50179)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1eqjgg-0002rV-SY
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:09:15 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqjga-0003PT-S6
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:09:09 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:49711)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1eqjga-0003PF-Oz
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:09:08 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:41102)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqjgV-0000D0-Nq
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 13:09:08 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqjgU-0003N1-Qp
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 13:09:03 -0500
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42715)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1eqjgK-0003F4-CH; Tue, 27 Feb 2018 13:08:52 -0500
Received: from [176.228.60.248] (port=2496 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1eqjgI-0008NF-UI; Tue, 27 Feb 2018 13:08:51 -0500
Date: Tue, 27 Feb 2018 20:08:56 +0200
Message-Id: <83zi3uz4nb.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
In-reply-to: <877eqyocro.fsf@HIDDEN> (message from Michael Heerdegen on Tue,
 27 Feb 2018 13:08:59 +0100)
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
 <877eqyocro.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
> Cc: bug-gnu-emacs@HIDDEN,  30626 <at> debbugs.gnu.org
> Date: Tue, 27 Feb 2018 13:08:59 +0100
> 
> #+begin_src emacs-lisp
> (seq-doseq (_ (stream-range 1 1000000)) nil)
> #+end_src
> 
> Note that this is executed as a loop due how to streams are implemented,
> although the definition of `seq-doseq' looks recursive.  But it seems
> that gc has a problem with the large number of conses created when
> processing that.

What can we do instead in such cases?  Stack-overflow protection
cannot work in GC, so you are shooting yourself in the foot by
creating such large recursive structures.  By the time we get to GC,
where the problem will happen, it's too late, because the memory was
already allocated.

Does anyone has a reasonable idea for avoiding the crash in such
programs?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 27 Feb 2018 18:00:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 13:00:45 2018
Received: from localhost ([127.0.0.1]:35896 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqjYT-0002dW-Kj
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:00:45 -0500
Received: from eggs.gnu.org ([208.118.235.92]:47324)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1eqjYS-0002dK-5A
 for 30626 <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:00:44 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqjYJ-0007qb-MI
 for 30626 <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:00:38 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42529)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1eqjYJ-0007qX-JK; Tue, 27 Feb 2018 13:00:35 -0500
Received: from [176.228.60.248] (port=2484 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1eqjYI-0003P6-SW; Tue, 27 Feb 2018 13:00:35 -0500
Date: Tue, 27 Feb 2018 20:00:39 +0200
Message-Id: <831sh61feg.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
In-reply-to: <87fu5moe4c.fsf@HIDDEN> (message from Michael Heerdegen on Tue,
 27 Feb 2018 12:39:47 +0100)
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 30626
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
> Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
> Date: Tue, 27 Feb 2018 12:39:47 +0100
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> > I guess it's a stack overflow: that function recurses into
> > subdirectories.
> >
> > To avoid such problems, the function should be rewritten to work by
> > BFS, not DFS.
> 
> Here is the backtrace I'm now able to produce.  Looks like the crash
> happens in gc:

GC is deeply-recursive, and you have exacerbated that by using up a
lot of stack.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 18:00:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 13:00:57 2018
Received: from localhost ([127.0.0.1]:35899 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqjYe-0002du-Sm
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:00:57 -0500
Received: from eggs.gnu.org ([208.118.235.92]:47382)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1eqjYd-0002dh-G9
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:00:55 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqjYX-0007yO-QJ
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:00:50 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:39218)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1eqjYX-0007yD-M7
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 13:00:49 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38341)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqjYW-00067P-9a
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 13:00:49 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqjYS-0007up-J4
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 13:00:48 -0500
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42529)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1eqjYJ-0007qX-JK; Tue, 27 Feb 2018 13:00:35 -0500
Received: from [176.228.60.248] (port=2484 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1eqjYI-0003P6-SW; Tue, 27 Feb 2018 13:00:35 -0500
Date: Tue, 27 Feb 2018 20:00:39 +0200
Message-Id: <831sh61feg.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Michael Heerdegen <michael_heerdegen@HIDDEN>
In-reply-to: <87fu5moe4c.fsf@HIDDEN> (message from Michael Heerdegen on Tue,
 27 Feb 2018 12:39:47 +0100)
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Michael Heerdegen <michael_heerdegen@HIDDEN>
> Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
> Date: Tue, 27 Feb 2018 12:39:47 +0100
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> > I guess it's a stack overflow: that function recurses into
> > subdirectories.
> >
> > To avoid such problems, the function should be rewritten to work by
> > BFS, not DFS.
> 
> Here is the backtrace I'm now able to produce.  Looks like the crash
> happens in gc:

GC is deeply-recursive, and you have exacerbated that by using up a
lot of stack.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 27 Feb 2018 12:09:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 07:09:08 2018
Received: from localhost ([127.0.0.1]:34459 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqe4B-0006Pa-S0
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 07:09:08 -0500
Received: from mout.web.de ([212.227.15.3]:35961)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqe4A-0006PB-Ib
 for 30626 <at> debbugs.gnu.org; Tue, 27 Feb 2018 07:09:07 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb001
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LfzgJ-1eOVBX1lGd-00pgjc; Tue, 27
 Feb 2018 13:09:00 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
Date: Tue, 27 Feb 2018 13:08:59 +0100
In-Reply-To: <87fu5moe4c.fsf@HIDDEN> (Michael Heerdegen's message of "Tue, 27
 Feb 2018 12:39:47 +0100")
Message-ID: <877eqyocro.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:bxTtByJhimEv05aXorB6dgYWWI5CxcjWxfvhjzLjahV1FhFYFNA
 T1BSRBISXpfR++tkkYh8KMv/YpFKpT8Bag5I9ZZI8Vck42LSKCESQaCarfKPHHGgOSA9/nE
 iewu5TIdrCh7pWctYVGeOJAQ6ETjviH/te/yjFxqa+Dk8BsmS0GGtgxJWSlY2Va2/frcaMo
 ddTL6YvGsUZVFBkKg9seA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:5ht05sbwaVY=:YMhtPfUxSe0lsYXrfC3g0I
 lp+b54KumohKGoMlp9KAsLkVjC7Z4Pzifm4LlFr8pIPqlMa+4NxCGggMQMq5jD1p0fKq3DNww
 O3J6kZrEJi1QL/6RYZ+iplDmqqJxJ/XhznPPWnp4+BaZ9/siI5IjFAuaozvPGEXaavEE5omzN
 L0ndlZnJI4EOGm/IEL66CW0dfTdSVJWmU7z8va7W+ZL+ogphw9ULfk89+LIJkmN8t0jEj7pYK
 GjwRvc15QxX7hdHB62DwLu0OQpfjoZnYM1sRE0qoUUBX9aTwvV/G0xy6BbjIZqcezF+J1V+rm
 yRHy0QOK01u1MdrwBIvABQUBwCcZn5ZdWiZRhPnW4CINT7QEZOA+vtvACUD3UvlCiU+GVu4M3
 AhZf8RM6x3D2I1veC12D4sR2h6I5d4HcGmXyWQf3O6anb4Ja5v5OXc7dBOGbyE7cdfwC8vzfN
 n2MsoUZSpm81oxFZD6Ygsue8eVL/lx95ax3FEdk2yDIKZc5ZgvoRZFbcgryKheFuDUk+ucOz7
 UQfLR2Z5H9arnNJ50EmyFKNY+0PjCdwv/czkI1PCqhebZWlNG5ZV6bXVgjjQkIOU83TSDO2Ix
 +LMWnqwn13jPxLHJ/a4cb/Ol8o33EblCMPKJR0+g0VHofgNa41KeA/WaIcw+D6uT86Gcif3Tz
 GKFiFrsVxx+fgABpg99gHAGAgQlNc5yECuZIC+dcs9L8LjMsYdyXaK+buPc/wgIFbFJ5Z81QS
 RuWqyojhIRJzjHTIBQWM5e4FxrNy2LjeHAHYoVYs51lIx0vBdQhrdCR06uYFoTOggByhADrxD
 ylglMvVN2syigdWcgugSJipT6S0b75fro8FhluRmrm5/iOg3kg=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Michael Heerdegen <michael_heerdegen@HIDDEN> writes:

> Here is the backtrace I'm now able to produce.  Looks like the crash
> happens in gc.

Here is a much simpler example that crashes as well:

#+begin_src emacs-lisp
(seq-doseq (_ (stream-range 1 1000000)) nil)
#+end_src

Note that this is executed as a loop due how to streams are implemented,
although the definition of `seq-doseq' looks recursive.  But it seems
that gc has a problem with the large number of conses created when
processing that.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 12:09:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 07:09:24 2018
Received: from localhost ([127.0.0.1]:34462 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqe4S-0006Q2-5k
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 07:09:24 -0500
Received: from eggs.gnu.org ([208.118.235.92]:56604)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqe4R-0006Pq-3L
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 07:09:23 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqe4L-00064H-5d
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 07:09:17 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:49960)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <michael_heerdegen@HIDDEN>)
 id 1eqe4L-00064D-1f
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 07:09:17 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47566)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqe4K-0003gj-23
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 07:09:16 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqe4J-00063T-9q
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 07:09:16 -0500
Received: from mout.web.de ([212.227.15.3]:59789)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <michael_heerdegen@HIDDEN>)
 id 1eqe4C-00060S-IY; Tue, 27 Feb 2018 07:09:08 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb001
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LfzgJ-1eOVBX1lGd-00pgjc; Tue, 27
 Feb 2018 13:09:00 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> <87fu5moe4c.fsf@HIDDEN>
Date: Tue, 27 Feb 2018 13:08:59 +0100
In-Reply-To: <87fu5moe4c.fsf@HIDDEN> (Michael Heerdegen's message of "Tue, 27
 Feb 2018 12:39:47 +0100")
Message-ID: <877eqyocro.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:bxTtByJhimEv05aXorB6dgYWWI5CxcjWxfvhjzLjahV1FhFYFNA
 T1BSRBISXpfR++tkkYh8KMv/YpFKpT8Bag5I9ZZI8Vck42LSKCESQaCarfKPHHGgOSA9/nE
 iewu5TIdrCh7pWctYVGeOJAQ6ETjviH/te/yjFxqa+Dk8BsmS0GGtgxJWSlY2Va2/frcaMo
 ddTL6YvGsUZVFBkKg9seA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:5ht05sbwaVY=:YMhtPfUxSe0lsYXrfC3g0I
 lp+b54KumohKGoMlp9KAsLkVjC7Z4Pzifm4LlFr8pIPqlMa+4NxCGggMQMq5jD1p0fKq3DNww
 O3J6kZrEJi1QL/6RYZ+iplDmqqJxJ/XhznPPWnp4+BaZ9/siI5IjFAuaozvPGEXaavEE5omzN
 L0ndlZnJI4EOGm/IEL66CW0dfTdSVJWmU7z8va7W+ZL+ogphw9ULfk89+LIJkmN8t0jEj7pYK
 GjwRvc15QxX7hdHB62DwLu0OQpfjoZnYM1sRE0qoUUBX9aTwvV/G0xy6BbjIZqcezF+J1V+rm
 yRHy0QOK01u1MdrwBIvABQUBwCcZn5ZdWiZRhPnW4CINT7QEZOA+vtvACUD3UvlCiU+GVu4M3
 AhZf8RM6x3D2I1veC12D4sR2h6I5d4HcGmXyWQf3O6anb4Ja5v5OXc7dBOGbyE7cdfwC8vzfN
 n2MsoUZSpm81oxFZD6Ygsue8eVL/lx95ax3FEdk2yDIKZc5ZgvoRZFbcgryKheFuDUk+ucOz7
 UQfLR2Z5H9arnNJ50EmyFKNY+0PjCdwv/czkI1PCqhebZWlNG5ZV6bXVgjjQkIOU83TSDO2Ix
 +LMWnqwn13jPxLHJ/a4cb/Ol8o33EblCMPKJR0+g0VHofgNa41KeA/WaIcw+D6uT86Gcif3Tz
 GKFiFrsVxx+fgABpg99gHAGAgQlNc5yECuZIC+dcs9L8LjMsYdyXaK+buPc/wgIFbFJ5Z81QS
 RuWqyojhIRJzjHTIBQWM5e4FxrNy2LjeHAHYoVYs51lIx0vBdQhrdCR06uYFoTOggByhADrxD
 ylglMvVN2syigdWcgugSJipT6S0b75fro8FhluRmrm5/iOg3kg=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
 [fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.1 (----)
X-Debbugs-Envelope-To: submit
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.1 (----)

Michael Heerdegen <michael_heerdegen@HIDDEN> writes:

> Here is the backtrace I'm now able to produce.  Looks like the crash
> happens in gc.

Here is a much simpler example that crashes as well:

#+begin_src emacs-lisp
(seq-doseq (_ (stream-range 1 1000000)) nil)
#+end_src

Note that this is executed as a loop due how to streams are implemented,
although the definition of `seq-doseq' looks recursive.  But it seems
that gc has a problem with the large number of conses created when
processing that.


Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 11:40:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 06:40:17 2018
Received: from localhost ([127.0.0.1]:34443 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqdcH-0005hN-76
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:40:17 -0500
Received: from eggs.gnu.org ([208.118.235.92]:48970)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqdcF-0005hB-Fb
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:40:16 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqdc7-0001i9-15
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:40:10 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:55293)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <michael_heerdegen@HIDDEN>)
 id 1eqdc6-0001i4-Tr
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:40:06 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:39906)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqdc4-00065u-Jc
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 06:40:06 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqdc2-0001gc-9V
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 06:40:04 -0500
Received: from mout.web.de ([212.227.15.4]:39229)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <michael_heerdegen@HIDDEN>)
 id 1eqdbx-0001ds-6o; Tue, 27 Feb 2018 06:39:57 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb001
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0MZDga-1f9HUO0K44-00KueF; Tue, 27
 Feb 2018 12:39:48 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN>
Date: Tue, 27 Feb 2018 12:39:47 +0100
In-Reply-To: <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> (Eli Zaretskii's
 message of "Tue, 27 Feb 2018 13:21:30 +0200")
Message-ID: <87fu5moe4c.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:X5EY5LVRgdjRC8Wib1/u1h8L92V/TRjeExzyCsXYPpox+hHXHtE
 rwruZ2J9w6prrViJrm6vwpxsSQVK2Y2d8c4Wou3q2dHagrnEGCnFHBq1doPhM1OTccTV+Uk
 IAeHTNYhnrTTl1YNVLq/+DjYOpBRwdFUthUXQ4sfUsEDhltkT8SLou4seNc3fEPYEq2hPw5
 LwphsktDq5HpPH17nFElg==
X-UI-Out-Filterresults: notjunk:1;V01:K0:qhxycHH1bkI=:Pwx8O02HntoQwvU2K9Q2cc
 ubJW1BWUb+lcP+i2hka97ltM6Xyio/5EZUiYOFpHgoQszFJ8//m3wys0cP1kjoB++BuyW7o47
 x86xAEEKREGwxHEXvTRAnQrDJbWM9ZfSCvT69npuGQvjqo+TsbXgFlFsL0C4CeJztAbTjYdLn
 +b6EhglRNcNXzdSVYZh4M5jh2i4PXnnWOO6y/4TeoZa7rdTJkxIkdJ+BmQtkxYqnSd3z5GgFx
 6DVS7CTxJb+y+sO5DwZpyng3uh8QWYDE141zvrj9MckAzf8xpi6WvoJBig/nMo47g2JrEsFz9
 jElT6+4AIAtdCr9YrvOFDUPoQMJ8Uavsil1zXbDA2pAalM0+g/M9f1nzoXOebyuQR9Z2qrEiV
 vXfFUcxAR/tw0Kknql8JZdzYDI3OGA9ujs+nU1UUydEeB+ophGU1jy22TNB5S6giOS/zQ6818
 NJToa4t2pWDmgJeymTxDrTA8rxcyyL64w5J/Z7BZpLz3McaDrm3AT2trQSjn2SdBDaefW/2tQ
 GZ1ebGSImULP0oAB/zCFI+0FSpkHXsQn2TZ5jY9OfWpi8QgZ3NUPUtrhSPeFVV0jR72IcVv5n
 ywBPjyWPt8dMML0+6/8nvV3tKT0pryJbt6A1e/rCya+JCjopK+rKxr+wqwkRYcrzd4RnU5RmA
 6ePmPOM5tPgYiKa8QxTD3xCgedBwsLdSrXiEa7TETFz8tZcakohdd0ozcv7BzjdUyoToL9+Pt
 HrqhxRB1enzs8bbD9iKN9g2bqVJNcY4//mX7808rXckY474ncwYUMl/QzQlkhG1U8ixVoKxlt
 pBxIetKjCoAmjImW9D5iRj/j3fS90L3YUWXCMNGwPvTnovsneg=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
 [fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.1 (----)
X-Debbugs-Envelope-To: submit
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.1 (----)

Eli Zaretskii <eliz@HIDDEN> writes:

> I guess it's a stack overflow: that function recurses into
> subdirectories.
>
> To avoid such problems, the function should be rewritten to work by
> BFS, not DFS.

Here is the backtrace I'm now able to produce.  Looks like the crash
happens in gc:

| [lots of more lines like these cut]
| #104019 0x00000000005e0526 in mark_object (arg=XIL(0x5a15413)) at alloc.c:6624
| #104020 0x00000000005dfab5 in mark_vectorlike (ptr=0x5a584e0) at alloc.c:6227
| #104021 0x00000000005e0526 in mark_object (arg=XIL(0x5bd7cf3)) at alloc.c:6624
| #104022 0x00000000005dfab5 in mark_vectorlike (ptr=0x5873a80) at alloc.c:6227
| #104023 0x00000000005e0526 in mark_object (arg=XIL(0x5bd7cb3)) at alloc.c:6624
| #104024 0x00000000006070ea in mark_specpdl (first=0x58a1348, ptr=0x58a1b90) at eval.c:3868
| #104025 0x00000000006856b2 in mark_one_thread (thread=0xcd5340 <main_thread>) at thread.c:614
| #104026 0x00000000006857c7 in mark_threads_callback (ignore=0x0) at thread.c:649
| #104027 0x00000000005dda03 in flush_stack_call_func (func=0x685781 <mark_threads_callback>, arg=0x0) at alloc.c:5220
| #104028 0x00000000006857f5 in mark_threads () at thread.c:656
| #104029 0x00000000005df2d2 in garbage_collect_1 (end=0x7fffffff63f0) at alloc.c:5997
| #104030 0x00000000005df929 in Fgarbage_collect () at alloc.c:6168
| #104031 0x000000000055746c in maybe_gc () at lisp.h:4749
| #104032 0x00000000006047cb in Ffuncall (nargs=4, args=0x7fffffff64d0) at eval.c:2751
| #104033 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83c4), vector=XIL(0x3255bb5), maxdepth=make_number(13), args_template=make_number(128), nargs=2, args=0x7fffffff6b38) at bytecode.c:629
| #104034 0x000000000060529c in funcall_lambda (fun=XIL(0x3255c15), nargs=2, arg_vector=0x7fffffff6b38) at eval.c:2970
| #104035 0x00000000006048d4 in Ffuncall (nargs=3, args=0x7fffffff6b30) at eval.c:2771
| #104036 0x000000000060395b in Fapply (nargs=3, args=0x7fffffff6b30) at eval.c:2346
| #104037 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 <Sapply>, numargs=3, args=0x7fffffff6b30) at eval.c:2824
| #104038 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffff6b28) at eval.c:2769
| #104039 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x15372775), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff6ff8) at bytecode.c:629
| #104040 0x000000000060529c in funcall_lambda (fun=XIL(0x153727c5), nargs=0, arg_vector=0x7fffffff6ff8) at eval.c:2970
| #104041 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff6ff0) at eval.c:2771
| #104042 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff7460) at bytecode.c:629
| #104043 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff7458) at eval.c:2970
| #104044 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff7450) at eval.c:2771
| #104045 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffff78e0) at bytecode.c:629
| #104046 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffff78d8) at eval.c:2970
| #104047 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff78d0) at eval.c:2771
| #104048 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x153727f5), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff7da8) at bytecode.c:629
| #104049 0x000000000060529c in funcall_lambda (fun=XIL(0x15372845), nargs=0, arg_vector=0x7fffffff7da8) at eval.c:2970
| #104050 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff7da0) at eval.c:2771
| #104051 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff8210) at bytecode.c:629
| #104052 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff8208) at eval.c:2970
| #104053 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff8200) at eval.c:2771
| #104054 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffff8690) at bytecode.c:629
| #104055 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffff8688) at eval.c:2970
| #104056 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff8680) at eval.c:2771
| #104057 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x15372875), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff8b58) at bytecode.c:629
| #104058 0x000000000060529c in funcall_lambda (fun=XIL(0x153728c5), nargs=0, arg_vector=0x7fffffff8b58) at eval.c:2970
| #104059 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff8b50) at eval.c:2771
| #104060 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff8fc0) at bytecode.c:629
| #104061 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff8fb8) at eval.c:2970
| #104062 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff8fb0) at eval.c:2771
| #104063 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffff9440) at bytecode.c:629
| #104064 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffff9438) at eval.c:2970
| #104065 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff9430) at eval.c:2771
| #104066 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x153728f5), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff9908) at bytecode.c:629
| #104067 0x000000000060529c in funcall_lambda (fun=XIL(0x15372945), nargs=0, arg_vector=0x7fffffff9908) at eval.c:2970
| #104068 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff9900) at eval.c:2771
| #104069 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff9d70) at bytecode.c:629
| #104070 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff9d68) at eval.c:2970
| #104071 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff9d60) at eval.c:2771
| #104072 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffffa1e8) at bytecode.c:629
| #104073 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffffa1e0) at eval.c:2970
| #104074 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffa1d8) at eval.c:2771
| #104075 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f8c64), vector=XIL(0x153729a5), maxdepth=make_number(7), args_template=make_number(256), nargs=0, args=0x7fffffffa6a8) at bytecode.c:629
| #104076 0x000000000060529c in funcall_lambda (fun=XIL(0x153729f5), nargs=0, arg_vector=0x7fffffffa6a8) at eval.c:2970
| #104077 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffffa6a0) at eval.c:2771
| #104078 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffffab10) at bytecode.c:629
| #104079 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffffab08) at eval.c:2970
| #104080 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffab00) at eval.c:2771
| #104081 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffffaf88) at bytecode.c:629
| #104082 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffffaf80) at eval.c:2970
| #104083 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffaf78) at eval.c:2771
| #104084 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f8c04), vector=XIL(0x331ede5), maxdepth=make_number(5), args_template=make_number(514), nargs=2, args=0x7fffffffb5d0) at bytecode.c:629
| #104085 0x000000000060529c in funcall_lambda (fun=XIL(0x331ee05), nargs=2, arg_vector=0x7fffffffb5c0) at eval.c:2970
| #104086 0x00000000006048d4 in Ffuncall (nargs=3, args=0x7fffffffb5b8) at eval.c:2771
| #104087 0x000000000060390d in Fapply (nargs=4, args=0x7fffffffb5b8) at eval.c:2342
| #104088 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 <Sapply>, numargs=4, args=0x7fffffffb5b8) at eval.c:2824
| #104089 0x0000000000604890 in Ffuncall (nargs=5, args=0x7fffffffb5b0) at eval.c:2769
| #104090 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42fb574), vector=XIL(0x425f285), maxdepth=make_number(15), args_template=make_number(642), nargs=2, args=0x7fffffffba40) at bytecode.c:629
| #104091 0x000000000060529c in funcall_lambda (fun=XIL(0x425f305), nargs=2, arg_vector=0x7fffffffba30) at eval.c:2970
| #104092 0x000000000060500d in apply_lambda (fun=XIL(0x425f305), args=XIL(0x5bd89c3), count=30) at eval.c:2906
| #104093 0x0000000000603602 in eval_sub (form=XIL(0x5bd89d3)) at eval.c:2279
| #104094 0x0000000000632c69 in readevalloop_eager_expand_eval (val=XIL(0x5bd8a73), macroexpand=XIL(0xda5d0)) at lread.c:1850
| #104095 0x00000000006335f0 in readevalloop (readcharfun=XIL(0x5b75ce5), infile0=0x0, sourcename=XIL(0x55c2e14), printflag=false, unibyte=XIL(0), readfun=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2036
| #104096 0x0000000000633a0a in Feval_buffer (buffer=XIL(0), printflag=XIL(0), filename=XIL(0x56ea3c4), unibyte=XIL(0), do_allow_print=XIL(0)) at lread.c:2103
| #104097 0x0000000000604d25 in funcall_subr (subr=0xc40240 <Seval_buffer>, numargs=0, args=0x7fffffffbfa0) at eval.c:2856
| #104098 0x0000000000604890 in Ffuncall (nargs=1, args=0x7fffffffbf98) at eval.c:2769
| #104099 0x00000000005fc8f7 in Ffuncall_interactively (nargs=1, args=0x7fffffffbf98) at callint.c:252
| #104100 0x0000000000604b87 in funcall_subr (subr=0xc3cfc0 <Sfuncall_interactively>, numargs=1, args=0x7fffffffbf98) at eval.c:2824
| #104101 0x0000000000604890 in Ffuncall (nargs=2, args=0x7fffffffbf90) at eval.c:2769
| #104102 0x00000000005fec50 in Fcall_interactively (function=XIL(0xad320), record_flag=XIL(0xaad0), keys=XIL(0x5a55745)) at callint.c:854
| #104103 0x0000000000604cac in funcall_subr (subr=0xc3d000 <Scall_interactively>, numargs=3, args=0x7fffffffc300) at eval.c:2849
| #104104 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffffc2f8) at eval.c:2769
| #104105 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x9f4bec), vector=XIL(0x9f4c0d), maxdepth=make_number(13), args_template=make_number(1025), nargs=2, args=0x7fffffffc868) at bytecode.c:629
| #104106 0x000000000060529c in funcall_lambda (fun=XIL(0x9f4bbd), nargs=2, arg_vector=0x7fffffffc858) at eval.c:2970
| #104107 0x00000000006048d4 in Ffuncall (nargs=3, args=0x7fffffffc850) at eval.c:2771
| #104108 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x9f489c), vector=XIL(0x9f48bd), maxdepth=make_number(15), args_template=make_number(769), nargs=3, args=0x7fffffffce00) at bytecode.c:629
| #104109 0x000000000060529c in funcall_lambda (fun=XIL(0x9f485d), nargs=3, arg_vector=0x7fffffffcde8) at eval.c:2970
| #104110 0x00000000006048d4 in Ffuncall (nargs=4, args=0x7fffffffcde0) at eval.c:2771
| #104111 0x0000000000603cdb in Fapply (nargs=2, args=0x7fffffffcfc0) at eval.c:2389
| #104112 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 <Sapply>, numargs=2, args=0x7fffffffcfc0) at eval.c:2824
| #104113 0x0000000000604890 in Ffuncall (nargs=3, args=0x7fffffffcfb8) at eval.c:2769
| #104114 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x505da94), vector=XIL(0x5044095), maxdepth=make_number(3), args_template=XIL(0), nargs=0, args=0x0) at bytecode.c:629
| #104115 0x0000000000605615 in funcall_lambda (fun=XIL(0x50c3185), nargs=5, arg_vector=0x5044095) at eval.c:3052
| #104116 0x00000000006048d4 in Ffuncall (nargs=6, args=0x7fffffffd450) at eval.c:2771
| #104117 0x0000000000603cdb in Fapply (nargs=3, args=0x7fffffffd648) at eval.c:2389
| #104118 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 <Sapply>, numargs=3, args=0x7fffffffd648) at eval.c:2824
| #104119 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffffd640) at eval.c:2769
| #104120 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x14402c4), vector=XIL(0x50441c5), maxdepth=make_number(5), args_template=make_number(128), nargs=4, args=0x7fffffffdbf0) at bytecode.c:629
| #104121 0x000000000060529c in funcall_lambda (fun=XIL(0x50441f5), nargs=4, arg_vector=0x7fffffffdbf0) at eval.c:2970
| #104122 0x00000000006048d4 in Ffuncall (nargs=5, args=0x7fffffffdbe8) at eval.c:2771
| #104123 0x00000000005fc8f7 in Ffuncall_interactively (nargs=5, args=0x7fffffffdbe8) at callint.c:252
| #104124 0x0000000000604b87 in funcall_subr (subr=0xc3cfc0 <Sfuncall_interactively>, numargs=5, args=0x7fffffffdbe8) at eval.c:2824
| #104125 0x0000000000604890 in Ffuncall (nargs=6, args=0x7fffffffdbe0) at eval.c:2769
| #104126 0x0000000000603cdb in Fapply (nargs=3, args=0x7fffffffdd90) at eval.c:2389
| #104127 0x00000000005fcd7d in Fcall_interactively (function=XIL(0xda3c0), record_flag=XIL(0), keys=XIL(0x5a3e0a5)) at callint.c:389
| #104128 0x0000000000604cac in funcall_subr (subr=0xc3d000 <Scall_interactively>, numargs=3, args=0x7fffffffdfe0) at eval.c:2849
| #104129 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffffdfd8) at eval.c:2769
| #104130 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x9f4bec), vector=XIL(0x9f4c0d), maxdepth=make_number(13), args_template=make_number(1025), nargs=1, args=0x7fffffffe520) at bytecode.c:629
| #104131 0x000000000060529c in funcall_lambda (fun=XIL(0x9f4bbd), nargs=1, arg_vector=0x7fffffffe518) at eval.c:2970
| #104132 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffe510) at eval.c:2771
| #104133 0x00000000006042a0 in call1 (fn=XIL(0x3f00), arg1=XIL(0xda3c0)) at eval.c:2620
| #104134 0x000000000055ec02 in command_loop_1 () at keyboard.c:1482
| #104135 0x00000000006013b2 in internal_condition_case (bfun=0x55e45e <command_loop_1>, handlers=XIL(0x5250), hfun=0x55dc1d <cmd_error>) at eval.c:1332
| #104136 0x000000000055e151 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1110
| #104137 0x0000000000600c94 in internal_catch (tag=XIL(0xc6f0), func=0x55e124 <command_loop_2>, arg=XIL(0)) at eval.c:1097
| #104138 0x000000000055e0ef in command_loop () at keyboard.c:1089
| #104139 0x000000000055d7ef in recursive_edit_1 () at keyboard.c:695
| #104140 0x000000000055d970 in Frecursive_edit () at keyboard.c:766
| #104141 0x000000000055b5f0 in main (argc=1, argv=0x7fffffffe9f8) at emacs.c:1713
| Cannot access memory at address 0x7fffff66ff4f



Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 27 Feb 2018 11:39:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 06:39:58 2018
Received: from localhost ([127.0.0.1]:34439 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqdbw-0005gL-9f
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:39:56 -0500
Received: from mout.web.de ([212.227.15.4]:50267)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqdbu-0005g7-TT
 for 30626 <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:39:55 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb001
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0MZDga-1f9HUO0K44-00KueF; Tue, 27
 Feb 2018 12:39:48 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
References: <87inaiss6l.fsf@HIDDEN>
 <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN>
Date: Tue, 27 Feb 2018 12:39:47 +0100
In-Reply-To: <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN> (Eli Zaretskii's
 message of "Tue, 27 Feb 2018 13:21:30 +0200")
Message-ID: <87fu5moe4c.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:X5EY5LVRgdjRC8Wib1/u1h8L92V/TRjeExzyCsXYPpox+hHXHtE
 rwruZ2J9w6prrViJrm6vwpxsSQVK2Y2d8c4Wou3q2dHagrnEGCnFHBq1doPhM1OTccTV+Uk
 IAeHTNYhnrTTl1YNVLq/+DjYOpBRwdFUthUXQ4sfUsEDhltkT8SLou4seNc3fEPYEq2hPw5
 LwphsktDq5HpPH17nFElg==
X-UI-Out-Filterresults: notjunk:1;V01:K0:qhxycHH1bkI=:Pwx8O02HntoQwvU2K9Q2cc
 ubJW1BWUb+lcP+i2hka97ltM6Xyio/5EZUiYOFpHgoQszFJ8//m3wys0cP1kjoB++BuyW7o47
 x86xAEEKREGwxHEXvTRAnQrDJbWM9ZfSCvT69npuGQvjqo+TsbXgFlFsL0C4CeJztAbTjYdLn
 +b6EhglRNcNXzdSVYZh4M5jh2i4PXnnWOO6y/4TeoZa7rdTJkxIkdJ+BmQtkxYqnSd3z5GgFx
 6DVS7CTxJb+y+sO5DwZpyng3uh8QWYDE141zvrj9MckAzf8xpi6WvoJBig/nMo47g2JrEsFz9
 jElT6+4AIAtdCr9YrvOFDUPoQMJ8Uavsil1zXbDA2pAalM0+g/M9f1nzoXOebyuQR9Z2qrEiV
 vXfFUcxAR/tw0Kknql8JZdzYDI3OGA9ujs+nU1UUydEeB+ophGU1jy22TNB5S6giOS/zQ6818
 NJToa4t2pWDmgJeymTxDrTA8rxcyyL64w5J/Z7BZpLz3McaDrm3AT2trQSjn2SdBDaefW/2tQ
 GZ1ebGSImULP0oAB/zCFI+0FSpkHXsQn2TZ5jY9OfWpi8QgZ3NUPUtrhSPeFVV0jR72IcVv5n
 ywBPjyWPt8dMML0+6/8nvV3tKT0pryJbt6A1e/rCya+JCjopK+rKxr+wqwkRYcrzd4RnU5RmA
 6ePmPOM5tPgYiKa8QxTD3xCgedBwsLdSrXiEa7TETFz8tZcakohdd0ozcv7BzjdUyoToL9+Pt
 HrqhxRB1enzs8bbD9iKN9g2bqVJNcY4//mX7808rXckY474ncwYUMl/QzQlkhG1U8ixVoKxlt
 pBxIetKjCoAmjImW9D5iRj/j3fS90L3YUWXCMNGwPvTnovsneg=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 30626
Cc: bug-gnu-emacs@HIDDEN, 30626 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Eli Zaretskii <eliz@HIDDEN> writes:

> I guess it's a stack overflow: that function recurses into
> subdirectories.
>
> To avoid such problems, the function should be rewritten to work by
> BFS, not DFS.

Here is the backtrace I'm now able to produce.  Looks like the crash
happens in gc:

| [lots of more lines like these cut]
| #104019 0x00000000005e0526 in mark_object (arg=XIL(0x5a15413)) at alloc.c:6624
| #104020 0x00000000005dfab5 in mark_vectorlike (ptr=0x5a584e0) at alloc.c:6227
| #104021 0x00000000005e0526 in mark_object (arg=XIL(0x5bd7cf3)) at alloc.c:6624
| #104022 0x00000000005dfab5 in mark_vectorlike (ptr=0x5873a80) at alloc.c:6227
| #104023 0x00000000005e0526 in mark_object (arg=XIL(0x5bd7cb3)) at alloc.c:6624
| #104024 0x00000000006070ea in mark_specpdl (first=0x58a1348, ptr=0x58a1b90) at eval.c:3868
| #104025 0x00000000006856b2 in mark_one_thread (thread=0xcd5340 <main_thread>) at thread.c:614
| #104026 0x00000000006857c7 in mark_threads_callback (ignore=0x0) at thread.c:649
| #104027 0x00000000005dda03 in flush_stack_call_func (func=0x685781 <mark_threads_callback>, arg=0x0) at alloc.c:5220
| #104028 0x00000000006857f5 in mark_threads () at thread.c:656
| #104029 0x00000000005df2d2 in garbage_collect_1 (end=0x7fffffff63f0) at alloc.c:5997
| #104030 0x00000000005df929 in Fgarbage_collect () at alloc.c:6168
| #104031 0x000000000055746c in maybe_gc () at lisp.h:4749
| #104032 0x00000000006047cb in Ffuncall (nargs=4, args=0x7fffffff64d0) at eval.c:2751
| #104033 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83c4), vector=XIL(0x3255bb5), maxdepth=make_number(13), args_template=make_number(128), nargs=2, args=0x7fffffff6b38) at bytecode.c:629
| #104034 0x000000000060529c in funcall_lambda (fun=XIL(0x3255c15), nargs=2, arg_vector=0x7fffffff6b38) at eval.c:2970
| #104035 0x00000000006048d4 in Ffuncall (nargs=3, args=0x7fffffff6b30) at eval.c:2771
| #104036 0x000000000060395b in Fapply (nargs=3, args=0x7fffffff6b30) at eval.c:2346
| #104037 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 <Sapply>, numargs=3, args=0x7fffffff6b30) at eval.c:2824
| #104038 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffff6b28) at eval.c:2769
| #104039 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x15372775), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff6ff8) at bytecode.c:629
| #104040 0x000000000060529c in funcall_lambda (fun=XIL(0x153727c5), nargs=0, arg_vector=0x7fffffff6ff8) at eval.c:2970
| #104041 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff6ff0) at eval.c:2771
| #104042 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff7460) at bytecode.c:629
| #104043 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff7458) at eval.c:2970
| #104044 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff7450) at eval.c:2771
| #104045 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffff78e0) at bytecode.c:629
| #104046 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffff78d8) at eval.c:2970
| #104047 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff78d0) at eval.c:2771
| #104048 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x153727f5), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff7da8) at bytecode.c:629
| #104049 0x000000000060529c in funcall_lambda (fun=XIL(0x15372845), nargs=0, arg_vector=0x7fffffff7da8) at eval.c:2970
| #104050 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff7da0) at eval.c:2771
| #104051 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff8210) at bytecode.c:629
| #104052 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff8208) at eval.c:2970
| #104053 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff8200) at eval.c:2771
| #104054 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffff8690) at bytecode.c:629
| #104055 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffff8688) at eval.c:2970
| #104056 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff8680) at eval.c:2771
| #104057 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x15372875), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff8b58) at bytecode.c:629
| #104058 0x000000000060529c in funcall_lambda (fun=XIL(0x153728c5), nargs=0, arg_vector=0x7fffffff8b58) at eval.c:2970
| #104059 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff8b50) at eval.c:2771
| #104060 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff8fc0) at bytecode.c:629
| #104061 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff8fb8) at eval.c:2970
| #104062 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff8fb0) at eval.c:2771
| #104063 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffff9440) at bytecode.c:629
| #104064 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffff9438) at eval.c:2970
| #104065 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff9430) at eval.c:2771
| #104066 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x153728f5), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff9908) at bytecode.c:629
| #104067 0x000000000060529c in funcall_lambda (fun=XIL(0x15372945), nargs=0, arg_vector=0x7fffffff9908) at eval.c:2970
| #104068 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff9900) at eval.c:2771
| #104069 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff9d70) at bytecode.c:629
| #104070 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff9d68) at eval.c:2970
| #104071 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff9d60) at eval.c:2771
| #104072 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffffa1e8) at bytecode.c:629
| #104073 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffffa1e0) at eval.c:2970
| #104074 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffa1d8) at eval.c:2771
| #104075 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f8c64), vector=XIL(0x153729a5), maxdepth=make_number(7), args_template=make_number(256), nargs=0, args=0x7fffffffa6a8) at bytecode.c:629
| #104076 0x000000000060529c in funcall_lambda (fun=XIL(0x153729f5), nargs=0, arg_vector=0x7fffffffa6a8) at eval.c:2970
| #104077 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffffa6a0) at eval.c:2771
| #104078 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffffab10) at bytecode.c:629
| #104079 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffffab08) at eval.c:2970
| #104080 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffab00) at eval.c:2771
| #104081 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffffaf88) at bytecode.c:629
| #104082 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffffaf80) at eval.c:2970
| #104083 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffaf78) at eval.c:2771
| #104084 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f8c04), vector=XIL(0x331ede5), maxdepth=make_number(5), args_template=make_number(514), nargs=2, args=0x7fffffffb5d0) at bytecode.c:629
| #104085 0x000000000060529c in funcall_lambda (fun=XIL(0x331ee05), nargs=2, arg_vector=0x7fffffffb5c0) at eval.c:2970
| #104086 0x00000000006048d4 in Ffuncall (nargs=3, args=0x7fffffffb5b8) at eval.c:2771
| #104087 0x000000000060390d in Fapply (nargs=4, args=0x7fffffffb5b8) at eval.c:2342
| #104088 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 <Sapply>, numargs=4, args=0x7fffffffb5b8) at eval.c:2824
| #104089 0x0000000000604890 in Ffuncall (nargs=5, args=0x7fffffffb5b0) at eval.c:2769
| #104090 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42fb574), vector=XIL(0x425f285), maxdepth=make_number(15), args_template=make_number(642), nargs=2, args=0x7fffffffba40) at bytecode.c:629
| #104091 0x000000000060529c in funcall_lambda (fun=XIL(0x425f305), nargs=2, arg_vector=0x7fffffffba30) at eval.c:2970
| #104092 0x000000000060500d in apply_lambda (fun=XIL(0x425f305), args=XIL(0x5bd89c3), count=30) at eval.c:2906
| #104093 0x0000000000603602 in eval_sub (form=XIL(0x5bd89d3)) at eval.c:2279
| #104094 0x0000000000632c69 in readevalloop_eager_expand_eval (val=XIL(0x5bd8a73), macroexpand=XIL(0xda5d0)) at lread.c:1850
| #104095 0x00000000006335f0 in readevalloop (readcharfun=XIL(0x5b75ce5), infile0=0x0, sourcename=XIL(0x55c2e14), printflag=false, unibyte=XIL(0), readfun=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2036
| #104096 0x0000000000633a0a in Feval_buffer (buffer=XIL(0), printflag=XIL(0), filename=XIL(0x56ea3c4), unibyte=XIL(0), do_allow_print=XIL(0)) at lread.c:2103
| #104097 0x0000000000604d25 in funcall_subr (subr=0xc40240 <Seval_buffer>, numargs=0, args=0x7fffffffbfa0) at eval.c:2856
| #104098 0x0000000000604890 in Ffuncall (nargs=1, args=0x7fffffffbf98) at eval.c:2769
| #104099 0x00000000005fc8f7 in Ffuncall_interactively (nargs=1, args=0x7fffffffbf98) at callint.c:252
| #104100 0x0000000000604b87 in funcall_subr (subr=0xc3cfc0 <Sfuncall_interactively>, numargs=1, args=0x7fffffffbf98) at eval.c:2824
| #104101 0x0000000000604890 in Ffuncall (nargs=2, args=0x7fffffffbf90) at eval.c:2769
| #104102 0x00000000005fec50 in Fcall_interactively (function=XIL(0xad320), record_flag=XIL(0xaad0), keys=XIL(0x5a55745)) at callint.c:854
| #104103 0x0000000000604cac in funcall_subr (subr=0xc3d000 <Scall_interactively>, numargs=3, args=0x7fffffffc300) at eval.c:2849
| #104104 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffffc2f8) at eval.c:2769
| #104105 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x9f4bec), vector=XIL(0x9f4c0d), maxdepth=make_number(13), args_template=make_number(1025), nargs=2, args=0x7fffffffc868) at bytecode.c:629
| #104106 0x000000000060529c in funcall_lambda (fun=XIL(0x9f4bbd), nargs=2, arg_vector=0x7fffffffc858) at eval.c:2970
| #104107 0x00000000006048d4 in Ffuncall (nargs=3, args=0x7fffffffc850) at eval.c:2771
| #104108 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x9f489c), vector=XIL(0x9f48bd), maxdepth=make_number(15), args_template=make_number(769), nargs=3, args=0x7fffffffce00) at bytecode.c:629
| #104109 0x000000000060529c in funcall_lambda (fun=XIL(0x9f485d), nargs=3, arg_vector=0x7fffffffcde8) at eval.c:2970
| #104110 0x00000000006048d4 in Ffuncall (nargs=4, args=0x7fffffffcde0) at eval.c:2771
| #104111 0x0000000000603cdb in Fapply (nargs=2, args=0x7fffffffcfc0) at eval.c:2389
| #104112 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 <Sapply>, numargs=2, args=0x7fffffffcfc0) at eval.c:2824
| #104113 0x0000000000604890 in Ffuncall (nargs=3, args=0x7fffffffcfb8) at eval.c:2769
| #104114 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x505da94), vector=XIL(0x5044095), maxdepth=make_number(3), args_template=XIL(0), nargs=0, args=0x0) at bytecode.c:629
| #104115 0x0000000000605615 in funcall_lambda (fun=XIL(0x50c3185), nargs=5, arg_vector=0x5044095) at eval.c:3052
| #104116 0x00000000006048d4 in Ffuncall (nargs=6, args=0x7fffffffd450) at eval.c:2771
| #104117 0x0000000000603cdb in Fapply (nargs=3, args=0x7fffffffd648) at eval.c:2389
| #104118 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 <Sapply>, numargs=3, args=0x7fffffffd648) at eval.c:2824
| #104119 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffffd640) at eval.c:2769
| #104120 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x14402c4), vector=XIL(0x50441c5), maxdepth=make_number(5), args_template=make_number(128), nargs=4, args=0x7fffffffdbf0) at bytecode.c:629
| #104121 0x000000000060529c in funcall_lambda (fun=XIL(0x50441f5), nargs=4, arg_vector=0x7fffffffdbf0) at eval.c:2970
| #104122 0x00000000006048d4 in Ffuncall (nargs=5, args=0x7fffffffdbe8) at eval.c:2771
| #104123 0x00000000005fc8f7 in Ffuncall_interactively (nargs=5, args=0x7fffffffdbe8) at callint.c:252
| #104124 0x0000000000604b87 in funcall_subr (subr=0xc3cfc0 <Sfuncall_interactively>, numargs=5, args=0x7fffffffdbe8) at eval.c:2824
| #104125 0x0000000000604890 in Ffuncall (nargs=6, args=0x7fffffffdbe0) at eval.c:2769
| #104126 0x0000000000603cdb in Fapply (nargs=3, args=0x7fffffffdd90) at eval.c:2389
| #104127 0x00000000005fcd7d in Fcall_interactively (function=XIL(0xda3c0), record_flag=XIL(0), keys=XIL(0x5a3e0a5)) at callint.c:389
| #104128 0x0000000000604cac in funcall_subr (subr=0xc3d000 <Scall_interactively>, numargs=3, args=0x7fffffffdfe0) at eval.c:2849
| #104129 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffffdfd8) at eval.c:2769
| #104130 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x9f4bec), vector=XIL(0x9f4c0d), maxdepth=make_number(13), args_template=make_number(1025), nargs=1, args=0x7fffffffe520) at bytecode.c:629
| #104131 0x000000000060529c in funcall_lambda (fun=XIL(0x9f4bbd), nargs=1, arg_vector=0x7fffffffe518) at eval.c:2970
| #104132 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffe510) at eval.c:2771
| #104133 0x00000000006042a0 in call1 (fn=XIL(0x3f00), arg1=XIL(0xda3c0)) at eval.c:2620
| #104134 0x000000000055ec02 in command_loop_1 () at keyboard.c:1482
| #104135 0x00000000006013b2 in internal_condition_case (bfun=0x55e45e <command_loop_1>, handlers=XIL(0x5250), hfun=0x55dc1d <cmd_error>) at eval.c:1332
| #104136 0x000000000055e151 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1110
| #104137 0x0000000000600c94 in internal_catch (tag=XIL(0xc6f0), func=0x55e124 <command_loop_2>, arg=XIL(0)) at eval.c:1097
| #104138 0x000000000055e0ef in command_loop () at keyboard.c:1089
| #104139 0x000000000055d7ef in recursive_edit_1 () at keyboard.c:695
| #104140 0x000000000055d970 in Frecursive_edit () at keyboard.c:766
| #104141 0x000000000055b5f0 in main (argc=1, argv=0x7fffffffe9f8) at emacs.c:1713
| Cannot access memory at address 0x7fffff66ff4f



Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at 30626 <at> debbugs.gnu.org:


Received: (at 30626) by debbugs.gnu.org; 27 Feb 2018 11:21:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 06:21:51 2018
Received: from localhost ([127.0.0.1]:34415 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqdKR-0005BY-Be
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:21:51 -0500
Received: from eggs.gnu.org ([208.118.235.92]:43695)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1eqdKP-0005B9-FE
 for 30626 <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:21:49 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqdKJ-0002AA-Hb
 for 30626 <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:21:44 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35957)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1eqdKB-00027w-It; Tue, 27 Feb 2018 06:21:35 -0500
Received: from [176.12.189.185] (port=20609 helo=[10.163.207.70])
 by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1eqdKA-00009e-3I; Tue, 27 Feb 2018 06:21:35 -0500
Date: Tue, 27 Feb 2018 13:21:30 +0200
User-Agent: K-9 Mail for Android
In-Reply-To: <87inaiss6l.fsf@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
To: bug-gnu-emacs@HIDDEN, Michael Heerdegen <michael_heerdegen@HIDDEN>,
 30626 <at> debbugs.gnu.org
From: Eli Zaretskii <eliz@HIDDEN>
Message-ID: <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 30626
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

On February 27, 2018 11:22:26 AM GMT+02:00, Michael Heerdegen <michael_heer=
degen@web=2Ede> wrote:
>=20
> Hello,
>=20
> Traversing a `stream-of-directory-files' over a huge directory
> hierarchy
> crashes my Emacs=2E
>=20
> Here is a recipe: I have a file
> "/home/micha/Treasure/today/stream-crash=2Eel" with these contents:
>=20
> #+begin_src emacs-lisp
> ;; -*- lexical-binding: t -*-
>=20
> (require 'stream)
>=20
> (seq-doseq (_file (stream-of-directory-files
>                    "/home/micha" t nil t nil
>   (lambda (file) (and (file-readable-p file) (file-regular-p file)))))
>   nil)
> #+end_src
>=20
> Then I
>=20
> micha> emacs -Q -L /home/micha/software/elpa/packages/stream\
>                 -l /home/micha/Treasure/today/stream-crash=2Eel
>=20
> and I get a segfault after ~ 1 minute=2E
>=20
> This kind of crash only seems to occur when the traversed directory is
> sufficiently large=2E


I guess it's a stack overflow: that function recurses into subdirectories=
=2E

To avoid such problems, the function should be rewritten to work by BFS, n=
ot DFS=2E




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 11:21:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 06:21:49 2018
Received: from localhost ([127.0.0.1]:34412 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqdKP-0005BJ-3q
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:21:49 -0500
Received: from eggs.gnu.org ([208.118.235.92]:43685)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1eqdKN-0005B5-57
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:21:47 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqdKH-00029U-2V
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:21:41 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:34101)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1eqdKG-00029L-V4
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 06:21:40 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:34649)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqdKF-0000hU-RU
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 06:21:40 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1eqdKB-000281-N5
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 06:21:39 -0500
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35957)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1eqdKB-00027w-It; Tue, 27 Feb 2018 06:21:35 -0500
Received: from [176.12.189.185] (port=20609 helo=[10.163.207.70])
 by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1eqdKA-00009e-3I; Tue, 27 Feb 2018 06:21:35 -0500
Date: Tue, 27 Feb 2018 13:21:30 +0200
User-Agent: K-9 Mail for Android
In-Reply-To: <87inaiss6l.fsf@HIDDEN>
References: <87inaiss6l.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: bug#30626: 26.0.91;
 Crash when traversing a `stream-of-directory-files'
To: bug-gnu-emacs@HIDDEN, Michael Heerdegen <michael_heerdegen@HIDDEN>,
 30626 <at> debbugs.gnu.org
From: Eli Zaretskii <eliz@HIDDEN>
Message-ID: <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

On February 27, 2018 11:22:26 AM GMT+02:00, Michael Heerdegen <michael_heer=
degen@web=2Ede> wrote:
>=20
> Hello,
>=20
> Traversing a `stream-of-directory-files' over a huge directory
> hierarchy
> crashes my Emacs=2E
>=20
> Here is a recipe: I have a file
> "/home/micha/Treasure/today/stream-crash=2Eel" with these contents:
>=20
> #+begin_src emacs-lisp
> ;; -*- lexical-binding: t -*-
>=20
> (require 'stream)
>=20
> (seq-doseq (_file (stream-of-directory-files
>                    "/home/micha" t nil t nil
>   (lambda (file) (and (file-readable-p file) (file-regular-p file)))))
>   nil)
> #+end_src
>=20
> Then I
>=20
> micha> emacs -Q -L /home/micha/software/elpa/packages/stream\
>                 -l /home/micha/Treasure/today/stream-crash=2Eel
>=20
> and I get a segfault after ~ 1 minute=2E
>=20
> This kind of crash only seems to occur when the traversed directory is
> sufficiently large=2E


I guess it's a stack overflow: that function recurses into subdirectories=
=2E

To avoid such problems, the function should be rewritten to work by BFS, n=
ot DFS=2E




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 09:22:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 27 04:22:41 2018
Received: from localhost ([127.0.0.1]:34291 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1eqbT7-0000PM-1B
	for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 04:22:41 -0500
Received: from eggs.gnu.org ([208.118.235.92]:42421)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqbT5-0000P9-CM
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 04:22:39 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqbSz-00050h-8o
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 04:22:34 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:51747)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <michael_heerdegen@HIDDEN>)
 id 1eqbSz-00050d-5r
 for submit <at> debbugs.gnu.org; Tue, 27 Feb 2018 04:22:33 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:33377)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqbSx-0000Te-St
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 04:22:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <michael_heerdegen@HIDDEN>) id 1eqbSu-0004yt-PD
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 04:22:31 -0500
Received: from mout.web.de ([212.227.15.4]:45033)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <michael_heerdegen@HIDDEN>)
 id 1eqbSu-0004yL-FY
 for bug-gnu-emacs@HIDDEN; Tue, 27 Feb 2018 04:22:28 -0500
Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb003
 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LwYtb-1eeyho0Ytb-018NMU; Tue, 27
 Feb 2018 10:22:27 +0100
From: Michael Heerdegen <michael_heerdegen@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 26.0.91; Crash when traversing a `stream-of-directory-files'
Date: Tue, 27 Feb 2018 10:22:26 +0100
Message-ID: <87inaiss6l.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K0:hvKP+gAsqC03NREGBt//pGSayUGuY2ABDOvOb0RAPGYDzRGAkn3
 OrMKuHQrTkTNg7jzTEdzLgvvdAVgVK6Tld3XZMWqySQZ85+rXbAhykqNt/Fds+xIQxW15Cn
 /9VuSBPGiVcKz4nKWvHs74ulsjl7SiYo4g1gA0zfY6zhVJMBIBj9sQITs6gXlielMmg9QP3
 yzjsb2q7ARqgf5ppcDzfA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:7KfwDX0AHwg=:UpjL3/WxjTtZPIH7JA/Szp
 ItxGqosS+uSkBXcqf5VoTIKdewxotSfcCE8J+S8RRsYWYXJ0tgxSrBcT4cKocP9gOoBEZuNyS
 qWAcvc3gEAgXDY/WH11QH8ZDmskgjI3f4fuWY4iDow/waiF0N22H+z12Q1YhSBIohCWTvQv97
 pvluiC4L5TzJBT+CIN29YdZlx/kEAAAgbSWyIrDXrRr6KMn6mIUzOsQNU806lq2nLxscqww8Q
 sCCYBmcp6ydc+mHcHMvUcsjb3aa0/q83YFKoNATA4v/Sz5Ytg2cQb+O2dSLzLcNdNH3a/h6fI
 wpO/tMNTGKiF4X6a1WSS7zQ8sI6XZYFnMJaFEnJrirnFhqLuvCmzjA5IfxR+2OIItXvh+03dh
 sj11c3K5bL1qdEcnoQA56wwsAnNbftbk0WEjiQ3UIveO1wtxPnpcpE3CEqaAesZ9RejGjFkEd
 F/HHDCrPjVwob7Imv3aNIsZAWcWopDtHQnmQ4jaVSJwofIo0L+Fu9qhUBSEPW6sqyNIne9GIJ
 DXQQSqQUVCM6Lry575QbNk0gU3bGd8wKNOzfbLCxjB678TfJJNFfROAzzDw0SQUR0KbUDnX8Q
 10BSrTZfpdZhHBWQjsJRtJlDQE8l+CneDZkVV5VT99jckZwFHn20Tm4hW+gTiFTWP6wme6r4U
 pTKJldw7KvnBtPmjFxIQnpyqFI4U5uIiW3e8CezEU4OyN0f8VsPQyyS1J1c49Zq+GaCVkUISn
 qtGB0UE9lbjJ/FUyIkex+t/XW+g8L/MV1JiIjeKJoVFknzATgsyV+BwshHeBo5mDCugMOA6zh
 lyViz7nj0Sinv+AbcbXfzUfpy/VDGGYfXldX8J1lrLwqYzBFzQ=
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
 [fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.1 (----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.1 (----)


Hello,

Traversing a `stream-of-directory-files' over a huge directory hierarchy
crashes my Emacs.

Here is a recipe: I have a file
"/home/micha/Treasure/today/stream-crash.el" with these contents:

#+begin_src emacs-lisp
;; -*- lexical-binding: t -*-

(require 'stream)

(seq-doseq (_file (stream-of-directory-files
                   "/home/micha" t nil t nil
                   (lambda (file) (and (file-readable-p file) (file-regular-p file)))))
  nil)
#+end_src

Then I

micha> emacs -Q -L /home/micha/software/elpa/packages/stream\
                -l /home/micha/Treasure/today/stream-crash.el

and I get a segfault after ~ 1 minute.

This kind of crash only seems to occur when the traversed directory is
sufficiently large.


Thanks in advance,

Michael.


In GNU Emacs 26.0.91 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.28)
 of 2018-02-26 built on drachen
Repository revision: ea8f9fd7be138708a52aad418d09d5d4ca6b2a7e
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description:	Debian GNU/Linux testing (buster)





Acknowledgement sent to Michael Heerdegen <michael_heerdegen@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#30626; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sun, 4 Mar 2018 17:15:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.