GNU bug report logs -
#45926
Tabulated-list-mode wasteful properties
Previous Next
Reported by: Boruch Baum <boruch_baum <at> gmx.com>
Date: Sun, 17 Jan 2021 07:34:02 UTC
Severity: minor
Tags: moreinfo, notabug
Done: Stefan Kangas <stefan <at> marxist.se>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 45926 in the body.
You can then email your comments to 45926 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Sun, 17 Jan 2021 07:34:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Boruch Baum <boruch_baum <at> gmx.com>
:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org
.
(Sun, 17 Jan 2021 07:34:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Tabulated lists can grow quite large. In fact, the most common use for
the mode is the *Packages* buffer listing, which for me today clocks in
at 5211 lines * five columns. In the case of the *Packages* buffer, Performing
(buffer-substring (point) (line-end-position))
on a sampling of lines yields 'PP eval' buffers of length ~500 - ~700
characters.
Most of this looks to me to be unnecessary duplicate data. In using the
mode for my own purpose, I see that the mode is putting all data of all
columns in an entry (row) in each and every single column, ie the size
grows exponentially by number of columns (in my personal case, ten
columns).
This isn't breaking anything, but it looks like bad design unnecessarily
hogging memory and filling buffers with cruft.
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Sun, 17 Jan 2021 13:41:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 45926 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Boruch Baum <boruch_baum <at> gmx.com> writes:
> Tabulated lists can grow quite large. In fact, the most common use for
> the mode is the *Packages* buffer listing, which for me today clocks in
> at 5211 lines * five columns. In the case of the *Packages* buffer, Performing
>
> (buffer-substring (point) (line-end-position))
>
> on a sampling of lines yields 'PP eval' buffers of length ~500 - ~700
> characters.
>
> Most of this looks to me to be unnecessary duplicate data. In using the
> mode for my own purpose, I see that the mode is putting all data of all
> columns in an entry (row) in each and every single column, ie the size
> grows exponentially by number of columns (in my personal case, ten
> columns).
>
> This isn't breaking anything, but it looks like bad design unnecessarily
> hogging memory and filling buffers with cruft.
Are you sure the data is actually duplicated and doesn't share
structure? I evaluated that expression over the xref package, and got
the attached string. Nothing looks obviously duplicated, except for the
shared #N= and #N# structures. Also, some of the repetition of shared
structures is probably just due to how string intervals are
represented/printed.
Not that any of this is necessarily optimal, of course, but it's not
immediately alarming to me, either, unless I'm missing something.
--
Basil
[pp-eval.el (application/emacs-lisp, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Sun, 17 Jan 2021 15:12:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 45926 <at> debbugs.gnu.org (full text, mbox):
On 2021-01-17 13:40, Basil L. Contovounesios wrote:
> Boruch Baum <boruch_baum <at> gmx.com> writes:
>
> Are you sure the data is actually duplicated and doesn't share
> structure?
Until you wrote, I definitely was. I think I still am, but I'm willing
to look again. It's easier for me since I just recently wrote a
ten-column tabulated list mode application. In my case I do see it
clearly because the text of each row in my tabulated list isn't
disturbed/confused by any text properties, and I do see an exact
duplication of each entire row entry in the property for each column of
that property.
I expect to publish the software within a day or two, and plan to
announce it on the emacs-devel and possibly other lists, at which time
you'll be able to see the internals for yourself. The package is planned
to be named 'crossword.el' and it brings to emacs the ability to
download and play crossword puzzles.
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Tue, 19 Jan 2021 05:26:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 45926 <at> debbugs.gnu.org (full text, mbox):
Boruch Baum <boruch_baum <at> gmx.com> writes:
> Until you wrote, I definitely was. I think I still am, but I'm willing
> to look again.
Like Basil, I looked at a tabulated list line, and I didn't see any
excessive duplication -- tabulated list mode puts the data on the entire
line, but it's the same data structure, so it's all shared data, I
think?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
Severity set to 'minor' from 'normal'
Request was from
Stefan Kangas <stefan <at> marxist.se>
to
control <at> debbugs.gnu.org
.
(Fri, 26 Feb 2021 01:17:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Fri, 12 Mar 2021 00:53:02 GMT)
Full text and
rfc822 format available.
Message #19 received at 45926 <at> debbugs.gnu.org (full text, mbox):
Lars Ingebrigtsen <larsi <at> gnus.org> writes:
> Boruch Baum <boruch_baum <at> gmx.com> writes:
>
>> Until you wrote, I definitely was. I think I still am, but I'm willing
>> to look again.
Borouch, did you have the time to look into if this data is indeed
shared or not? Thanks.
> Like Basil, I looked at a tabulated list line, and I didn't see any
> excessive duplication -- tabulated list mode puts the data on the entire
> line, but it's the same data structure, so it's all shared data, I
> think?
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Fri, 12 Mar 2021 02:58:01 GMT)
Full text and
rfc822 format available.
Message #22 received at 45926 <at> debbugs.gnu.org (full text, mbox):
On 2021-03-11 18:52, Stefan Kangas wrote:
> Lars Ingebrigtsen <larsi <at> gnus.org> writes:
>
> > Boruch Baum <boruch_baum <at> gmx.com> writes:
> >
> >> Until you wrote, I definitely was. I think I still am, but I'm willing
> >> to look again.
>
> Borouch, did you have the time to look into if this data is indeed
> shared or not? Thanks.
Oh. I thought I had done that right away ...
Doing it again now. Yes. It's very simple to check for yourself. Open a
buffer that uses the mode and evaluate from any table line ...
(buffer-substring (line-beginning-position) (line-end-position))
You should be able to clearly see all the duplication. It's easiest when
you have pretty-print set up to display the output in a dedicated
buffer, but even without that you can isearch through the duplications.
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Fri, 12 Mar 2021 03:40:02 GMT)
Full text and
rfc822 format available.
Message #25 received at 45926 <at> debbugs.gnu.org (full text, mbox):
Boruch Baum <boruch_baum <at> gmx.com> writes:
>> Borouch, did you have the time to look into if this data is indeed
>> shared or not? Thanks.
>
> Oh. I thought I had done that right away ...
>
> Doing it again now. Yes. It's very simple to check for yourself. Open a
> buffer that uses the mode and evaluate from any table line ...
>
> (buffer-substring (line-beginning-position) (line-end-position))
>
> You should be able to clearly see all the duplication. It's easiest when
> you have pretty-print set up to display the output in a dedicated
> buffer, but even without that you can isearch through the duplications.
I think the interesting question is if that data is duplicated in memory
or if these are just pointers to the same shared data.
I would try modifying this data with something like `setcdr' to see if
it updates everywhere -- if the answer yes, then this data is shared and
there is no problem here.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Fri, 12 Mar 2021 05:47:01 GMT)
Full text and
rfc822 format available.
Message #28 received at 45926 <at> debbugs.gnu.org (full text, mbox):
On 2021-03-11 21:39, Stefan Kangas wrote:
> I would try modifying this data with something like `setcdr' to see if
> it updates everywhere -- if the answer yes, then this data is shared and
> there is no problem here.
Why are you telling me this when it's just as easy for you to do it
yourself?
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Fri, 12 Mar 2021 10:52:01 GMT)
Full text and
rfc822 format available.
Message #31 received at 45926 <at> debbugs.gnu.org (full text, mbox):
Boruch Baum <boruch_baum <at> gmx.com> writes:
> Doing it again now. Yes. It's very simple to check for yourself. Open a
> buffer that uses the mode and evaluate from any table line ...
>
> (buffer-substring (line-beginning-position) (line-end-position))
>
> You should be able to clearly see all the duplication. It's easiest when
> you have pretty-print set up to display the output in a dedicated
> buffer, but even without that you can isearch through the duplications.
Have you tried setting print-circle non-nil first? This will help you
see what is actually duplicated and what is shared.
--
Basil
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Fri, 12 Mar 2021 11:02:01 GMT)
Full text and
rfc822 format available.
Message #34 received at 45926 <at> debbugs.gnu.org (full text, mbox):
Boruch Baum <boruch_baum <at> gmx.com> writes:
> On 2021-03-11 21:39, Stefan Kangas wrote:
>> I would try modifying this data with something like `setcdr' to see if
>> it updates everywhere -- if the answer yes, then this data is shared and
>> there is no problem here.
>
> Why are you telling me this when it's just as easy for you to do it
> yourself?
The opinions voiced so far are that there does not seem to be excessive
duplication of data, since most of it is shared.
You can help the discussion by providing either evidence to the
contrary, or a patch that demonstrably lowers the memory usage of
tabulated-list-mode and the package list.
Stefan is being helpful by nudging a bug discussion that fell silent; it
is unfair to expect him to do all the work (and where's the fun in that
anyway ;).
--
Basil
Added tag(s) moreinfo and notabug.
Request was from
Stefan Kangas <stefan <at> marxist.se>
to
control <at> debbugs.gnu.org
.
(Sat, 13 Mar 2021 07:13:02 GMT)
Full text and
rfc822 format available.
Reply sent
to
Stefan Kangas <stefan <at> marxist.se>
:
You have taken responsibility.
(Wed, 21 Apr 2021 03:36:03 GMT)
Full text and
rfc822 format available.
Notification sent
to
Boruch Baum <boruch_baum <at> gmx.com>
:
bug acknowledged by developer.
(Wed, 21 Apr 2021 03:36:03 GMT)
Full text and
rfc822 format available.
Message #41 received at 45926-done <at> debbugs.gnu.org (full text, mbox):
Stefan Kangas <stefan <at> marxist.se> writes:
> Boruch Baum <boruch_baum <at> gmx.com> writes:
>
>>> Borouch, did you have the time to look into if this data is indeed
>>> shared or not? Thanks.
>>
>> Oh. I thought I had done that right away ...
>>
>> Doing it again now. Yes. It's very simple to check for yourself. Open a
>> buffer that uses the mode and evaluate from any table line ...
>>
>> (buffer-substring (line-beginning-position) (line-end-position))
>>
>> You should be able to clearly see all the duplication. It's easiest when
>> you have pretty-print set up to display the output in a dedicated
>> buffer, but even without that you can isearch through the duplications.
>
> I think the interesting question is if that data is duplicated in memory
> or if these are just pointers to the same shared data.
>
> I would try modifying this data with something like `setcdr' to see if
> it updates everywhere -- if the answer yes, then this data is shared and
> there is no problem here.
More information was requested, but none was given within 5 weeks, so
I'm closing this bug.
If anyone can provide further information on this issue to show that
there is a bug here, please reply to this email (use "Reply to all" in
your email client) and we can reopen the bug report.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Wed, 21 Apr 2021 04:34:01 GMT)
Full text and
rfc822 format available.
Message #44 received at 45926 <at> debbugs.gnu.org (full text, mbox):
* Boruch Baum <boruch_baum <at> gmx.com> [2021-03-12 05:58]:
> On 2021-03-11 18:52, Stefan Kangas wrote:
> > Lars Ingebrigtsen <larsi <at> gnus.org> writes:
> >
> > > Boruch Baum <boruch_baum <at> gmx.com> writes:
> > >
> > >> Until you wrote, I definitely was. I think I still am, but I'm willing
> > >> to look again.
> >
> > Borouch, did you have the time to look into if this data is indeed
> > shared or not? Thanks.
>
> Oh. I thought I had done that right away ...
>
> Doing it again now. Yes. It's very simple to check for yourself. Open a
> buffer that uses the mode and evaluate from any table line ...
>
> (buffer-substring (line-beginning-position) (line-end-position))
>
> You should be able to clearly see all the duplication. It's easiest when
> you have pretty-print set up to display the output in a dedicated
> buffer, but even without that you can isearch through the
> duplications.
I have done this excercise, as I have people management in the
database.
If tabulated-list-entries is ⇒ ((320295 ["320295" "Boruch Baum" "Emacs Users"]))
Then this below is what I get when invoking: (buffer-substring (line-beginning-position) (line-end-position)):
#(" 320295 Boruch Baum Emacs Users" 0 1 (tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 1 7 (help-echo "ID: 320295" tabulated-list-column-name "ID" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 7 8 (tabulated-list-column-name "ID" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 8 10 (display (space :align-to 10) tabulated-list-column-name "ID" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 10 22 (help-echo "Name: Boruch Baum" tabulated-list-column-name "Name" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 22 23 (tabulated-list-column-name "Name" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 23 51 (display (space :align-to 51) tabulated-list-column-name "Name" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 51 62 (help-echo "Account: Emacs Users" tabulated-list-column-name "Account" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]))
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#45926
; Package
emacs
.
(Wed, 21 Apr 2021 04:39:02 GMT)
Full text and
rfc822 format available.
Message #47 received at 45926 <at> debbugs.gnu.org (full text, mbox):
* Basil L. Contovounesios <contovob <at> tcd.ie> [2021-03-12 13:52]:
> Boruch Baum <boruch_baum <at> gmx.com> writes:
>
> > Doing it again now. Yes. It's very simple to check for yourself. Open a
> > buffer that uses the mode and evaluate from any table line ...
> >
> > (buffer-substring (line-beginning-position) (line-end-position))
> >
> > You should be able to clearly see all the duplication. It's easiest when
> > you have pretty-print set up to display the output in a dedicated
> > buffer, but even without that you can isearch through the duplications.
>
> Have you tried setting print-circle non-nil first? This will help you
> see what is actually duplicated and what is shared.
I have done that exercise where my tabulated-list-entries is only
this:
⇒ ((320295 ["320295" "Boruch Baum" "Emacs Users"]))
and
(buffer-substring (line-beginning-position) (line-end-position))
gives me this:
a ⇒ #(" 320295 Boruch Baum Emacs Users" 0 1 (tabulated-list-id 320295 tabulated-list-entry #1=["320295" "Boruch Baum" "Emacs Users"]) 1 7 (help-echo "ID: 320295" tabulated-list-column-name #3="ID" tabulated-list-id 320295 tabulated-list-entry #1#) 7 8 (tabulated-list-column-name #3# tabulated-list-id 320295 tabulated-list-entry #1#) 8 10 (display (space :align-to 10) tabulated-list-column-name #3# tabulated-list-id 320295 tabulated-list-entry #1#) 10 22 (help-echo "Name: Boruch Baum" tabulated-list-column-name #2="Name" tabulated-list-id 320295 tabulated-list-entry #1#) 22 23 (tabulated-list-column-name #2# tabulated-list-id 320295 tabulated-list-entry #1#) 23 51 (display (space :align-to 51) tabulated-list-column-name #2# tabulated-list-id 320295 tabulated-list-entry #1#) 51 62 (help-echo "Account: Emacs Users" tabulated-list-column-name "Account" tabulated-list-id 320295 tabulated-list-entry #1#))
If non-nil, shared substructures anywhere in the structure are printed
with ‘#N=’ before the first occurrence (in the order of the print
representation) and ‘#N#’ in place of each subsequent occurrence,
where N is a positive decimal integer.
Thus I can see that those are shared structures mostly.
Jean
Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns
Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Wed, 19 May 2021 11:24:05 GMT)
Full text and
rfc822 format available.
This bug report was last modified 2 years and 315 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.