GNU bug report logs -
#18051
24.3.92; ls-lisp: Sorting; make ls-lisp-string-lessp a normal function?
Previous Next
Reported by: michael_heerdegen <at> web.de
Date: Fri, 18 Jul 2014 06:24:01 UTC
Severity: wishlist
Found in version 24.3.92
Done: Eli Zaretskii <eliz <at> gnu.org>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18051 in the body.
You can then email your comments to 18051 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 06:24:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
michael_heerdegen <at> web.de:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Fri, 18 Jul 2014 06:24:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Hello,
Some users will want to configure the sorting predicate used by ls-lisp,
for example, to get a natural sorting of version numbers, or to sort in
files whose names start with a dot as if they had no dot, etc.
Currently, sorting is even hardcoded because `ls-lisp-string-lessp' is a
defsubst. If it was a normal function, one could advice it.
Or, with some more efforts, sorting order could be made configurable via
options, and the -v switch could be implemented.
Regards,
Michael.
In GNU Emacs 24.3.92.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2)
of 2014-07-17 on drachen
Windowing system distributor `The X.Org Foundation', version 11.0.11599904
System Description: Debian GNU/Linux testing (jessie)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 06:55:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Date: Fri, 18 Jul 2014 08:22:43 +0200
>
> Some users will want to configure the sorting predicate used by ls-lisp,
> for example, to get a natural sorting of version numbers, or to sort in
> files whose names start with a dot as if they had no dot, etc.
>
> Currently, sorting is even hardcoded because `ls-lisp-string-lessp' is a
> defsubst. If it was a normal function, one could advice it.
>
> Or, with some more efforts, sorting order could be made configurable via
> options, and the -v switch could be implemented.
ls-lisp emulates the Unix and GNU 'ls'. So I will generally oppose to
introducing any option into it that cannot be had with an external
'ls' program, as long as the latter is the main method of getting a
Dired buffer. (If Emacs ever decides that ls-lisp becomes the main
method, and will use it by default on all supported platforms, this
objection will no longer be valid, of course.)
An alternative to what you want would be a Dired-level feature, which
then will be available also to those who don't use ls-lisp.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 07:34:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> ls-lisp emulates the Unix and GNU 'ls'. So I will generally oppose to
> introducing any option into it that cannot be had with an external
> 'ls' program, as long as the latter is the main method of getting a
> Dired buffer. (If Emacs ever decides that ls-lisp becomes the main
> method, and will use it by default on all supported platforms, this
> objection will no longer be valid, of course.)
That's a bit what I expected, and makes sense. I would welcome ls-lisp
to become the default, btw.
Sorting in dired could generally be improved a lot. If you want sorting
by size, you need to edit the switches, which is not very handy (there
are addons for that job). I needed to define the whole
`dired-sort-toggle-or-edit' because it hardcodes -t whereby I prefer -c.
> An alternative to what you want would be a Dired-level feature, which
> then will be available also to those who don't use ls-lisp.
That would be a really good thing (if it doesn't slow down things). But
before this is reality ... could I please get my `ls-lisp-string-lessp'
defined as a function?
Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 08:54:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: 18051 <at> debbugs.gnu.org
> Date: Fri, 18 Jul 2014 09:33:21 +0200
>
> could I please get my `ls-lisp-string-lessp' defined as a function?
You can always redefine the functions that call it, no?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 09:26:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Michael Heerdegen <michael_heerdegen <at> web.de> writes:
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
>> ls-lisp emulates the Unix and GNU 'ls'. So I will generally oppose to
>> introducing any option into it that cannot be had with an external
>> 'ls' program, as long as the latter is the main method of getting a
>> Dired buffer. (If Emacs ever decides that ls-lisp becomes the main
>> method, and will use it by default on all supported platforms, this
>> objection will no longer be valid, of course.)
>
> That's a bit what I expected, and makes sense. I would welcome ls-lisp
> to become the default, btw.
Tramp uses ls-lisp only in case it cannot use a native method on the
remote host. Experience shows, that ls-lisp has a much worse performance
for remote directories than native implementations.
I would oppose to make ls-lisp the default, and to add functionality to
it which would not be available otherwise. Such additional functionality
must be added to file name functions with a file name handler, if
desired.
> Michael.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 09:34:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 18051 <at> debbugs.gnu.org
> Date: Fri, 18 Jul 2014 11:24:59 +0200
>
> Tramp uses ls-lisp only in case it cannot use a native method on the
> remote host. Experience shows, that ls-lisp has a much worse performance
> for remote directories than native implementations.
Any insight as to why this happens? Perhaps the Tramp implementation
of directory-files-and-attributes needs some love?
> I would oppose to make ls-lisp the default, and to add functionality to
> it which would not be available otherwise.
If this is because of Tramp, nothing prevents us from using 'ls' on
the remote host, and then manipulate the results locally in Lisp,
right? So I'm not sure I understand the rationale for your
objections. Perhaps revealing more details will help.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 09:38:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> > could I please get my `ls-lisp-string-lessp' defined as a function?
>
> You can always redefine the functions that call it, no?
That would cover the whole switch processing algorithm,
`ls-lisp-handle-switches', 60 lines of code. I would shadow any future
change in that code by redefining that huge function. That's why I
wanted to avoid that.
What's the advantage of `ls-lisp-string-lessp' being a defsubst? If it
really makes it significantly faster (it's called a lot of times, of
course), then ok, let's keep it.
Thanks,
Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 09:47:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: 18051 <at> debbugs.gnu.org
> Date: Fri, 18 Jul 2014 11:37:19 +0200
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> > > could I please get my `ls-lisp-string-lessp' defined as a function?
> >
> > You can always redefine the functions that call it, no?
>
> That would cover the whole switch processing algorithm,
> `ls-lisp-handle-switches', 60 lines of code. I would shadow any future
> change in that code by redefining that huge function. That's why I
> wanted to avoid that.
You are going to override the behavior of the package anyway, so I
don't see the big difference.
> What's the advantage of `ls-lisp-string-lessp' being a defsubst?
From my POV, making sure the package always behaves as designed, I
guess. You agreed with my motivation, so it sounds like a
contradiction to me to still push for the change.
Anyway, if others think the comparison of file names should be up for
grabs, I won't fight the change just because I think it's wrong.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 10:14:01 GMT)
Full text and
rfc822 format available.
Message #29 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
Hi Eli,
>> Tramp uses ls-lisp only in case it cannot use a native method on the
>> remote host. Experience shows, that ls-lisp has a much worse performance
>> for remote directories than native implementations.
>
> Any insight as to why this happens? Perhaps the Tramp implementation
> of directory-files-and-attributes needs some love?
Maybe it is a misunderstanding. Tramp's native implementation is much
faster, because it sends exactly one remote command. For ssh-like
connections, this is for example
# echo "("; (/bin/ls --color=never -a | sed -e s/\$/\"/g -e s/^/\"/g | xargs \stat -c '("%n" ("%N") %h %ue0 %ge0 %Xe0 %Ye0 %Ze0 %se0 "%A" t %ie0 -1)' 2>/dev/null); echo ")" 2>/dev/null
This is much faster than ls-lisp, which must determine file-attributes
for every single file in a remote directory (which is a remote command
on its own).
>> I would oppose to make ls-lisp the default, and to add functionality to
>> it which would not be available otherwise.
>
> If this is because of Tramp, nothing prevents us from using 'ls' on
> the remote host, and then manipulate the results locally in Lisp,
> right? So I'm not sure I understand the rationale for your
> objections. Perhaps revealing more details will help.
I would oppose only if there is an additional mandatory functionality in
ls-lisp other file name primitives are urged to use. If there would
be changes in, let's say, directory-files-and-attributes, there's no
problem for me. But that's not what Michael has asked for.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 10:19:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> From my POV, making sure the package always behaves as designed, I
> guess. You agreed with my motivation, so it sounds like a
> contradiction to me to still push for the change.
Seems I again misunderstood what you meant with:
> [...] So I will generally oppose to introducing any option into it
> that cannot be had with an external 'ls' program, as long as the
> latter is the main method of getting a
I had read that as introducing "user options", but you obviously meant
introducing options for changing the behavior using any means.
> Anyway, if others think the comparison of file names should be up for
> grabs, I won't fight the change just because I think it's wrong.
At least we agree that there's room for improvement.
Anyway, what I want to reach:
(1) sort in files starting with a dot as if they had no dot
(2) -v sorting (sorting versions correctly)
isn't that both possible with ls? (2) obviously; I think (1) depends on
the system's language setting?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 12:59:02 GMT)
Full text and
rfc822 format available.
Message #35 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Fri, 18 Jul 2014 12:12:48 +0200
>
> >> Tramp uses ls-lisp only in case it cannot use a native method on the
> >> remote host. Experience shows, that ls-lisp has a much worse performance
> >> for remote directories than native implementations.
> >
> > Any insight as to why this happens? Perhaps the Tramp implementation
> > of directory-files-and-attributes needs some love?
>
> Maybe it is a misunderstanding. Tramp's native implementation is much
> faster, because it sends exactly one remote command. For ssh-like
> connections, this is for example
>
> # echo "("; (/bin/ls --color=never -a | sed -e s/\$/\"/g -e s/^/\"/g | xargs \stat -c '("%n" ("%N") %h %ue0 %ge0 %Xe0 %Ye0 %Ze0 %se0 "%A" t %ie0 -1)' 2>/dev/null); echo ")" 2>/dev/null
We could easily add this to ls-lisp, in case the directory is remote.
Right now, it simply doesn't support remote directories, because I
didn't know there was any interest in that.
> I would oppose only if there is an additional mandatory functionality in
> ls-lisp other file name primitives are urged to use. If there would
> be changes in, let's say, directory-files-and-attributes, there's no
> problem for me. But that's not what Michael has asked for.
I don't think you understood what Michael wanted, but I'll let Michael
speak for himself.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 13:04:02 GMT)
Full text and
rfc822 format available.
Message #38 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: 18051 <at> debbugs.gnu.org
> Date: Fri, 18 Jul 2014 12:18:19 +0200
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> > From my POV, making sure the package always behaves as designed, I
> > guess. You agreed with my motivation, so it sounds like a
> > contradiction to me to still push for the change.
>
> Seems I again misunderstood what you meant with:
>
> > [...] So I will generally oppose to introducing any option into it
> > that cannot be had with an external 'ls' program, as long as the
> > latter is the main method of getting a
>
> I had read that as introducing "user options", but you obviously meant
> introducing options for changing the behavior using any means.
Yes, I meant adding code that would support functionality not
available when 'ls' is used.
> > Anyway, if others think the comparison of file names should be up for
> > grabs, I won't fight the change just because I think it's wrong.
>
> At least we agree that there's room for improvement.
That's trivial: there always is, including in ls-lisp.
> Anyway, what I want to reach:
>
> (1) sort in files starting with a dot as if they had no dot
Why? Personally, it would mightily confuse me: I always expect to
find all the dot-files together. This is useful, e.g., when I'm
looking for init file related to some feature, but I don't know the
exact name of that file.
But if 'ls' supports that, so should ls-lisp.
> (2) -v sorting (sorting versions correctly)
Isn't this what "ls -v" does? If so, and if ls-lisp doesn't currently
support that, patches to add such support are welcome.
> isn't that both possible with ls? (2) obviously; I think (1) depends on
> the system's language setting?
Patches to support any feature available in some 'ls' are more than
welcome. TIA.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 13:20:01 GMT)
Full text and
rfc822 format available.
Message #41 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> Maybe it is a misunderstanding. Tramp's native implementation is much
>> faster, because it sends exactly one remote command. For ssh-like
>> connections, this is for example
>>
>> # echo "("; (/bin/ls --color=never -a | sed -e s/\$/\"/g -e s/^/\"/g
>> | xargs \stat -c '("%n" ("%N") %h %ue0 %ge0 %Xe0 %Ye0 %Ze0 %se0 "%A"
>> t %ie0 -1)' 2>/dev/null); echo ")" 2>/dev/null
>
> We could easily add this to ls-lisp, in case the directory is remote.
> Right now, it simply doesn't support remote directories, because I
> didn't know there was any interest in that.
No, that's not needed. Tramp does its job for different target
architectures in different ways. For example, if the stat command is not
available on the remote host, it uses another implementation with perl
for directory-files-and-attributes, and so on.
ls-lisp does not support file name handlers, and Tramp uses it only
internally in case it doesn't know better. Support for remote
directories would mean to add a file name handler for ls-lisp - is this
what you have in mind? I don't believe it is necessary.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 13:45:02 GMT)
Full text and
rfc822 format available.
Message #44 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Fri, 18 Jul 2014 15:18:40 +0200
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> >> Maybe it is a misunderstanding. Tramp's native implementation is much
> >> faster, because it sends exactly one remote command. For ssh-like
> >> connections, this is for example
> >>
> >> # echo "("; (/bin/ls --color=never -a | sed -e s/\$/\"/g -e s/^/\"/g
> >> | xargs \stat -c '("%n" ("%N") %h %ue0 %ge0 %Xe0 %Ye0 %Ze0 %se0 "%A"
> >> t %ie0 -1)' 2>/dev/null); echo ")" 2>/dev/null
> >
> > We could easily add this to ls-lisp, in case the directory is remote.
> > Right now, it simply doesn't support remote directories, because I
> > didn't know there was any interest in that.
>
> No, that's not needed. Tramp does its job for different target
> architectures in different ways. For example, if the stat command is not
> available on the remote host, it uses another implementation with perl
> for directory-files-and-attributes, and so on.
We are talking past each other. What I meant was to add to ls-lisp
support for remote directories, which will simply invoke Tramp's
handlers for that.
> Support for remote directories would mean to add a file name handler
> for ls-lisp - is this what you have in mind?
Yes.
> I don't believe it is necessary.
It is necessary if ls-lisp will ever become the default.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 18 Jul 2014 16:22:01 GMT)
Full text and
rfc822 format available.
Message #47 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> Support for remote directories would mean to add a file name handler
>> for ls-lisp - is this what you have in mind?
>
> Yes.
>
>> I don't believe it is necessary.
>
> It is necessary if ls-lisp will ever become the default.
Well, in this case I don't know which functionality shall be added to
ls-lisp-insert-directory, which couldn't be added directly to
insert-directory. The latter function calls file name handlers already,
if needed.
(I still have the impression we're speaking about different
topics. Sorry for my stupidness)
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 19 Jul 2014 01:26:02 GMT)
Full text and
rfc822 format available.
Message #50 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> > Anyway, what I want to reach:
> >
> > (1) sort in files starting with a dot as if they had no dot
>
> Why? Personally, it would mightily confuse me: I always expect to
> find all the dot-files together. This is useful, e.g., when I'm
> looking for init file related to some feature, but I don't know the
> exact name of that file.
But currently all dot files are listed before all the other files.
I find that annoying most of the time. Dunno yet if I like my current
setup, I'll see - but over the current sorting, I would much prefer
having all the dot files at the end. (I don't know if this is possible
with ls.)
> But if 'ls' supports that, so should ls-lisp.
It depends on "locale" settings. I tried some settings of the LC_ALL
variable. With "C" or "POSIX", I get the same sorting as with ls-lisp.
OTOH, with "en_US.utf8" or "de_DE.utf8" I get the sorting I described,
with dot files merged with the other files.
> > (2) -v sorting (sorting versions correctly)
>
> Isn't this what "ls -v" does? If so, and if ls-lisp doesn't currently
> support that, patches to add such support are welcome.
ls -v sorts backup versions in their natural order (which is not the
lexicographic order). Yes, would be good to have that in ls-lisp, and
should not be too hard. I can give it a try when I get the time. But
I'm not sure what would have to be done about the locale depend part.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 19 Jul 2014 08:18:01 GMT)
Full text and
rfc822 format available.
Message #53 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: 18051 <at> debbugs.gnu.org
> Date: Sat, 19 Jul 2014 03:25:01 +0200
>
> over the current sorting, I would much prefer having all the dot
> files at the end. (I don't know if this is possible with ls.)
If 'ls' cannot do that, I think we should have this in Dired.
> > But if 'ls' supports that, so should ls-lisp.
>
> It depends on "locale" settings. I tried some settings of the LC_ALL
> variable. With "C" or "POSIX", I get the same sorting as with ls-lisp.
> OTOH, with "en_US.utf8" or "de_DE.utf8" I get the sorting I described,
> with dot files merged with the other files.
AFAICT, 'ls' uses strcoll. I don't see how can that effectively
ignore the leading dot, but maybe I'm missing something. OTOH, if the
UTF-8 codeset says the leading dot should be ignored, then ls-lisp
should do the same by default, at least when the locale's codeset is
UTF-8.
Can you see the answer in 'ls' sources? Does just "en_US" or
"en_US.8859-1" change the order?
> > > (2) -v sorting (sorting versions correctly)
> >
> > Isn't this what "ls -v" does? If so, and if ls-lisp doesn't currently
> > support that, patches to add such support are welcome.
>
> ls -v sorts backup versions in their natural order (which is not the
> lexicographic order). Yes, would be good to have that in ls-lisp, and
> should not be too hard. I can give it a try when I get the time.
Thanks.
> I'm not sure what would have to be done about the locale depend part.
Assuming it's indeed the locale thing, Emacs doesn't yet support
locale-specific sorting. But we could do that in some ad-hoc manner
anyway.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 19 Jul 2014 10:53:01 GMT)
Full text and
rfc822 format available.
Message #56 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> AFAICT, 'ls' uses strcoll. I don't see how can that effectively
> ignore the leading dot, but maybe I'm missing something.
Yes, it uses strcoll.
> Can you see the answer in 'ls' sources?
No, I don't see anything related in the sources, it just calls strcoll
(or strcmp as fallback), nothing more.
I compiled some test program calling strcoll, and there it didn't ignore
the dot. Sorry, I don't why this is different in ls.
> Does just "en_US" or "en_US.8859-1" change the order?
Yes! ls -al with en_US behaves the same as with en_US.utf8, dots are
ignored.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 19 Jul 2014 10:57:01 GMT)
Full text and
rfc822 format available.
Message #59 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> Date: Sat, 19 Jul 2014 11:17:15 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 18051 <at> debbugs.gnu.org
>
> > > But if 'ls' supports that, so should ls-lisp.
> >
> > It depends on "locale" settings. I tried some settings of the LC_ALL
> > variable. With "C" or "POSIX", I get the same sorting as with ls-lisp.
> > OTOH, with "en_US.utf8" or "de_DE.utf8" I get the sorting I described,
> > with dot files merged with the other files.
>
> AFAICT, 'ls' uses strcoll. I don't see how can that effectively
> ignore the leading dot, but maybe I'm missing something.
I think I know the answer: those versions of 'ls' that do this are
based on libc implementation that supports UTS#10, the Unicode
Collation Algorithm, or at least part of it. UTS#10 specifies a
multi-level comparison, whereby base characters, accents, and
letter-case variants are compared before punctuation characters.
> OTOH, if the UTF-8 codeset says the leading dot should be ignored,
> then ls-lisp should do the same by default, at least when the
> locale's codeset is UTF-8.
For this, we would need a UTS#10 compatible compare-strings. Then
ls-lisp could simply use it when the locale is .UTF-8.
Alternatively, we could have an approximation to that, just for
sorting non-punctuation characters before the punctuation characters.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 05:51:02 GMT)
Full text and
rfc822 format available.
Message #62 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> I don't think you understood what Michael wanted, but I'll let Michael
> speak for himself.
I don't use Tramp often currently.
Generally, I switched to ls-lisp because I liked that it gives me more
control over how dired looks like.
Sorting is one only thing (that could probably be done with Tramp).
There are other things. What I don't like from ls is that it shows
symlinks like this:
lrwxrwxrwx ... ...
rwxrwxrwx is redundant. When you use M on a link in dired, you actually
set the modes of the target. I want to see the target file's modes, so
I use this:
,----------------------------------------------------------------------
| (defun my-ls-lisp-treat-symlinks-ad (file-alist &rest _)
| "Make it show modes of truenames for symlinks."
| (mapc (lambda (file-line)
| (let ((filename (expand-file-name (car file-line)
| default-directory))
| modes-string)
| (when (file-symlink-p filename)
| (setq modes-string (nth 8 (file-attributes
| (file-truename filename))))
| (if (not modes-string) ;; link could be dead!
| (setq modes-string "l?????????")
| (aset modes-string 0 ?l))
| (setf (nth 9 file-line) modes-string))))
| file-alist)
| file-alist)
|
| (advice-add 'ls-lisp-handle-switches :after #'my-ls-lisp-treat-symlinks-ad)
`----------------------------------------------------------------------
AFAIK this can't be reached with ls. Just one example. Trying to do
such things with Tramp would probably indeed slow it down.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 06:09:02 GMT)
Full text and
rfc822 format available.
Message #65 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: Michael Albinus <michael.albinus <at> gmx.de>, 18051 <at> debbugs.gnu.org
> Date: Sun, 20 Jul 2014 07:49:48 +0200
>
> There are other things. What I don't like from ls is that it shows
> symlinks like this:
>
> lrwxrwxrwx ... ...
>
> rwxrwxrwx is redundant.
It's what 'lstat' returns.
> When you use M on a link in dired, you actually
> set the modes of the target. I want to see the target file's modes, so
> I use this:
>
> ,----------------------------------------------------------------------
> | (defun my-ls-lisp-treat-symlinks-ad (file-alist &rest _)
> | "Make it show modes of truenames for symlinks."
> | (mapc (lambda (file-line)
> | (let ((filename (expand-file-name (car file-line)
> | default-directory))
> | modes-string)
> | (when (file-symlink-p filename)
> | (setq modes-string (nth 8 (file-attributes
> | (file-truename filename))))
> | (if (not modes-string) ;; link could be dead!
> | (setq modes-string "l?????????")
> | (aset modes-string 0 ?l))
> | (setf (nth 9 file-line) modes-string))))
> | file-alist)
> | file-alist)
> |
> | (advice-add 'ls-lisp-handle-switches :after #'my-ls-lisp-treat-symlinks-ad)
> `----------------------------------------------------------------------
>
> AFAIK this can't be reached with ls.
Doesn't "ls -L" give you that?
> Just one example. Trying to do such things with Tramp would
> probably indeed slow it down.
IMO, the right way to do this is to have an additional argument to
file-attributes, follow-symlinks, with the obvious semantics.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 06:19:01 GMT)
Full text and
rfc822 format available.
Message #68 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Michael Heerdegen <michael_heerdegen <at> web.de> writes:
> | (advice-add 'ls-lisp-handle-switches
> :after #'my-ls-lisp-treat-symlinks-ad)
And I understand that users should not be encouraged to do such things.
Being 1:1 ls compatible has many advantages, but it makes dired
unusually inflexible compared to other Emacs packages. That's why I
said I wished ls-lisp - or some other configurable mechanism - would be
the default. I didn't think of Tramp when I said that.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 06:22:02 GMT)
Full text and
rfc822 format available.
Message #71 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> Doesn't "ls -L" give you that?
That's worse. Then I don't even see at all that the file is a symlink,
it is shown as a regular file. I had tried that, and found it very
confusing.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 06:35:01 GMT)
Full text and
rfc822 format available.
Message #74 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: michael.albinus <at> gmx.de, 18051 <at> debbugs.gnu.org
> Date: Sun, 20 Jul 2014 08:21:27 +0200
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> > Doesn't "ls -L" give you that?
>
> That's worse. Then I don't even see at all that the file is a symlink,
> it is shown as a regular file. I had tried that, and found it very
> confusing.
Why is it important to you to know that the file is a symlink, if you
always want to change its target?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 07:31:01 GMT)
Full text and
rfc822 format available.
Message #77 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> Why is it important to you to know that the file is a symlink, if you
> always want to change its target?
That's what I want for file modes - but still I want to know/see what
I'm doing!
And it makes a difference in the organization of the file system. You
can use symlinks as some kind of shortcut to move within the file system
more quickly. It makes a difference if you remove a shortcut or if you
erase a whole directory with all its files from your hard drive. It
also makes a difference when you think you make a local change, and
actually cause changes "somewhere else" in the file system.
The funniest thing you could do would be to delete the target because
you think you have two identical versions of a file/directory in your
filesystem, and then are shocked because the second version turns out to
be a dead link after that.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 08:15:02 GMT)
Full text and
rfc822 format available.
Message #80 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: michael.albinus <at> gmx.de, 18051 <at> debbugs.gnu.org
> Date: Sun, 20 Jul 2014 09:30:32 +0200
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> > Why is it important to you to know that the file is a symlink, if you
> > always want to change its target?
>
> That's what I want for file modes - but still I want to know/see what
> I'm doing!
But the same problem exists with the size, the time stamp, the
UID/GID, the inode -- you name it. I fail to see how seeing the mode
bits of the target is important, but the same is not true for the rest
of the attributes.
Or are you just saying that you want to see the attributes of the
target, and _also_ the fact that the file is a symlink?
> And it makes a difference in the organization of the file system. You
> can use symlinks as some kind of shortcut to move within the file system
> more quickly. It makes a difference if you remove a shortcut or if you
> erase a whole directory with all its files from your hard drive. It
> also makes a difference when you think you make a local change, and
> actually cause changes "somewhere else" in the file system.
This just means that we should have an easy way of switching between
the -L view and the non-L view. Would that solve the problem?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 08:26:02 GMT)
Full text and
rfc822 format available.
Message #83 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> Or are you just saying that you want to see the attributes of the
> target, and _also_ the fact that the file is a symlink?
Yes, exactly.
> This just means that we should have an easy way of switching between
> the -L view and the non-L view. Would that solve the problem?
That would not be bad, although personally I would prefer one view
providing all relevant information, even when that's not ls conform.
But even more important would be to an easy way to sort by size and
such.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 08:39:02 GMT)
Full text and
rfc822 format available.
Message #86 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: michael.albinus <at> gmx.de, 18051 <at> debbugs.gnu.org
> Date: Sun, 20 Jul 2014 10:24:46 +0200
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> > Or are you just saying that you want to see the attributes of the
> > target, and _also_ the fact that the file is a symlink?
>
> Yes, exactly.
>
> > This just means that we should have an easy way of switching between
> > the -L view and the non-L view. Would that solve the problem?
>
> That would not be bad, although personally I would prefer one view
> providing all relevant information, even when that's not ls conform.
Should be possible on the Dired level.
> But even more important would be to an easy way to sort by size and
> such.
That is already available (e.g., sorting by size is triggered by the
"-S" switch).
I think that one important conclusion from this discussion is that we
need to have a way to sort files names in a way that is at least
partially compliant with UTS#10, to produce listings that are similar
to what 'ls' does on GNU systems under UTF-8 locales. I hope someone
will step forward to write the necessary code.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 09:16:01 GMT)
Full text and
rfc822 format available.
Message #89 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> That is already available (e.g., sorting by size is triggered by the
> "-S" switch).
But very inconveniently: C-u s [edit minibuffer] RET.
It would be more convenient when s without prefix would cycle between
more then two sorting orders, and when this would be configurable.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 09:20:02 GMT)
Full text and
rfc822 format available.
Message #92 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: michael.albinus <at> gmx.de, 18051 <at> debbugs.gnu.org
> Date: Sun, 20 Jul 2014 11:15:38 +0200
>
> It would be more convenient when s without prefix would cycle between
> more then two sorting orders, and when this would be configurable.
Sounds like a good idea to me.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 11:45:02 GMT)
Full text and
rfc822 format available.
Message #95 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> I think that one important conclusion from this discussion is that we
> need to have a way to sort files names in a way that is at least
> partially compliant with UTS#10, to produce listings that are similar
> to what 'ls' does on GNU systems under UTF-8 locales. I hope someone
> will step forward to write the necessary code.
Programs like `ls' honor the LC_COLLATE environment variable. Emacs
shall do it as well. This shouldn't affect only directory listings, but
could be used also for string searches.
Maybe we should expose glib's g_utf8_collate() on Lisp level. On systems
without glib, we might emulate it partially. Packages like ls-lisp could
use it then for sorting.
I have no clear forecast on my time budget next weeks. If possible, I
would play with this.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 12:00:04 GMT)
Full text and
rfc822 format available.
Message #98 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, 18051 <at> debbugs.gnu.org
> Date: Sun, 20 Jul 2014 13:44:07 +0200
>
> Programs like `ls' honor the LC_COLLATE environment variable. Emacs
> shall do it as well.
That's clear, and is not the issue. The issue is why (and how) does
having UTF-8 in the codeset part of the locale cause sorting to sort
as 'ls' does on GNU platforms. And the answer is that the sorting
should implement UTS#10, which I'm not sure every platform does in its
standard C library.
> This shouldn't affect only directory listings, but could be used
> also for string searches.
That is a much larger job, and it's not clear how to do it best.
Emacs supports different languages in different buffers, and setting
and resetting LC_COLLATE for each buffer is not a good idea, IMO,
because thread-local locales are not well supported outside glibc
(AFAIK).
> Maybe we should expose glib's g_utf8_collate() on Lisp level.
Are you sure this does the job? Glib docs are minimal, and don't seem
to mention UTS#10. E.g., if g_utf8_collate relies on the underlying
libc's strcoll, we are back at square one.
> On systems without glib, we might emulate it partially. Packages
> like ls-lisp could use it then for sorting.
I think we need our own implementation in any case. If nothing else,
that would solve the issue of encoding strings into UTF-8 before
calling external C functions.
> I have no clear forecast on my time budget next weeks. If possible, I
> would play with this.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 14:23:01 GMT)
Full text and
rfc822 format available.
Message #101 received at 18051 <at> debbugs.gnu.org (full text, mbox):
>> | (advice-add 'ls-lisp-handle-switches
>> | :after #'my-ls-lisp-treat-symlinks-ad)
> And I understand that users should not be encouraged to do such things.
Actually, while they're not encouraged, I definitely don't want to
discourage users from using defadvice or advice-add.
It's strongly discouraged within Emacs, and mildly discouraged for
external packages, but it's not discouraged for end-users.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 15:27:01 GMT)
Full text and
rfc822 format available.
Message #104 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> Maybe we should expose glib's g_utf8_collate() on Lisp level.
>
> Are you sure this does the job? Glib docs are minimal, and don't seem
> to mention UTS#10. E.g., if g_utf8_collate relies on the underlying
> libc's strcoll, we are back at square one.
Well, I've checked the code of g_utf8_collate in glib 2.36. Shortly, it does
--8<---------------cut here---------------start------------->8---
#ifdef HAVE_CARBON
UCCompareTextDefault (kUCCollateStandardOptions,
str1_utf16, len1, str2_utf16, len2,
NULL, &retval);
#elif defined(__STDC_ISO_10646__)
result = wcscoll ((wchar_t *)str1_norm, (wchar_t *)str2_norm);
#else /* !__STDC_ISO_10646__ */
result = strcoll (str1_norm, str2_norm);
#endif
--8<---------------cut here---------------end--------------->8---
Likely, wcscoll implements only ISO 14651 (a subset of UCA these days),
and likely wcscoll supports single byte characters only. I will run some
tests next days.
An alternative would be libicu, which seems to implement UCA
completely. I have no idea whether there are licensing issues when
linking with Emacs, 'tho.
Maybe Paul knows better which library to use? I've seen in GNU grep's
Changelogs, that wcscoll was used, but removed last year. I haven't
checked (yet) what is the replacement.
>> On systems without glib, we might emulate it partially. Packages
>> like ls-lisp could use it then for sorting.
>
> I think we need our own implementation in any case. If nothing else,
> that would solve the issue of encoding strings into UTF-8 before
> calling external C functions.
Yep. But given the complexity of UCA, we will start slowly with a subset
of the algorithm only. This and performance considerations will still
demand for a native C library, if available.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 20 Jul 2014 16:18:02 GMT)
Full text and
rfc822 format available.
Message #107 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Sun, 20 Jul 2014 17:26:04 +0200
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> >> Maybe we should expose glib's g_utf8_collate() on Lisp level.
> >
> > Are you sure this does the job? Glib docs are minimal, and don't seem
> > to mention UTS#10. E.g., if g_utf8_collate relies on the underlying
> > libc's strcoll, we are back at square one.
>
> Well, I've checked the code of g_utf8_collate in glib 2.36. Shortly, it does
>
> --8<---------------cut here---------------start------------->8---
> #ifdef HAVE_CARBON
>
> UCCompareTextDefault (kUCCollateStandardOptions,
> str1_utf16, len1, str2_utf16, len2,
> NULL, &retval);
>
> #elif defined(__STDC_ISO_10646__)
>
> result = wcscoll ((wchar_t *)str1_norm, (wchar_t *)str2_norm);
>
> #else /* !__STDC_ISO_10646__ */
>
> result = strcoll (str1_norm, str2_norm);
>
> #endif
> --8<---------------cut here---------------end--------------->8---
As expected, it simply relies on the Standard C library's wcscoll
implementation.
> Likely, wcscoll implements only ISO 14651 (a subset of UCA these days),
> and likely wcscoll supports single byte characters only.
No, I expect wcscoll, at least in its glibc implementation, to support
the entire Unicode range.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 16 Aug 2014 21:53:02 GMT)
Full text and
rfc822 format available.
Message #110 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Michael Albinus <michael.albinus <at> gmx.de> writes:
>>> On systems without glib, we might emulate it partially. Packages
>>> like ls-lisp could use it then for sorting.
>>
>> I think we need our own implementation in any case. If nothing else,
>> that would solve the issue of encoding strings into UTF-8 before
>> calling external C functions.
>
> Yep. But given the complexity of UCA, we will start slowly with a subset
> of the algorithm only. This and performance considerations will still
> demand for a native C library, if available.
Just being curious, I've taken g_utf8_collate from the glib for a
test. It doesn't work bad.
I have added two functions `gstring-lessp' and `gstring-equalp', which
are meant to be the collation counterparts of `string-lessp' and
`string-equal'. Here are some tests, taken from UTS#10, chapter 1.1
"Multi-Level Comparison":
--8<---------------cut here---------------start------------->8---
(sort '("role" "roles" "rule") 'string-lessp)
=> ("role" "roles" "rule")
(sort '("role" "roles" "rule") 'gstring-lessp)
=> ("role" "roles" "rule")
--8<---------------cut here---------------end--------------->8---
No surprise they return the same result, this is level 1
comparison. Just base characters are compared.
--8<---------------cut here---------------start------------->8---
(sort '("role" "rôle" "roles") 'string-lessp)
=> ("role" "roles" "rôle")
(sort '("role" "rôle" "roles") 'gstring-lessp)
=> ("role" "rôle" "roles")
--8<---------------cut here---------------end--------------->8---
Accent differences are typically ignored in collation, if the base
letters differ. And so on, further tests applied from there ...
The collation rules could even be influenced by setting the locale
environment. The following example is taken from ISO 14651:2011,
appendix D.3. If LC_COLLATE is set to C.utf8, `string-lessp' and
`gstring-lessp' behave the same:
--8<---------------cut here---------------start------------->8---
(sort '("Alzheimer" "czar" "cæsium" "cølibat" "Aachen" "Aalborg" "Århus") 'stri\ng-lessp)
=> ("Aachen" "Aalborg" "Alzheimer" "czar" "cæsium" "cølibat" "Århus")
(sort '("Alzheimer" "czar" "cæsium" "cølibat" "Aachen" "Aalborg" "Århus") 'gstring-lessp)
=> ("Aachen" "Aalborg" "Alzheimer" "czar" "cæsium" "cølibat" "Århus")
--8<---------------cut here---------------end--------------->8---
When I set LC_COLLATE to en_US.utf8, accent differences are ignored,
again:
--8<---------------cut here---------------start------------->8---
(sort '("Alzheimer" "czar" "cæsium" "cølibat" "Aachen" "Aalborg" "Århus") 'gstring-lessp)
=> ("Aachen" "Aalborg" "Alzheimer" "Århus" "cæsium" "cølibat" "czar")
--8<---------------cut here---------------end--------------->8---
But setting LC_COLLATE to da_DK.utf8, the order differs, because "cz" is
less than "cæ", and "aa" is equivalent to "å" but greater than "z".
--8<---------------cut here---------------start------------->8---
(sort '("Alzheimer" "czar" "cæsium" "cølibat" "Aachen" "Aalborg" "Århus") 'gstring-lessp)
("Alzheimer" "czar" "cæsium" "cølibat" "Aachen" "Aalborg" "Århus")
--8<---------------cut here---------------end--------------->8---
Well, for practical use cases it seems to be worth to include
g_utf8_collate into Emacs. Of course, it could be used only in case glib
is linked, so we might still need an own Lisp implementation. I don't
know how well g_utf8_collate works for non Latin characters, 'tho.
And the test files CollationTest_NON_IGNORABLE.txt and
CollationTest_SHIFTED.txt from UTS#10 do not run completely
successful. I have no idea, whether it is due to a limitation of
g_utf8_collate, or whether it is because I have taken the latest Unicode
7.0.0 test files, which might include tests which haven't reached
GNU/Linux distributions yet. (Or whether my implementation is still
erroneous).
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 17 Aug 2014 16:39:02 GMT)
Full text and
rfc822 format available.
Message #113 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Sat, 16 Aug 2014 23:52:16 +0200
>
> Just being curious, I've taken g_utf8_collate from the glib for a
> test. It doesn't work bad.
Are you sure this is implemented in Glib, not in the underlying libc
(glibc in your case, I presume)?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 17 Aug 2014 17:56:02 GMT)
Full text and
rfc822 format available.
Message #116 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> Date: Sun, 17 Aug 2014 19:38:36 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
>
> > From: Michael Albinus <michael.albinus <at> gmx.de>
> > Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> > Date: Sat, 16 Aug 2014 23:52:16 +0200
> >
> > Just being curious, I've taken g_utf8_collate from the glib for a
> > test. It doesn't work bad.
>
> Are you sure this is implemented in Glib, not in the underlying libc
> (glibc in your case, I presume)?
Answering myself here: by reading the source of g_utf8_collate, it is
clear that the implementation is elsewhere. In particular, in any
environment that defines __STDC_ISO_10646__ (as does glibc),
g_utf8_collate simply calls wcscoll, after converting the UTF-8
strings to wide-character strings.
So I think a better alternative would be to base the implementation of
this feature on the system libraries directly. I think most modern
platforms have the necessary facilities.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 17 Aug 2014 18:47:03 GMT)
Full text and
rfc822 format available.
Message #119 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> Answering myself here: by reading the source of g_utf8_collate, it is
> clear that the implementation is elsewhere. In particular, in any
> environment that defines __STDC_ISO_10646__ (as does glibc),
> g_utf8_collate simply calls wcscoll, after converting the UTF-8
> strings to wide-character strings.
>
> So I think a better alternative would be to base the implementation of
> this feature on the system libraries directly. I think most modern
> platforms have the necessary facilities.
But OTOH, g_utf8_collate handles also other cases, like the #ifdef
HAVE_CARBON case.
So what, maybe it is sufficient to take over the implementation from
glib, indeed. There's not too much logic added there, and we would avoid
the glib dependency.
What I would really like to test are non-Latin coding points. I'm a noob
for such characters (glad to speak German, English and a little bit
Français); do you or somebody else has some test cases for
`gstring-lessp' or `gstring-equalp', which shall return different
results than `string-lessp' and `string-equal'?
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 17 Aug 2014 18:53:02 GMT)
Full text and
rfc822 format available.
Message #122 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Sun, 17 Aug 2014 20:46:11 +0200
>
> So what, maybe it is sufficient to take over the implementation from
> glib, indeed. There's not too much logic added there, and we would avoid
> the glib dependency.
That's what I had in mind, yes.
> do you or somebody else has some test cases for `gstring-lessp' or
> `gstring-equalp', which shall return different results than
> `string-lessp' and `string-equal'?
I don't have such test cases, but I'd start by looking in the glib
test suite.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Thu, 21 Aug 2014 09:07:02 GMT)
Full text and
rfc822 format available.
Message #125 received at 18051 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:
>> So what, maybe it is sufficient to take over the implementation from
>> glib, indeed. There's not too much logic added there, and we would avoid
>> the glib dependency.
>
> That's what I had in mind, yes.
Finally, I came out with the appended patch. Comments appreciated.
>> do you or somebody else has some test cases for `gstring-lessp' or
>> `gstring-equalp', which shall return different results than
>> `string-lessp' and `string-equal'?
>
> I don't have such test cases, but I'd start by looking in the glib
> test suite.
There aren't so many glib test cases for collation. I need to search further.
Best regards, Michael.
[collate-patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Thu, 21 Aug 2014 14:42:01 GMT)
Full text and
rfc822 format available.
Message #128 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Thu, 21 Aug 2014 11:05:43 +0200
>
> >> So what, maybe it is sufficient to take over the implementation from
> >> glib, indeed. There's not too much logic added there, and we would avoid
> >> the glib dependency.
> >
> > That's what I had in mind, yes.
>
> Finally, I came out with the appended patch. Comments appreciated.
Thanks.
I have 2 comments:
. I suggest to factor out the part that converts to wchar_t, sets up
the locale, and calls strcoll. The code you wrote makes certain
assumptions about 'setlocale', and also about the wchar_t
representation. Factoring those system-dependent parts out will
minimize the number of #ifdef's needed to provide such features for
other platforms.
. I think glibc has a 'newlocale' API that is better suited to this
kind of jobs. In particular, 'setlocale' changes the locale of the
entire program, which is bad news for other threads that might be
using some locale-aware functions while the main thread calls
string-collate-lessp. (We have more than 1 thread in Emacs built
with GTK, for example, and who knows what those threads might be
doing?)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 22 Aug 2014 14:24:02 GMT)
Full text and
rfc822 format available.
Message #131 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
Hi Eli,
> . I suggest to factor out the part that converts to wchar_t, sets up
> the locale, and calls strcoll. The code you wrote makes certain
> assumptions about 'setlocale', and also about the wchar_t
> representation. Factoring those system-dependent parts out will
> minimize the number of #ifdef's needed to provide such features for
> other platforms.
I see. But I don't know how to factor out. Shall I move str_collate to
another file? Or to a new file? Something else?
> . I think glibc has a 'newlocale' API that is better suited to this
> kind of jobs. In particular, 'setlocale' changes the locale of the
> entire program, which is bad news for other threads that might be
> using some locale-aware functions while the main thread calls
> string-collate-lessp. (We have more than 1 thread in Emacs built
> with GTK, for example, and who knows what those threads might be
> doing?)
OK, done.
I have added also configure checks HAVE_NEWLOCALE, HAVE_USELOCALE and
HAVE_FREELOCALE for the respective glibc functions. I don't know whether
it is overengineering, and whether I could simply apply the existing
HAVE_SETLOCALE check. I believe all these functions do exist in parallel
in locale.h, don't they?
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 23 Aug 2014 09:06:02 GMT)
Full text and
rfc822 format available.
Message #134 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Fri, 22 Aug 2014 16:23:34 +0200
>
> > . I suggest to factor out the part that converts to wchar_t, sets up
> > the locale, and calls strcoll. The code you wrote makes certain
> > assumptions about 'setlocale', and also about the wchar_t
> > representation. Factoring those system-dependent parts out will
> > minimize the number of #ifdef's needed to provide such features for
> > other platforms.
>
> I see. But I don't know how to factor out. Shall I move str_collate to
> another file? Or to a new file? Something else?
I think everything in str_collate starting with the "Convert byte
stream to code pointers." comment (btw, I guess you meant "code
points" here) should be in a separate function, and the best place for
that function is sysdep.c. At least on MS-Windows, both the part that
converts a Lisp string into wchar_t array, and the part that performs
a locale-sensitive string comparison, will be implemented differently.
> > . I think glibc has a 'newlocale' API that is better suited to this
> > kind of jobs. In particular, 'setlocale' changes the locale of the
> > entire program, which is bad news for other threads that might be
> > using some locale-aware functions while the main thread calls
> > string-collate-lessp. (We have more than 1 thread in Emacs built
> > with GTK, for example, and who knows what those threads might be
> > doing?)
>
> OK, done.
Thanks. (You didn't attach the new patch.)
Btw, I wonder whether we should have a way to pass the locale string
explicitly, instead of relying on $LC_COLLATE.
> I have added also configure checks HAVE_NEWLOCALE, HAVE_USELOCALE and
> HAVE_FREELOCALE for the respective glibc functions. I don't know whether
> it is overengineering, and whether I could simply apply the existing
> HAVE_SETLOCALE check. I believe all these functions do exist in parallel
> in locale.h, don't they?
I'll defer to glibc experts on that. My knowledge of 'newlocale'
facilities is limited to what I saw in Guile's i18n.c module.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 23 Aug 2014 16:44:02 GMT)
Full text and
rfc822 format available.
Message #137 received at 18051 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:
> I think everything in str_collate starting with the "Convert byte
> stream to code pointers." comment (btw, I guess you meant "code
> points" here) should be in a separate function, and the best place for
> that function is sysdep.c. At least on MS-Windows, both the part that
> converts a Lisp string into wchar_t array, and the part that performs
> a locale-sensitive string comparison, will be implemented differently.
Well, I've moved (most of) str_collate to sysdep.c.
> Thanks. (You didn't attach the new patch.)
Oops. Appended this time.
> Btw, I wonder whether we should have a way to pass the locale string
> explicitly, instead of relying on $LC_COLLATE.
We could add an optional argument to string-collate-*. But this would
break signature equivalence with string-lessp and string-equal,
respectively.
Or we could introduce a global var, which shall be let-bound to the
locale string.
>> I have added also configure checks HAVE_NEWLOCALE, HAVE_USELOCALE and
>> HAVE_FREELOCALE for the respective glibc functions. I don't know whether
>> it is overengineering, and whether I could simply apply the existing
>> HAVE_SETLOCALE check. I believe all these functions do exist in parallel
>> in locale.h, don't they?
>
> I'll defer to glibc experts on that. My knowledge of 'newlocale'
> facilities is limited to what I saw in Guile's i18n.c module.
According to the manpages, setlocale is conforming to "C89, C99,
POSIX.1-2001". {new,use,free}locale are conforming to "POSIX.1-2008".
So we must check for HAVE_USELOCALE, indeed. Checks for HAVE_NEWLOCALE
and HAVE_FREELOCALE are not necessary, the functions exist in parallel
to uselocale (introduced in glibc 2.3).
This raises the question, whether we shall use also my first setlocale
approach in case of uselocale absence?
Best regards, Michael.
[collate-patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 23 Aug 2014 17:34:02 GMT)
Full text and
rfc822 format available.
Message #140 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Sat, 23 Aug 2014 18:42:44 +0200
>
> > Btw, I wonder whether we should have a way to pass the locale string
> > explicitly, instead of relying on $LC_COLLATE.
>
> We could add an optional argument to string-collate-*. But this would
> break signature equivalence with string-lessp and string-equal,
> respectively.
>
> Or we could introduce a global var, which shall be let-bound to the
> locale string.
Or have a new optional argument in string-lessp etc., or introduce a
new set of APIs which will accept a locale, and have string-lessp
etc. call them with that argument nil.
> This raises the question, whether we shall use also my first setlocale
> approach in case of uselocale absence?
I think so, yes.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 23 Aug 2014 20:33:02 GMT)
Full text and
rfc822 format available.
Message #143 received at 18051 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:
>> > Btw, I wonder whether we should have a way to pass the locale string
>> > explicitly, instead of relying on $LC_COLLATE.
>>
>> We could add an optional argument to string-collate-*. But this would
>> break signature equivalence with string-lessp and string-equal,
>> respectively.
>>
>> Or we could introduce a global var, which shall be let-bound to the
>> locale string.
>
> Or have a new optional argument in string-lessp etc., or introduce a
> new set of APIs which will accept a locale, and have string-lessp
> etc. call them with that argument nil.
An optional argument to string-lessp could be inconvenient. IMHO, the
most important use-case of string-lessp is being a PREDICATE of
sort. This does not support optional arguments.
>> This raises the question, whether we shall use also my first setlocale
>> approach in case of uselocale absence?
>
> I think so, yes.
Extended patch appended.
Best regards, Michael.
[collate-patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 24 Aug 2014 14:55:01 GMT)
Full text and
rfc822 format available.
Message #146 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Sat, 23 Aug 2014 22:32:00 +0200
>
> >> > Btw, I wonder whether we should have a way to pass the locale string
> >> > explicitly, instead of relying on $LC_COLLATE.
> >>
> >> We could add an optional argument to string-collate-*. But this would
> >> break signature equivalence with string-lessp and string-equal,
> >> respectively.
> >>
> >> Or we could introduce a global var, which shall be let-bound to the
> >> locale string.
> >
> > Or have a new optional argument in string-lessp etc., or introduce a
> > new set of APIs which will accept a locale, and have string-lessp
> > etc. call them with that argument nil.
>
> An optional argument to string-lessp could be inconvenient. IMHO, the
> most important use-case of string-lessp is being a PREDICATE of
> sort. This does not support optional arguments.
In those cases, we should add the same optional argument to the sort
function.
> Extended patch appended.
Thanks.
I wonder what should this do if the new locale cannot be
instantiated/installed. As you wrote the code, it will silently use
the current locale, but I wonder if that's TRT.
Other than that, I think you should install this.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sun, 24 Aug 2014 16:19:02 GMT)
Full text and
rfc822 format available.
Message #149 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> > Or have a new optional argument in string-lessp etc., or introduce a
>> > new set of APIs which will accept a locale, and have string-lessp
>> > etc. call them with that argument nil.
>>
>> An optional argument to string-lessp could be inconvenient. IMHO, the
>> most important use-case of string-lessp is being a PREDICATE of
>> sort. This does not support optional arguments.
>
> In those cases, we should add the same optional argument to the sort
> function.
That would be an option. But it would take decades, until all
appearences of sort have been adapted. That's why I'm in favor of a
global variable, which could be let-bounded.
But I don't oppose strongly to your proposal.
> I wonder what should this do if the new locale cannot be
> instantiated/installed. As you wrote the code, it will silently use
> the current locale, but I wonder if that's TRT.
We could raise a warning. But if it is used in sort with a huge list of
strings, the user would be flooded with those warnings.
Maybe we could add a function, which would tell whether a given locale
value would have effect. The user could check then.
> Other than that, I think you should install this.
Done. Next days I'm on the road; the work still missing (extending Elisp
manual, writing ert tests, adapting ls-lisp) must be postponed to later
this week, or the week after.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 25 Aug 2014 05:49:02 GMT)
Full text and
rfc822 format available.
Message #152 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Dmitry Antipov wrote:
> ../../trunk/src/sysdep.c:3527:1: error: no previous prototype for
> ‘str_collate’ [-Werror=missing-prototypes]
I fixed that problem, along with some other minor glitches associated
with the patch, in trunk bzr 117733.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 25 Aug 2014 06:20:02 GMT)
Full text and
rfc822 format available.
Message #155 received at 18051 <at> debbugs.gnu.org (full text, mbox):
On 08/25/2014 09:48 AM, Paul Eggert wrote:
> I fixed that problem, along with some other minor glitches
> associated with the patch, in trunk bzr 117733.
Thanks.
BTW, I think that collation functions with 3rd optional argument
to specify locale settings will be a bit more versatile, e.g.
(string-collate-lessp a b "es_ES.UTF-8")
Dmitry
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 25 Aug 2014 06:42:02 GMT)
Full text and
rfc822 format available.
Message #158 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Dmitry Antipov <dmantipov <at> yandex.ru> writes:
> BTW, I think that collation functions with 3rd optional argument
> to specify locale settings will be a bit more versatile, e.g.
>
> (string-collate-lessp a b "es_ES.UTF-8")
We discuss this already, see <http://lists.gnu.org/archive/html/bug-gnu-emacs/2014-08/msg00623.html>
My major reservation to this approach is that it doesn't fit well using
string-collate-lessp as predicate of sort. That's why I have proposed a
global variable as alternative, which could be let-bounded.
> Dmitry
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 25 Aug 2014 15:02:02 GMT)
Full text and
rfc822 format available.
Message #161 received at 18051 <at> debbugs.gnu.org (full text, mbox):
>> An optional argument to string-lessp could be inconvenient. IMHO, the
>> most important use-case of string-lessp is being a PREDICATE of
>> sort. This does not support optional arguments.
Of course it does:
(sort foo (lambda (x y) (string-lessp x y 'optional-arg)))
-- Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 25 Aug 2014 15:04:01 GMT)
Full text and
rfc822 format available.
Message #164 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Date: Mon, 25 Aug 2014 08:41:03 +0200
> Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 18051 <at> debbugs.gnu.org
>
> > BTW, I think that collation functions with 3rd optional argument
> > to specify locale settings will be a bit more versatile, e.g.
> >
> > (string-collate-lessp a b "es_ES.UTF-8")
>
> We discuss this already, see
> <http://lists.gnu.org/archive/html/bug-gnu-emacs/2014-08/msg00623.html>
>
> My major reservation to this approach is that it doesn't fit well using
> string-collate-lessp as predicate of sort. That's why I have proposed a
> global variable as alternative, which could be let-bounded.
I think that binding a variable will indeed be cleaner. Using
process-environment for that purpose should be reserved for the
application level. Also, what if LC_COLLATE is not set in the
environment, but 'setlocale' does return some value for it? shouldn't
we use that?
Here are a few more thoughts about related issues:
1. Why does str_collate return a ptrdiff_t value? AFAIK, wcscoll
etc. return int data type, and of rather small values.
2. Should we signal an error if the input strings are not pure-ASCII
or multibyte? Unibyte strings will at best cause incorrect
results. And what about strings with invalid codepoints,
e.g. those outside of the Unicode range, which can happen inside
Lisp strings?
3. What about errors in wcscoll? The current code ignores them;
however, the value returned by wcscoll in case of an error is not
documented, so it could be random. Should we signal an error if
errno gets set by wcscoll?
4. How to control the optional features of the collating sequence? I
mean, for example, the fact that punctuation characters are ignored
in the .UTF-8 locales on glibc hosts (or so it seems). At least on
Windows, a somewhat higher degree of control is available, but it
must be specified separately of the locale ID. E.g., the
comparison function accepts flags to ignore punctuation and
symbols, width differences, diacritics, etc. Should we have another
variable, perhaps w32-specific, to request these features?
Alternatively, we could use .UTF-8 on Windows to communicate that,
although that sounds like a kludge.
5. The locale names on Windows are different from Posix: Windows uses
3-letter abbreviations of the country and the language,
e.g. "fra_FRA" instead of the Posix "fr_FR". Do we want the locale
string values used for let-binding the above-mentioned variable to
be portable across systems? Then we'd need some conversion
database on MS-Windows.
6. I think we will want case-insensitive version of this function.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 25 Aug 2014 16:02:03 GMT)
Full text and
rfc822 format available.
Message #167 received at 18051 <at> debbugs.gnu.org (full text, mbox):
This is now implemented for MS-Windows as well.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 25 Aug 2014 16:46:02 GMT)
Full text and
rfc822 format available.
Message #170 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Michael Albinus wrote:
> An optional argument to string-lessp could be inconvenient. IMHO, the
> most important use-case of string-lessp is being a PREDICATE of
> sort. This does not support optional arguments.
So make the argument optional, but defaulting to what the locale says.
BTW, this is http://debbugs.gnu.org/2263 (and 12008).
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 25 Aug 2014 17:37:01 GMT)
Full text and
rfc822 format available.
Message #173 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Glenn Morris <rgm <at> gnu.org>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Mon, 25 Aug 2014 12:45:07 -0400
>
> BTW, this is http://debbugs.gnu.org/2263 (and 12008).
They should obviously be closed now, both of them.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 08:50:02 GMT)
Full text and
rfc822 format available.
Message #176 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier <monnier <at> IRO.UMontreal.CA> writes:
>>> An optional argument to string-lessp could be inconvenient. IMHO, the
>>> most important use-case of string-lessp is being a PREDICATE of
>>> sort. This does not support optional arguments.
>
> Of course it does:
>
> (sort foo (lambda (x y) (string-lessp x y 'optional-arg)))
Yes, but this would also expect optional-arg to be a variable which can
be set by the user. That's similar to what I have proposed, I believe.
> -- Stefan
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 11:26:02 GMT)
Full text and
rfc822 format available.
Message #179 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> Here are a few more thoughts about related issues:
>
> 1. Why does str_collate return a ptrdiff_t value? AFAIK, wcscoll
> etc. return int data type, and of rather small values.
Hm, yes. Both wcscoll and w32_compare_strings return int, so I've
changed that for str_collate accordingly.
> 2. Should we signal an error if the input strings are not pure-ASCII
> or multibyte? Unibyte strings will at best cause incorrect
> results.
Maybe we shall convert the strings to multibyte, via string_to_multibyte()?
If the string is already multibyte, it doesn't harm.
> And what about strings with invalid codepoints,
> e.g. those outside of the Unicode range, which can happen inside
> Lisp strings?
> 3. What about errors in wcscoll? The current code ignores them;
> however, the value returned by wcscoll in case of an error is not
> documented, so it could be random. Should we signal an error if
> errno gets set by wcscoll?
wcscoll sets EINVAL when the codepoint is out of range. I've added a
check for this case, returning an error.
(string-collate-equalp (string 1) (string ?\U0020FFFF))
=> error: Non-Unicode character: 0x20ffff
> 4. How to control the optional features of the collating sequence? I
> mean, for example, the fact that punctuation characters are ignored
> in the .UTF-8 locales on glibc hosts (or so it seems). At least on
> Windows, a somewhat higher degree of control is available, but it
> must be specified separately of the locale ID. E.g., the
> comparison function accepts flags to ignore punctuation and
> symbols, width differences, diacritics, etc. Should we have another
> variable, perhaps w32-specific, to request these features?
> Alternatively, we could use .UTF-8 on Windows to communicate that,
> although that sounds like a kludge.
In Posix systems, I'm not aware of configuring such optional features
via glibc. The most granular selection is what you dou with LC_COLLATE.
If we want to offer more granular settings, we would need to use a library
like libicu (http://icu-project.org/). Could be done, but should be optional.
> 5. The locale names on Windows are different from Posix: Windows uses
> 3-letter abbreviations of the country and the language,
> e.g. "fra_FRA" instead of the Posix "fr_FR". Do we want the locale
> string values used for let-binding the above-mentioned variable to
> be portable across systems? Then we'd need some conversion
> database on MS-Windows.
Here I'm a bit undecided. We could let it to the users to find the
proper locale name, but this is inconvenient. OTOH it would be much work
to install a mapping system, and we would need to maintain it. What if
there would be a new "en_SC" (Scotland) locale? We would need to
maintain such changes in Emacs forever ...
> 6. I think we will want case-insensitive version of this function.
That's also on my todo list. But I'm a little bit undecided whether we
shall add it to string-collate-* functions, or whether there shall be
further functions.
Maybe we could use sort-fold-case for this as indication? Or is this too
specific?
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 15:37:02 GMT)
Full text and
rfc822 format available.
Message #182 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
> Date: Wed, 27 Aug 2014 10:49:05 +0200
>
> Stefan Monnier <monnier <at> IRO.UMontreal.CA> writes:
>
> >>> An optional argument to string-lessp could be inconvenient. IMHO, the
> >>> most important use-case of string-lessp is being a PREDICATE of
> >>> sort. This does not support optional arguments.
> >
> > Of course it does:
> >
> > (sort foo (lambda (x y) (string-lessp x y 'optional-arg)))
>
> Yes, but this would also expect optional-arg to be a variable which can
> be set by the user. That's similar to what I have proposed, I believe.
True. So I suggest to define a new variable, say,
string-collate-options, which is a key-value list with up to 2
members: ':locale' (a string), and ':case-fold' (a flag). If the
locale's codeset is UTF-8, the collation on Windows will emulate what
glibc evidently does. Lisp programs will bind string-collate-options
to the value they need.
Then we can remove the reference to process-environment from the code
of string_collate.
WDYT?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 15:41:01 GMT)
Full text and
rfc822 format available.
Message #185 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: dmantipov <at> yandex.ru, eggert <at> cs.ucla.edu, 18051 <at> debbugs.gnu.org
> Date: Wed, 27 Aug 2014 13:24:48 +0200
>
> > 2. Should we signal an error if the input strings are not pure-ASCII
> > or multibyte? Unibyte strings will at best cause incorrect
> > results.
>
> Maybe we shall convert the strings to multibyte, via string_to_multibyte()?
That will not help.
I say code that invokes these functions with unibyte non-ASCII strings
has a bug that should be flagged.
> > 5. The locale names on Windows are different from Posix: Windows uses
> > 3-letter abbreviations of the country and the language,
> > e.g. "fra_FRA" instead of the Posix "fr_FR". Do we want the locale
> > string values used for let-binding the above-mentioned variable to
> > be portable across systems? Then we'd need some conversion
> > database on MS-Windows.
>
> Here I'm a bit undecided. We could let it to the users to find the
> proper locale name, but this is inconvenient. OTOH it would be much work
> to install a mapping system, and we would need to maintain it. What if
> there would be a new "en_SC" (Scotland) locale? We would need to
> maintain such changes in Emacs forever ...
I think these interfaces will almost always be used with the current
locale. So with that in mind, I think we can document this issue, and
then safely leave this problem to the code that needs to use
non-default locales.
> > 6. I think we will want case-insensitive version of this function.
>
> That's also on my todo list. But I'm a little bit undecided whether we
> shall add it to string-collate-* functions, or whether there shall be
> further functions.
>
> Maybe we could use sort-fold-case for this as indication? Or is this too
> specific?
See my suggestion in the other message.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 15:49:01 GMT)
Full text and
rfc822 format available.
Message #188 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Michael Albinus wrote:
>> (sort foo (lambda (x y) (string-lessp x y 'optional-arg)))
>
> Yes, but this would also expect optional-arg to be a variable which can
> be set by the user.
I'm missing something, because I don't get why you want me to write (in
authors.el):
(let ((process-environment
(cons "LC_COLLATE=en_US.UTF-8"
process-environment)))
(sort authors-author-list
(lambda (a b) (string-collate-lessp (car a) (car b)))))
rather than the obviously-better:
(sort authors-author-list
(lambda (a b) (string-collate-lessp (car a) (car b) "en_US.UTF-8")))
Normally one controls functions through their arguments, not the
environment.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 16:54:02 GMT)
Full text and
rfc822 format available.
Message #191 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Glenn Morris <rgm <at> gnu.org>
> Date: Wed, 27 Aug 2014 11:48:36 -0400
> Cc: michael_heerdegen <at> web.de, 18051 <at> debbugs.gnu.org
>
> Normally one controls functions through their arguments, not the
> environment.
We also have the other variety, e.g. see coding-system-for-read.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 18:03:02 GMT)
Full text and
rfc822 format available.
Message #194 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> True. So I suggest to define a new variable, say,
> string-collate-options, which is a key-value list with up to 2
> members: ':locale' (a string), and ':case-fold' (a flag). If the
> locale's codeset is UTF-8, the collation on Windows will emulate what
> glibc evidently does. Lisp programs will bind string-collate-options
> to the value they need.
>
> Then we can remove the reference to process-environment from the code
> of string_collate.
>
> WDYT?
Sounds OK to me.
Nest regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 18:09:02 GMT)
Full text and
rfc822 format available.
Message #197 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Glenn Morris <rgm <at> gnu.org> writes:
> I'm missing something, because I don't get why you want me to write (in
> authors.el):
>
> (let ((process-environment
> (cons "LC_COLLATE=en_US.UTF-8"
> process-environment)))
> (sort authors-author-list
> (lambda (a b) (string-collate-lessp (car a) (car b)))))
>
> rather than the obviously-better:
>
> (sort authors-author-list
> (lambda (a b) (string-collate-lessp (car a) (car b) "en_US.UTF-8")))
>
> Normally one controls functions through their arguments, not the
> environment.
authors.el is a special case:
- Your sort predicate is not an existing function, but a
lambda. Usually, I would expect something like
(sort any-list 'string-collate-lessp)
- You use a hard-coded value for the locale. The intention is to make it
configurable for the user.
If, for example, a user wants to use another collation order but the one
given by "en_US.UTF-8", you end up in offering a variable which can be
set. Don't know whether this is desirable in authors.el, 'tho.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 18:13:01 GMT)
Full text and
rfc822 format available.
Message #200 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> > 2. Should we signal an error if the input strings are not pure-ASCII
>> > or multibyte? Unibyte strings will at best cause incorrect
>> > results.
>>
>> Maybe we shall convert the strings to multibyte, via string_to_multibyte()?
>
> That will not help.
>
> I say code that invokes these functions with unibyte non-ASCII strings
> has a bug that should be flagged.
Well, you have much more experience with unicode than I have.
>> > 5. The locale names on Windows are different from Posix: Windows uses
>> > 3-letter abbreviations of the country and the language,
>> > e.g. "fra_FRA" instead of the Posix "fr_FR". Do we want the locale
>> > string values used for let-binding the above-mentioned variable to
>> > be portable across systems? Then we'd need some conversion
>> > database on MS-Windows.
>>
>> Here I'm a bit undecided. We could let it to the users to find the
>> proper locale name, but this is inconvenient. OTOH it would be much work
>> to install a mapping system, and we would need to maintain it. What if
>> there would be a new "en_SC" (Scotland) locale? We would need to
>> maintain such changes in Emacs forever ...
>
> I think these interfaces will almost always be used with the current
> locale. So with that in mind, I think we can document this issue, and
> then safely leave this problem to the code that needs to use
> non-default locales.
I don't get this. What do you propose here? Set the locale specific to
the system Emacs is running, or do you propose a mapping to something
which is portable over system boundaries?
> Thanks.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 18:27:01 GMT)
Full text and
rfc822 format available.
Message #203 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: dmantipov <at> yandex.ru, eggert <at> cs.ucla.edu, 18051 <at> debbugs.gnu.org
> Date: Wed, 27 Aug 2014 20:12:12 +0200
>
> >> > 5. The locale names on Windows are different from Posix: Windows uses
> >> > 3-letter abbreviations of the country and the language,
> >> > e.g. "fra_FRA" instead of the Posix "fr_FR". Do we want the locale
> >> > string values used for let-binding the above-mentioned variable to
> >> > be portable across systems? Then we'd need some conversion
> >> > database on MS-Windows.
> >>
> >> Here I'm a bit undecided. We could let it to the users to find the
> >> proper locale name, but this is inconvenient. OTOH it would be much work
> >> to install a mapping system, and we would need to maintain it. What if
> >> there would be a new "en_SC" (Scotland) locale? We would need to
> >> maintain such changes in Emacs forever ...
> >
> > I think these interfaces will almost always be used with the current
> > locale. So with that in mind, I think we can document this issue, and
> > then safely leave this problem to the code that needs to use
> > non-default locales.
>
> I don't get this. What do you propose here? Set the locale specific to
> the system Emacs is running, or do you propose a mapping to something
> which is portable over system boundaries?
The former. IOW, the (rare, IMO) Lisp program that wants to override
the default locale will have to figure out how to do that in a way
that works on all the supported platforms. E.g., one way is
(let ((locale (if (eq system-type 'windows-nt)
"enu_USA"
"en_US")))
...
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 18:31:02 GMT)
Full text and
rfc822 format available.
Message #206 received at 18051 <at> debbugs.gnu.org (full text, mbox):
You don't know how this feature will be used, because you just added it.
Sometimes people will want to use it to "sort in the user's specified
locale", sometimes they will want to use it to "sort according to some
specific locale". An optional argument defaulting to LC_COLLATE makes
both uses easy. (Adding a global lisp variable that overrides LC_COLLATE
seems pointless but harmless to me.)
Anyway, at this point I give up trying to get an optional argument added
to a function.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 19:01:02 GMT)
Full text and
rfc822 format available.
Message #209 received at 18051 <at> debbugs.gnu.org (full text, mbox):
I found the following issues and installed what I hope are fixes as
trunk bzr 117751.
First, the code should use wcscoll_t rather than uselocale, as uselocale
modifies thread state and this is less robust; for example, it wasn't
safe to call 'error' right after the first call to uselocale.
Second, if the locale is invalid, string-collate-lessp should throw an
error, the same way it throws an error when the strings are invalid.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 19:10:01 GMT)
Full text and
rfc822 format available.
Message #212 received at 18051 <at> debbugs.gnu.org (full text, mbox):
A couple more things.
First, the current algorithm looks only at LC_COLLATE, but the usual
approach is to default LC_COLLATE to LANG if LC_COLLATE isn't set, and
to have LC_ALL override LC_COLLATE. Shouldn't Emacs take a similar
approach, for compatibility?
More generally, it strikes me that string-collate-lessp will be quite
slow due to the overhead of looking up the locale environment string and
creating and destroying a locale for each string comparison. Instead,
shouldn't Emacs should have a locale object that the Emacs Lisp
programmer can create, an object that encapsulates the low level
locale_t object, and which can be passed as an optional argument to
string-collate-lessp? That way, string-collate-p would never have to
inspect the environment itself, or to create or destroy a locale.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 19:55:02 GMT)
Full text and
rfc822 format available.
Message #215 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> Date: Wed, 27 Aug 2014 12:08:52 -0700
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> CC: dmantipov <at> yandex.ru, 18051 <at> debbugs.gnu.org
>
> First, the current algorithm looks only at LC_COLLATE, but the usual
> approach is to default LC_COLLATE to LANG if LC_COLLATE isn't set, and
> to have LC_ALL override LC_COLLATE. Shouldn't Emacs take a similar
> approach, for compatibility?
I think we agreed to have a variable that holds the non-default locale
as a Lisp string. LANG and LC_COLLATE will then be used internally by
newlocale and/or wcscoll_t, as users expect. I don't think it's
appropriate for a primitive to take arguments from environment
variables, certainly not those on process-environment. If some Lisp
application would want to do that, let them.
> More generally, it strikes me that string-collate-lessp will be quite
> slow due to the overhead of looking up the locale environment string and
> creating and destroying a locale for each string comparison.
The lookup will no longer be relevant, when we switch to a variable.
As for creating and destroying the locale, I guess you are right.
> Instead, shouldn't Emacs should have a locale object that the Emacs
> Lisp programmer can create, an object that encapsulates the low
> level locale_t object, and which can be passed as an optional
> argument to string-collate-lessp?
That's what Guile does. But it will complicate using these functions
in sorting routines. Perhaps binding a variable to the object will
do.
Alternatively, a simple one-slot cache internal to string_collate will
probably remove most of the overhead. (You will see that
w32_compare_strings already employs a similar cache.)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 21:29:02 GMT)
Full text and
rfc822 format available.
Message #218 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii wrote:
> I think we agreed to have a variable that holds the non-default locale
> as a Lisp string.
Ah, sorry, missed that (it is a long thread...). Makes sense. I assume
this is on someone's TODO list since it's not done that way now.
> Perhaps binding a variable to the object will do.
We could do both: i.e., give the comparison function an optional
argument that defaults to the value of the bound variable. I'd think
the value should be a locale object, though, not a string like "en_US".
And perhaps the object should also record whether the comparison is
case-sensitive, and other stuff like that.
> Alternatively, a simple one-slot cache internal to string_collate will
> probably remove most of the overhead.
It would now, but it would also add another obstacle to adding
multithreading capabilities, as the locking around the cache would
inhibit scalability. So I'd rather avoid such a cache if it's easy.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 21:39:03 GMT)
Full text and
rfc822 format available.
Message #221 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Paul Eggert <eggert <at> cs.ucla.edu> writes:
> Ah, sorry, missed that (it is a long thread...). Makes sense. I
> assume this is on someone's TODO list since it's not done that way
> now.
Eli, that means you or me :-)
I do not want to interfere your work, but in case you are busy with
other tasks, I could do. Pls let me know.
>> Perhaps binding a variable to the object will do.
>
> We could do both: i.e., give the comparison function an optional
> argument that defaults to the value of the bound variable. I'd think
> the value should be a locale object, though, not a string like
> "en_US". And perhaps the object should also record whether the
> comparison is case-sensitive, and other stuff like that.
Good idea, that would also make Glenn happy. (That's not a joke, I mean
it seriously!)
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Wed, 27 Aug 2014 23:58:01 GMT)
Full text and
rfc822 format available.
Message #224 received at 18051 <at> debbugs.gnu.org (full text, mbox):
On Wed, 27 Aug 2014 18:56:52 +0000, Paul Eggert wrote:
> ------------------------------------------------------------
> revno: 117751
[...]
> message:
> Improve robustness of new string-collation code.
> * configure.ac (newlocale): Check for this, not for uselocale.
> * src/sysdep.c (LC_COLLATE, LC_COLLATE_MASK, freelocale, locale_t)
> (newlocale, wcscoll_l): Define substitutes for platforms that
> lack them, so as to simplify the mainline code.
> (str_collate): Simplify the code by assuming the above definitions.
> Use wcscoll_l, not uselocale, as uselocale is too fragile. For
> example, the old version left the Emacs in the wrong locale if
> wcscoll reported an error. Use 'int', not ptrdiff_t, for the int
> result. Report an error if newlocale fails.
Emacs build gets stopped on Cygwin probably due to this change.
gcc [...options below...] sysdep.c
sysdep.c: In function 'str_collate':
sysdep.c:3706:33: error: 'LC_COLLATE_MASK' undeclared (first use in this function)
locale_t loc = newlocale (LC_COLLATE_MASK, SSDATA (lc_collate), 0);
^
sysdep.c:3706:33: note: each undeclared identifier is reported only once for each function it appears in
Makefile:336: recipe for target 'sysdep.o' failed
make[1]: *** [sysdep.o] Error 1
Thanks.
gcc options:
-std=gnu99 -c -Demacs -I. -I. -I../lib -I../lib -D_REENTRANT
-I/usr/include/gtk-3.0 -I/usr/include/at-spi2-atk/2.0
-I/usr/include/gtk-3.0 -I/usr/include/gio-unix-2.0/ I/usr/include/cairo
--I/usr/include/pango-1.0 I/usr/include/harfbuzz -I/usr/include/pango-1.0
-I/usr/include/atk-1.0 -I/usr/include/cairo I/usr/include/pixman-1
--I/usr/include/freetype2 I/usr/include/libpng15 -I/usr/include/freetype2
-I/usr/include/libpng15 -I/usr/include/gdk-pixbuf-2.0
-I/usr/include/libpng15 -I/usr/include/glib-2.0
-I/usr/lib/glib-2.0/include -I/usr/include/freetype2
-I/usr/include/libpng15 -I/usr/include/freetype2 I/usr/include/libpng15
--D_REENTRANT -I/usr/include/librsvg-2.0 I/usr/include/gdk-pixbuf-2.0
--I/usr/include/libpng15 I/usr/include/cairo -I/usr/include/glib-2.0
-I/usr/lib/glib-2.0/include -I/usr/include/pixman-1
-I/usr/include/freetype2 -I/usr/include/libpng15 I/usr/include/freetype2
--I/usr/include/libpng15 -fopenmp I/usr/include/ImageMagick
--I/usr/include/libpng15 I/usr/include/libxml2 -I/usr/include/dbus-1.0
-I/usr/lib/dbus-1.0/include -D_REENTRANT I/usr/include/glib-2.0
--I/usr/lib/glib-2.0/include D_REENTRANT -I/usr/include/gconf/2
--I/usr/include/dbus-1.0 I/usr/lib/dbus-1.0/include
--I/usr/include/glib-2.0 I/usr/lib/glib-2.0/include
--I/usr/include/glib-2.0 I/usr/lib/glib-2.0/include
--I/usr/include/freetype2 I/usr/include/libpng15 -I/usr/include/freetype2
-I/usr/include/libpng15 -I/usr/include/freetype2 I/usr/include/libpng15
--MMD -MF deps/sysdep.d -MP I/usr/include/p11-kit-1 -D_REENTRANT
--I/usr/include/glib-2.0 I/usr/lib/glib-2.0/include -g3 -O2
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Thu, 28 Aug 2014 00:52:02 GMT)
Full text and
rfc822 format available.
Message #227 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Katsumi Yamaoka wrote:
> sysdep.c:3706:33: error: 'LC_COLLATE_MASK' undeclared
Thanks, I installed a fix in trunk bzr 117753.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Thu, 28 Aug 2014 02:40:02 GMT)
Full text and
rfc822 format available.
Message #230 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, dmantipov <at> yandex.ru, 18051 <at> debbugs.gnu.org
> Date: Wed, 27 Aug 2014 23:37:35 +0200
>
> Paul Eggert <eggert <at> cs.ucla.edu> writes:
>
> > Ah, sorry, missed that (it is a long thread...). Makes sense. I
> > assume this is on someone's TODO list since it's not done that way
> > now.
>
> Eli, that means you or me :-)
>
> I do not want to interfere your work, but in case you are busy with
> other tasks, I could do. Pls let me know.
Feel free, and thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Thu, 28 Aug 2014 03:10:02 GMT)
Full text and
rfc822 format available.
Message #233 received at 18051 <at> debbugs.gnu.org (full text, mbox):
On Wed, 27 Aug 2014 17:51:34 -0700, Paul Eggert wrote:
> Katsumi Yamaoka wrote:
>> sysdep.c:3706:33: error: 'LC_COLLATE_MASK' undeclared
> Thanks, I installed a fix in trunk bzr 117753.
Now the new build is running. Thank you!
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Thu, 28 Aug 2014 03:25:01 GMT)
Full text and
rfc822 format available.
Message #236 received at 18051 <at> debbugs.gnu.org (full text, mbox):
>> Normally one controls functions through their arguments, not the
>> environment.
> We also have the other variety, e.g. see coding-system-for-read.
Yes, we have a lot of that, but that's only used when adding an optional
argument was not really practical, in which case dynamic scoping comes
to the rescue (but with several caveats).
In the present case I don't see why we can't use an optional argument,
so an optional arg would be preferable.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 29 Aug 2014 09:01:01 GMT)
Full text and
rfc822 format available.
Message #239 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> Good idea, that would also make Glenn happy. (That's not a joke, I mean
> it seriously!)
It would make me happy as well. I have not yet started to convert my
fairly insane sorting functions to the new ones because mine are
generally based on case-insensitiveness. Also I'm not yet sure how the
new predicates will relate to functions like `compare-strings' (which
IIUC is needed until now to make sorting case-insensitive),
`sort-lines', `sort-subr' and the like. I'd hope that all of these
could profit from the new functions.
In any case, many thanks to you and Eli for the work.
martin
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 29 Aug 2014 10:00:04 GMT)
Full text and
rfc822 format available.
Message #242 received at 18051 <at> debbugs.gnu.org (full text, mbox):
martin rudalics <rudalics <at> gmx.at> writes:
> I have not yet started to convert my
> fairly insane sorting functions to the new ones because mine are
> generally based on case-insensitiveness.
I'm just working on this. `string-collate-lessp' will have the signature
(string-collate-lessp S1 S2 &optional LOCALE IGNORE-CASE)
> Also I'm not yet sure how the
> new predicates will relate to functions like `compare-strings' (which
> IIUC is needed until now to make sorting case-insensitive),
Likely, there shall also be `collate-strings'.
> `sort-lines', `sort-subr' and the like. I'd hope that all of these
> could profit from the new functions.
`sort-subr' has PREDICATE as argument, you could take
`string-collate-lessp'. Maybe with some adaptions in `sort-subr', in
order to use also LOCALE and IGNORE-CASE.
`sort-lines' uses `sort-subr', without PREDIACATE. Might be also extended.
> martin
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 29 Aug 2014 10:07:01 GMT)
Full text and
rfc822 format available.
Message #245 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> Date: Fri, 29 Aug 2014 10:59:37 +0200
> From: martin rudalics <rudalics <at> gmx.at>
> Cc: dmantipov <at> yandex.ru, 18051 <at> debbugs.gnu.org
>
> > Good idea, that would also make Glenn happy. (That's not a joke, I mean
> > it seriously!)
>
> It would make me happy as well. I have not yet started to convert my
> fairly insane sorting functions to the new ones because mine are
> generally based on case-insensitiveness. Also I'm not yet sure how the
> new predicates will relate to functions like `compare-strings' (which
> IIUC is needed until now to make sorting case-insensitive),
> `sort-lines', `sort-subr' and the like. I'd hope that all of these
> could profit from the new functions.
Case-insensitive versions of the new functions are yet to be written;
stay tuned.
For now, on MS-Windows, you can have that if you use the
NORM_IGNORECASE flag as the second argument of CompareStringW inside
w32_compare_strings.
For Posix, I guess we should run the 2 strings through towupper (or
towupper_l, if it exists), and then compare the results with
wcscoll/wcscoll_l.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 29 Aug 2014 17:23:02 GMT)
Full text and
rfc822 format available.
Message #248 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> I'm just working on this. `string-collate-lessp' will have the signature
>
> (string-collate-lessp S1 S2 &optional LOCALE IGNORE-CASE)
Fine. One additional question: Couldn't we also try to fix searching
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13041
with the new functions?
martin
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 29 Aug 2014 17:57:01 GMT)
Full text and
rfc822 format available.
Message #251 received at 18051 <at> debbugs.gnu.org (full text, mbox):
martin rudalics <rudalics <at> gmx.at> writes:
> Fine. One additional question: Couldn't we also try to fix searching
>
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13041
>
> with the new functions?
Don't know (yet). Pushed on my TODO.
> martin
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 29 Aug 2014 18:03:02 GMT)
Full text and
rfc822 format available.
Message #254 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> Case-insensitive versions of the new functions are yet to be written;
> stay tuned.
I've just committed a patch to the trunk which adds optional arguments
LOCALE and IGNORE-CASE to the collation functions.
> For now, on MS-Windows, you can have that if you use the
> NORM_IGNORECASE flag as the second argument of CompareStringW inside
> w32_compare_strings.
As usual, this I haven't implemented. I would let it to you, Eli.
> For Posix, I guess we should run the 2 strings through towupper (or
> towupper_l, if it exists), and then compare the results with
> wcscoll/wcscoll_l.
Yes.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 29 Aug 2014 19:32:02 GMT)
Full text and
rfc822 format available.
Message #257 received at 18051 <at> debbugs.gnu.org (full text, mbox):
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: martin rudalics <rudalics <at> gmx.at>, eggert <at> cs.ucla.edu, dmantipov <at> yandex.ru, 18051 <at> debbugs.gnu.org
> Date: Fri, 29 Aug 2014 20:01:50 +0200
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> > Case-insensitive versions of the new functions are yet to be written;
> > stay tuned.
>
> I've just committed a patch to the trunk which adds optional arguments
> LOCALE and IGNORE-CASE to the collation functions.
Thanks.
> > For now, on MS-Windows, you can have that if you use the
> > NORM_IGNORECASE flag as the second argument of CompareStringW inside
> > w32_compare_strings.
>
> As usual, this I haven't implemented. I would let it to you, Eli.
As usual, done.
I needed to introduce a w32-specific variable, which needs to be bound
to a non-nil value in order to have UTS#10 (a.k.a. "Unicode Collation
Algorithm", or "UCA") compliant collation order, which ignores
punctuation differences, on MS-Windows. This is because Windows
doesn't support UTF-8 as a codeset in its locales (and Windows locales
have different names anyway). This means that if a Lisp program needs
to make sure it gets a UCA-compliant collation order on all platforms,
it will have to pass a "xx_YY.UTF-8" locale on Posix platforms, and on
Windows bind that w32-specific variable to a non-nil value.
Btw, I think we will need a lot of verbiage in the ELisp manual to
make sure people understand what to expect from these functions. In
particular, the results are extremely locale- and platform-specific,
so one cannot expect exactly the same results in all cases, only
something similar.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 29 Aug 2014 21:02:01 GMT)
Full text and
rfc822 format available.
Message #260 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> Btw, I think we will need a lot of verbiage in the ELisp manual to
> make sure people understand what to expect from these functions. In
> particular, the results are extremely locale- and platform-specific,
> so one cannot expect exactly the same results in all cases, only
> something similar.
Oh yes. I will start on this next days (as usual, I'm short in time) as
well as adding test cases to fns-tests.el.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 01 Sep 2014 15:22:01 GMT)
Full text and
rfc822 format available.
Message #263 received at 18051 <at> debbugs.gnu.org (full text, mbox):
In trunk revision 117797, ls-lisp acquired the ability to sort file
names using the new string-collate-lessp function, thus producing
results that should be similar, if not identical, to what GNU ls does,
at least on GNU/Linux in the same locale. (On MS-Windows, the
behavior will be similar; it cannot be identical because Windows
doesn't implement UTS#10 (a.k.a. "UCA", the Unicode Collation
Algorithm) to the letter in its locale-dependent collation routines.)
Trunk revision 117798 implements the GNU ls -v switch in ls-lisp.
Michael (Heerdegen), as you were the one who requested these features,
please give them some testing and see if you like them.
Many thanks to Michael Albinus for all the hard work on the
infrastructure that made this possible.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Mon, 01 Sep 2014 20:47:02 GMT)
Full text and
rfc822 format available.
Message #266 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> Michael (Heerdegen), as you were the one who requested these features,
> please give them some testing and see if you like them.
I'll try and test ASAP.
> Many thanks to Michael Albinus for all the hard work on the
> infrastructure that made this possible.
I have to thank both of you! Presumably it was not easy.
Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Fri, 17 Oct 2014 20:27:01 GMT)
Full text and
rfc822 format available.
Message #269 received at 18051 <at> debbugs.gnu.org (full text, mbox):
Hi Eli and Michael,
> > Michael (Heerdegen), as you were the one who requested these features,
> > please give them some testing and see if you like them.
>
> I'll try and test ASAP.
I used that stuff for a while now, and I think everything worked as
expected.
If I remember correctly, I saw just one tiny inconsistency: with the new
ls-lisp -v switch the sorting position of a backup file named foo~ was
different from ls -v when also numbered backup files foo~n~ of the same
file existed. Dunno if this is relevant, it's a corner case.
For string collation and locales, I must say that I'm no expert at that
field and don't really know what tests could be useful for testing. I
can only say that everything seems to be ok with the locales I am using.
Let me know when you think that I could nonetheless be of any help
there.
Thanks both of you again for your work,
Michael.
Reply sent
to
Eli Zaretskii <eliz <at> gnu.org>:
You have taken responsibility.
(Sat, 18 Oct 2014 05:39:01 GMT)
Full text and
rfc822 format available.
Notification sent
to
michael_heerdegen <at> web.de:
bug acknowledged by developer.
(Sat, 18 Oct 2014 05:39:02 GMT)
Full text and
rfc822 format available.
Message #274 received at 18051-done <at> debbugs.gnu.org (full text, mbox):
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: 18051 <at> debbugs.gnu.org, Michael Albinus <michael.albinus <at> gmx.de>
> Date: Fri, 17 Oct 2014 22:26:32 +0200
>
> If I remember correctly, I saw just one tiny inconsistency: with the new
> ls-lisp -v switch the sorting position of a backup file named foo~ was
> different from ls -v when also numbered backup files foo~n~ of the same
> file existed.
Is that on Windows or on Unix?
On Windows, this is expected, as only an approximation to the Unicode
Collation Algorithm is available there.
On GNU/Linux, it would be strange, since 'ls' uses the same functions
as Emacs now does in ls-lisp.
> For string collation and locales, I must say that I'm no expert at that
> field and don't really know what tests could be useful for testing. I
> can only say that everything seems to be ok with the locales I am using.
That's good enough for me, so I'm closing the bug.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18051; Package
emacs.
(Sat, 18 Oct 2014 14:28:01 GMT)
Full text and
rfc822 format available.
Message #277 received at 18051-done <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> > with the new ls-lisp -v switch the sorting position of a backup file
> > named foo~ was different from ls -v when also numbered backup files
> > foo~n~ of the same file existed.
> On GNU/Linux, it would be strange, since 'ls' uses the same functions
> as Emacs now does in ls-lisp.
Gnu/Linux. But I can't reproduce this anymore, it works as expected,
probably I was mistaken.
Thanks,
Michael.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Sun, 16 Nov 2014 12:24:03 GMT)
Full text and
rfc822 format available.
This bug report was last modified 9 years and 173 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.