GNU bug report logs - #39002
[feature requests] calendar-hebrew [code included]

Previous Next

To reply to this bug, email your comments to 39002 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Boruch Baum <boruch_baum <at> gmx.com>
To: Emacs Bug Reporting <bug-gnu-emacs <at> gnu.org>
Subject: [feature requests] calendar-hebrew [code included]
Date: Tue, 7 Jan 2020 01:28:30 -0500

[Message part 1 (text/plain, inline)]

I've turned to playing with the emacs calendar and diary, and have
several suggestions for its surprisingly good Hebrew support. Attached
is an working elisp proof-of-concept.
Attach: /home/optimum/Projects/emacs-related/calendar-ivrit/cal-ivrit.el cal-ivrit.el

1) sunset-awareness. Currently, emacs does warn the user that a Hebrew
   date being presented is for the period midnight-sunset, but its
   trivial for emacs to check whether the current time is after sunset
   and to increment the date accordingly. In such cases, the date is
   labeled `eve of foo' to make it clear that sunset has been taken into
   account.

2) Hebrew data in Hebrew. Instead of transliterating Hebrew words into
   Latin characters, print them in unicode Hebrew.

2.1) This exposed three bugs in the bidi rendering of the diary buffer:

    A) Hebrew text lines in the diary are rendered left justified.

    B) Hebrew text lines in the diary are rendered in reverse sequence.

    C) RTL and LTR lines are rendered out of sequence to each other.

2.1.1) Bug A can be avoided by locally redefining the regexes that
       identify bidi paragraphs, and I've included that in the attached
       code.

2.1.2) Bugs B and C may possibly be related to bug #15541.

3) The code includes a generalized function for rendering Hebrew numbers
   for Hebrew dates, which is an improvement on a function
   `Footnote-hebrew-numeric' submitted in bug report #29759. There
   should only be a single function, generically named.

4) Parasha information is presented for the entire week leading up to
   its Shabbat reading. Currently, it's only displayed on Saturdays.

--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1  7286 0036 9E45 1595 8BC0

[cal-ivrit.el (text/plain, attachment)]

Message #8 received at 39002 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Boruch Baum <boruch_baum <at> gmx.com>
Cc: 39002 <at> debbugs.gnu.org
Subject: Re: bug#39002: [feature requests] calendar-hebrew [code included]
Date: Tue, 07 Jan 2020 17:54:22 +0200

> Date: Tue, 7 Jan 2020 01:28:30 -0500
> From: Boruch Baum <boruch_baum <at> gmx.com>
> 
> 2.1) This exposed three bugs in the bidi rendering of the diary buffer:
> 
>     A) Hebrew text lines in the diary are rendered left justified.
> 
>     B) Hebrew text lines in the diary are rendered in reverse sequence.
> 
>     C) RTL and LTR lines are rendered out of sequence to each other.
> 
> 2.1.1) Bug A can be avoided by locally redefining the regexes that
>        identify bidi paragraphs, and I've included that in the attached
>        code.
> 
> 2.1.2) Bugs B and C may possibly be related to bug #15541.

I'm interested to understand better these problems.  Please give
detailed instructions for how to reproduce them with your calendar
code, or show a screenshot, because I don't think I understand well
enough from the above description what the display looks like.

Thanks.

Message #11 received at 39002 <at> debbugs.gnu.org (full text, mbox):

From: Boruch Baum <boruch_baum <at> gmx.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 39002 <at> debbugs.gnu.org
Subject: Re: bug#39002: [feature requests] calendar-hebrew [code included]
Date: Tue, 7 Jan 2020 12:11:41 -0500

[Message part 1 (text/plain, inline)]

On 2020-01-07 17:54, Eli Zaretskii wrote:
> > Date: Tue, 7 Jan 2020 01:28:30 -0500
> > From: Boruch Baum <boruch_baum <at> gmx.com>
> >
> > 2.1) This exposed three bugs in the bidi rendering of the diary buffer:
> >
> >     A) Hebrew text lines in the diary are rendered left justified.
> >
> >     B) Hebrew text lines in the diary are rendered in reverse sequence.
> >
> >     C) RTL and LTR lines are rendered out of sequence to each other.
> >
> > 2.1.1) Bug A can be avoided by locally redefining the regexes that
> >        identify bidi paragraphs, and I've included that in the attached
> >        code.
> >
> > 2.1.2) Bugs B and C may possibly be related to bug #15541.
>
> I'm interested to understand better these problems.  Please give
> detailed instructions for how to reproduce them with your calendar
> code, or show a screenshot, because I don't think I understand well
> enough from the above description what the display looks like.
>
> Thanks.

Attached is a screenshot displaying three relevant buffers. Below the
calendar buffer is the content of my ~/.emacs.d/diary file. Note the
sequence of elements: 1) Title line (LTR, implicit; 2) Hebrew date
(RTL); 3) sun times (LTR); 4) Parasha (RTL). However, in the diary
output buffer, at the bottom of the screen shot, the lines are displayed
in the sequence 1,4,2,3. That covers bugs B & C. Bug A and a fix for it
would be reproduced by applying / removing the hook function at line 372
of the previously submitted `cal-ivrit.el'


  (defun cal-ivrit-diary-fancy-display-mode-hook-function ()
    (setq bidi-paragraph-start-re "^")
    (setq bidi-paragraph-separate-re "^"))

  (add-hook 'diary-fancy-display-mode-hook
    'cal-ivrit-diary-fancy-display-mode-hook-function)

For an illustration of emacs bug #15541 (nine+ years old!), see the
attached org-mode file.

--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1  7286 0036 9E45 1595 8BC0

[emacs_bug_report_39002.png (image/png, attachment)]

[emacs_bug_15541.org (text/plain, attachment)]

Message #14 received at 39002 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Boruch Baum <boruch_baum <at> gmx.com>
Cc: 39002 <at> debbugs.gnu.org
Subject: Re: bug#39002: [feature requests] calendar-hebrew [code included]
Date: Tue, 07 Jan 2020 19:33:25 +0200

> Date: Tue, 7 Jan 2020 12:11:41 -0500
> From: Boruch Baum <boruch_baum <at> gmx.com>
> Cc: 39002 <at> debbugs.gnu.org
> 
> Attached is a screenshot displaying three relevant buffers. Below the
> calendar buffer is the content of my ~/.emacs.d/diary file. Note the
> sequence of elements: 1) Title line (LTR, implicit; 2) Hebrew date
> (RTL); 3) sun times (LTR); 4) Parasha (RTL). However, in the diary
> output buffer, at the bottom of the screen shot, the lines are displayed
> in the sequence 1,4,2,3.

I don't think I see the part with the 1,4,2,3 order (sorry if I'm
blind), but did you try to use the function
bidi-string-mark-left-to-right?  IIUC the problem, that function was
created exactly for these use cases, where bidi reordering causes a
jumble in what is supposed to be columnar display of several
substrings.  You should wrap each substring in the wrapper produced by
bidi-string-mark-left-to-right, and then concatenate the results with
the "|" or whatever separators in-between.

IOW, it should not be necessary to change the paragraph direction in
these cases.

> That covers bugs B & C. Bug A and a fix for it
> would be reproduced by applying / removing the hook function at line 372
> of the previously submitted `cal-ivrit.el'
> 
> 
>   (defun cal-ivrit-diary-fancy-display-mode-hook-function ()
>     (setq bidi-paragraph-start-re "^")
>     (setq bidi-paragraph-separate-re "^"))
> 
>   (add-hook 'diary-fancy-display-mode-hook
>     'cal-ivrit-diary-fancy-display-mode-hook-function)

Removing this and then doing what?

> For an illustration of emacs bug #15541 (nine+ years old!), see the
> attached org-mode file.

It is not a bug, it's the intended behavior.  If you have Org files
with such long headings of RTL text, you should set the variable
bidi-paragraph-direction to nil in that buffer (or even to
right-to-left, if all of the headings use RTL text).

Message #17 received at 39002 <at> debbugs.gnu.org (full text, mbox):

From: Boruch Baum <boruch_baum <at> gmx.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 39002 <at> debbugs.gnu.org
Subject: Re: bug#39002: [feature requests] calendar-hebrew [code included]
Date: Tue, 7 Jan 2020 13:29:27 -0500

On 2020-01-07 19:33, Eli Zaretskii wrote:
> > Date: Tue, 7 Jan 2020 12:11:41 -0500
> >
> > Attached is a screenshot displaying three relevant buffers. Below the
> > calendar buffer is the content of my ~/.emacs.d/diary file. Note the
> > sequence of elements: 1) Title line (LTR, implicit; 2) Hebrew date
> > (RTL); 3) sun times (LTR); 4) Parasha (RTL). However, in the diary
> > output buffer, at the bottom of the screen shot, the lines are displayed
> > in the sequence 1,4,2,3.
>
> I don't think I see the part with the 1,4,2,3 order

Look at the sequence of displayed lines in the screenshot - for example,
the first line after the LTR heading line is the parasha RTL line, but
the diary file (the middle buffer) says that the parasha should be the
fourth line, after the sun times.

> did you try to use the function bidi-string-mark-left-to-right?

I don't remember, but I can give it a try. I did play with manually
inserting several unicode bidi control sequences, and none were helpful.

> IIUC the problem, that function was created exactly for these use
> cases, where bidi reordering causes a jumble in what is supposed to be
> columnar display of several substrings.

But these aren't sub-strings, they're discrete paragraphs, as defined by
the bidi regex. Even without the regex redefinition, they are discrete
lines.

> You should wrap each substring in the wrapper produced by
> bidi-string-mark-left-to-right, and then concatenate the results with
> the "|" or whatever separators in-between.

That's doable, but generally burdensome, especially for discrete lines /
paragraphs.

> IOW, it should not be necessary to change the paragraph direction in
> these cases.

But in the current environment, it's much easier. It's the difference of
a one-time setting of a general rule (the bidi paragraph regex), and
having to programmatically perform concatenations for each and every
relevant string in the buffer.

> > That covers bugs B & C. Bug A and a fix for it
> > would be reproduced by applying / removing the hook function at line 372
> > of the previously submitted `cal-ivrit.el'
> >
> >
> >   (defun cal-ivrit-diary-fancy-display-mode-hook-function ()
> >     (setq bidi-paragraph-start-re "^")
> >     (setq bidi-paragraph-separate-re "^"))
> >
> >   (add-hook 'diary-fancy-display-mode-hook
> >     'cal-ivrit-diary-fancy-display-mode-hook-function)
>
> Removing this and then doing what?

With the hook function installed, you can open a diary page and see the
Hebrew lines properly aligned. Without the function, those lines justify
left.

> > For an illustration of emacs bug #15541 (nine+ years old!), see the
> > attached org-mode file.
>
> It is not a bug, it's the intended behavior.

It's marked "won't fix". See the discussion there.

> If you have Org files with such long headings of RTL text, you should
> set the variable bidi-paragraph-direction to nil in that buffer (or
> even to right-to-left, if all of the headings use RTL text).

The common case is a single line Hebrew heading, followed by a window
resize, shrinking the number of columns. I presented a case of very long
headings in order for it to obvious that what is happening is that the
lines are being displayed down-up.

Finally, I don't want to lose focus that this report was primarily a
feature request, with working code submitted. This bug discussion is
important, but if you feel it's worth continuing, should I file it/them
as separate reports?

--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1  7286 0036 9E45 1595 8BC0

Message #20 received at 39002 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Boruch Baum <boruch_baum <at> gmx.com>
Cc: 39002 <at> debbugs.gnu.org
Subject: Re: bug#39002: [feature requests] calendar-hebrew [code included]
Date: Tue, 07 Jan 2020 20:48:33 +0200

> Date: Tue, 7 Jan 2020 13:29:27 -0500
> From: Boruch Baum <boruch_baum <at> gmx.com>
> Cc: 39002 <at> debbugs.gnu.org
> 
> > I don't think I see the part with the 1,4,2,3 order
> 
> Look at the sequence of displayed lines in the screenshot - for example,
> the first line after the LTR heading line is the parasha RTL line, but
> the diary file (the middle buffer) says that the parasha should be the
> fourth line, after the sun times.

Sorry, I'm still in the dark here.  What is "the LTR heading line"?
what is its text?  (There are quite a few lines in the screenshot
which could be candidates for that description).

Also, you are talking about "lines", but are those logical lines
(i.e. each one ends with a newline), or did Emacs wrap a single long
line?

> > did you try to use the function bidi-string-mark-left-to-right?
> 
> I don't remember, but I can give it a try. I did play with manually
> inserting several unicode bidi control sequences, and none were helpful.

That function exists because the correct solution isn't obvious.

> > IIUC the problem, that function was created exactly for these use
> > cases, where bidi reordering causes a jumble in what is supposed to be
> > columnar display of several substrings.
> 
> But these aren't sub-strings, they're discrete paragraphs, as defined by
> the bidi regex. Even without the regex redefinition, they are discrete
> lines.

If they are separate line with newlines between them, I don't see how
the order of the lines could change.  Bidi reordering doesn't reorder
logical lines.

> > IOW, it should not be necessary to change the paragraph direction in
> > these cases.
> 
> But in the current environment, it's much easier.

Maybe, but it doesn't mean it's TRT.  (But I'm not sure I understand
the problem now, so maybe I was talking nonsense.  It would be easier
if you just told me how to activate your code, so I could see the
issues with my own eyes.  I don't use calendar and diary in Emacs, so
I need detailed instructions.)

> It's the difference of a one-time setting of a general rule (the
> bidi paragraph regex), and having to programmatically perform
> concatenations for each and every relevant string in the buffer.

This solution is quite dramatic, so it should be avoided if another,
less dramatic one is possible.

> > >   (defun cal-ivrit-diary-fancy-display-mode-hook-function ()
> > >     (setq bidi-paragraph-start-re "^")
> > >     (setq bidi-paragraph-separate-re "^"))
> > >
> > >   (add-hook 'diary-fancy-display-mode-hook
> > >     'cal-ivrit-diary-fancy-display-mode-hook-function)
> >
> > Removing this and then doing what?
> 
> With the hook function installed, you can open a diary page and see the
> Hebrew lines properly aligned. Without the function, those lines justify
> left.

Sorry, what does "open a diary page" entail?  Again, I don't use these
features, so I need detailed instructions.

> > If you have Org files with such long headings of RTL text, you should
> > set the variable bidi-paragraph-direction to nil in that buffer (or
> > even to right-to-left, if all of the headings use RTL text).
> 
> The common case is a single line Hebrew heading, followed by a window
> resize, shrinking the number of columns. I presented a case of very long
> headings in order for it to obvious that what is happening is that the
> lines are being displayed down-up.

My proposal works on those cases as well.

> Finally, I don't want to lose focus that this report was primarily a
> feature request, with working code submitted. This bug discussion is
> important, but if you feel it's worth continuing, should I file it/them
> as separate reports?

I think it all is relevant, because I'd like to make sure your
bidi-related tricks are indeed the right solutions to the issues you
saw.  So I'd appreciate if you could help me understand better what
were the original problems.

Thanks.

Message #23 received at 39002 <at> debbugs.gnu.org (full text, mbox):

From: Boruch Baum <boruch_baum <at> gmx.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 39002 <at> debbugs.gnu.org
Subject: Re: bug#39002: [feature requests] calendar-hebrew [code included]
Date: Tue, 7 Jan 2020 14:49:02 -0500

On 2020-01-07 20:48, Eli Zaretskii wrote:
> > Date: Tue, 7 Jan 2020 13:29:27 -0500
> > From: Boruch Baum <boruch_baum <at> gmx.com>
> > Cc: 39002 <at> debbugs.gnu.org
> >
> > > I don't think I see the part with the 1,4,2,3 order
> >
> > Look at the sequence of displayed lines in the screenshot - for example,
> > the first line after the LTR heading line is the parasha RTL line, but
> > the diary file (the middle buffer) says that the parasha should be the
> > fourth line, after the sun times.
>
> Sorry, I'm still in the dark here.  What is "the LTR heading line"?
> what is its text?

Tuesday, January 7, 2020:  Tzom Teveth
======================================

> Also, you are talking about "lines", but are those logical lines
> (i.e. each one ends with a newline),

It seems so, since altering the bidi paragraph regex alters the
justification.

> or did Emacs wrap a single long line?

Good question. How could I tell the difference? Wouldn't the buffer
result look the same?

> > > IIUC the problem, that function was created exactly for these use
> > > cases, where bidi reordering causes a jumble in what is supposed to be
> > > columnar display of several substrings.
> >
> > But these aren't sub-strings, they're discrete paragraphs, as defined by
> > the bidi regex. Even without the regex redefinition, they are discrete
> > lines.
>
> If they are separate line with newlines between them, I don't see how
> the order of the lines could change.  Bidi reordering doesn't reorder
> logical lines.

... ok ... but you do have the screenshot figuratively staring right
back at you ... that is the essence of the bugs reported.

> > > IOW, it should not be necessary to change the paragraph direction in
> > > these cases.
> >
> > But in the current environment, it's much easier.
>
> Maybe, but it doesn't mean it's TRT.  (But I'm not sure I understand
> the problem now, so maybe I was talking nonsense.  It would be easier
> if you just told me how to activate your code, so I could see the
> issues with my own eyes.  I don't use calendar and diary in Emacs, so
> I need detailed instructions.)

The code I sent along with the original report includes the customary
commentary section with installation notes / instructions / etc.
Evaluate that file. Set variables `calendar-longitude',
`calendar-latitude', and `calendar-location'. I think also that variable
`diary-display-function' may need to be set to ‘diary-fancy-display'.
Add the hook function as described there and in my previous post(s?).
The commentary section also includes sample content for what you need to
put in ~/.emacs.d/diary (the middle third of the screenshot I sent), but
you see the content used in the screenshot I sent, which is slightly
different. Then start the calendar with .... `M-x calendar` (!). That
would be the top third of the screenshot I sent. Use the arrow keys to
navigate dates in the display, and press `d' to open a diary page for
the currently selected date (That would be the bottom third of the
much-cited screenshot). Edit ~/.emacs.d/diary to change what the diary
page will display.

>  I don't use calendar and diary in Emacs, so I need detailed
> instructions.)

The calendar.el and diary-lib.el packages each say their author is
Edward M. Reingold <reingold <at> cs.uiuc.edu>, and their maintainer is
Glenn Morris><rgm <at> gnu.org>. Can they suckered into this thread?

> > With the hook function installed, you can open a diary page and see the
> > Hebrew lines properly aligned. Without the function, those lines justify
> > left.
>
> Sorry, what does "open a diary page" entail?  Again, I don't use these
> features, so I need detailed instructions.

I also had trouble getting up to speed on these features (#38859,
#38866). I explained the procedure in detail above, but briefly: 'M-x
calendar', followed by 'd'.

--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1  7286 0036 9E45 1595 8BC0

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.