GNU bug report logs - #9794
24.0.90; `format-time-string' no good for %Z

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 9794 in the body.
You can then email your comments to 9794 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: <bug-gnu-emacs <at> gnu.org>
Subject: 24.0.90; `format-time-string' no good for %Z
Date: Tue, 18 Oct 2011 23:44:15 -0700

emacs -Q
 
M-: (format-time-string "Started: %a %b %e %T %Y (%Z)" (current-time))
 
For me that produces this:
 
"Started: Tue Oct 18 23:40:28 2011 ()"
 
The %Z doesn't work at all.  This is a regression that started in Emacs
22.  In Emacs 20 and 21 (emacs -Q) it works correctly, displaying this:
 
"Started: Tue Oct 18 23:40:28 2011 (Pacific Daylight Time)"

If Emacs 20 can pick up the name Pacific Daylight Time correctly, from wherever
it gets it, then so should Emacs 24 be able to do so.  No user config (e.g.
setting env vars) should be necessary.  (That doesn't preclude user config - the
point is that even without it %Z should DTRT.)

In GNU Emacs 24.0.90.1 (i386-mingw-nt5.1.2600)
 of 2011-10-18 on MARVIN
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (4.6) --no-opt --cflags -I"C:/Program
Files (x86)/GnuWin32/include" -ID:/devel/emacs/libXpm-3.5.8/include
-ID:/devel/emacs/libXpm-3.5.8/src -ID:/devel/emacs/gnutls-2.10.5-x86/include
--ldflags -LD:/devel/emacs/gnutls-2.10.5-x86/lib'
 
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  value of $XMODIFIERS: nil
  locale-coding-system: cp1252
  default enable-multibyte-characters: t
 

Message #10 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: <9794 <at> debbugs.gnu.org>
Subject: RE: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 00:43:28 -0700

Didn't realize this bug was already filed as #641.

It's unfortunate that nothing was ever done about this regression.

I stumbled on it again only because of someone else's code that uses "...(%Z)".
And that is likely to be fairly common, given that people not on Windows will
not know that %Z in Emacs is broken on Windows.  How many Emacs users use
Windows?  How long will this be ignored, so they continue to see "...()" instead
of something meaningful to them?

Stefan's bottom line in the #641 bug thread was this:

"I could live with it, tho its usefulness is far from obvious.
 In any case it would not be for Emacs-23 and I'd recommend potential
 hackers to work on something else as that's more likely to be useful."

Poor Emacs.  Poor users.  It's usefulness is completely obvious: Windows users
lose information that is meant to help them.  And just because a non-Windows
developer thinks that letting them see "(Pacific Daylight Time)" is not useful
to them and "()" is more meaningful.  Sheesh.

Message #13 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 09:47:40 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Date: Tue, 18 Oct 2011 23:44:15 -0700
> 
> emacs -Q
>  
> M-: (format-time-string "Started: %a %b %e %T %Y (%Z)" (current-time))
>  
> For me that produces this:
>  
> "Started: Tue Oct 18 23:40:28 2011 ()"
>  
> The %Z doesn't work at all.  This is a regression that started in Emacs
> 22.  In Emacs 20 and 21 (emacs -Q) it works correctly, displaying this:
>  
> "Started: Tue Oct 18 23:40:28 2011 (Pacific Daylight Time)"
> 
> If Emacs 20 can pick up the name Pacific Daylight Time correctly, from wherever
> it gets it, then so should Emacs 24 be able to do so.  No user config (e.g.
> setting env vars) should be necessary.  (That doesn't preclude user config - the
> point is that even without it %Z should DTRT.)

This is an old bug #641, see there regarding the explanation why the
current behavior is correct.

Message #16 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 10:33:29 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Date: Wed, 19 Oct 2011 00:43:28 -0700
> 
> And just because a non-Windows developer thinks that letting them
> see "(Pacific Daylight Time)" is not useful to them and "()" is more
> meaningful.  Sheesh.

No, it's because a _Windows_ developer found out that the Windows
time-zone names violate international standards for what %Z should
produce, which breaks other Emacs features that use the results.

Message #19 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Jason Rumney <jasonr <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9794 <at> debbugs.gnu.org, Drew Adams <drew.adams <at> oracle.com>
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 21:20:06 +0800

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: "Drew Adams" <drew.adams <at> oracle.com>
>> Date: Wed, 19 Oct 2011 00:43:28 -0700
>> 
>> And just because a non-Windows developer thinks that letting them
>> see "(Pacific Daylight Time)" is not useful to them and "()" is more
>> meaningful.  Sheesh.
>
> No, it's because a _Windows_ developer found out that the Windows
> time-zone names violate international standards for what %Z should
> produce, which breaks other Emacs features that use the results.

The international standards alone aren't a problem - GNU software in
general does not follow standards slavishly. The real problem is that
for many uses of time format strings (which correctly check for an empty
%Z string and use %z as a backup), in mail, news, HTTP headers, XML
documents and similar uses which rely on the strings being standards
compliant, the non-compliant long forms returned by Windows tzname()
cause real problems which are much more severe than the inconveniences
that this change has caused.

One proposal in that thread was to introduce a new format specifier to
print the long names (on non-Windows platforms it could output the
commonly used "Continent/City" format). Another proposal was that %EZ
could be used, which is especially fitting, for the Windows timezone
names, which are apparently locale sensitive (which was one of the
reported problems that led to them being removed in the first place).

Message #22 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Jason Rumney'" <jasonr <at> gnu.org>, "'Eli Zaretskii'" <eliz <at> gnu.org>
Cc: 9794 <at> debbugs.gnu.org
Subject: RE: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 07:28:51 -0700

> >> And just because a non-Windows developer thinks that letting them
> >> see "(Pacific Daylight Time)" is not useful to them and "()" is
> >> more meaningful.  Sheesh.
> >
> > No, it's because a _Windows_ developer found out that the Windows
> > time-zone names violate international standards for what %Z should
> > produce, which breaks other Emacs features that use the results.
> 
> The international standards alone aren't a problem - GNU software in
> general does not follow standards slavishly. The real problem is that
> for many uses of time format strings (which correctly check 
> for an empty %Z string and use %z as a backup), in mail, news, HTTP
> headers, XML documents and similar uses which rely on the strings
> being standards compliant, the non-compliant long forms returned by
> Windows tzname() cause real problems which are much more severe than
> the inconveniences that this change has caused.
> 
> One proposal in that thread was to introduce a new format specifier to
> print the long names (on non-Windows platforms it could output the
> commonly used "Continent/City" format). Another proposal was that %EZ
> could be used, which is especially fitting, for the Windows timezone
> names, which are apparently locale sensitive (which was one of the
> reported problems that led to them being removed in the first place).

Yes - that should be a no-brainer.  The only question should be about the
details: what format specifiers with what behavior.

I suggest format specifiers that let you alternatively do all of the following
for the case of time zone names (i.e. "pretty" names, not just %z numbers).

1. Use only POSIX-compliant time-zone pretty names, which can mean "" (empty -
no available POSIX name).

2. Use any available nonempty time-zone pretty names, with priority to nonempty
POSIX-compliant pretty names.

3. Same as #2, but with priority to system-supplied names, even when a
corresponding nonempty POSIX name is available.

4. Use only nonempty POSIX-compliant pretty names, when available, and fall back
to what %z does in cases where the POSIX name is empty.

#2 and #3 would also fall back to %z when no nonempty name is available.

Again, such details should be open for discussion, but it should be clear that
_some_ fix should be found so that programmers are able to provide "pretty" time
zone info to users whenever such info is available, whether that info is POSIX
or not.

Forcing the loss of useful time-zone info just because a user-understandable
time-zone description is not understandable to POSIX puts POSIX on top and
people at the bottom.  Users and Emacs programmers deserve better than that.

Message #25 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Eli Zaretskii'" <eliz <at> gnu.org>
Cc: 9794 <at> debbugs.gnu.org
Subject: RE: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 07:28:59 -0700

> This is an old bug #641, see there regarding the explanation why the
> current behavior is correct.

Yes, it is an old bug, with _no_ good explanation why the regression shouldn't
be fixed and plenty of reasons why it should.  Stefan even stated that he could
"live with" fixing it, though that fix "would not be for Emacs 23".

Emacs is not limited to POSIX.  %Z is Emacs.  %Z before the regression provided
useful info for everyone, including Windows users.

At a minimum, %Z should be made to fall back to %z or something, so that at
least SOME time zone info is provided to the user, instead of just an empty
string.  If you _also_ want to be able to be "POSIX-compliant" then provide a
different format indicator from %Z for that.

It is inexcusably misguided to simply remove the time zone info for Windows
users, all in the name of respecting POSIXness.  Users deserve better.  At least
give them the %z info if you don't like the natural-language time-zone info that
Windows provides: better "(-0700)" than "()".

Message #28 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Eli Zaretskii'" <eliz <at> gnu.org>
Cc: 9794 <at> debbugs.gnu.org
Subject: RE: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 07:29:48 -0700

> > And just because a non-Windows developer thinks that letting them
> > see "(Pacific Daylight Time)" is not useful to them and "()" is more
> > meaningful.  Sheesh.
> 
> No, it's because a _Windows_ developer found out that the Windows
> time-zone names violate international standards for what %Z should
> produce, which breaks other Emacs features that use the results.

No.  There is nothing to prevent Emacs from DTRT and providing _both_ the
Windows time-zone info and POSIX-standard info, using separate formatting codes.
Read the #641 thread.

Nothing in the GNU charter/mission/philosophy requires Emacs to _limit_ itself
to POSIX compliance.  Providing a POSIX-compliant time-zone formatting code is
one thing - a good thing, no doubt.  Not providing other available, non-POSIX
time-zone information that can be useful to users is another thing - a sorry
mistake.

Message #31 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 17:13:03 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Cc: <9794 <at> debbugs.gnu.org>
> Date: Wed, 19 Oct 2011 07:29:48 -0700
> 
> There is nothing to prevent Emacs from DTRT and providing _both_ the
> Windows time-zone info and POSIX-standard info, using separate formatting codes.

Nothing but a volunteer who would write the code to do it.

Message #34 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jason Rumney <jasonr <at> gnu.org>
Cc: 9794 <at> debbugs.gnu.org, drew.adams <at> oracle.com
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 17:26:45 +0200

> From: Jason Rumney <jasonr <at> gnu.org>
> Cc: Drew Adams <drew.adams <at> oracle.com>,  9794 <at> debbugs.gnu.org
> Date: Wed, 19 Oct 2011 21:20:06 +0800
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > No, it's because a _Windows_ developer found out that the Windows
> > time-zone names violate international standards for what %Z should
> > produce, which breaks other Emacs features that use the results.
> 
> The international standards alone aren't a problem - GNU software in
> general does not follow standards slavishly. The real problem is that
> for many uses of time format strings (which correctly check for an empty
> %Z string and use %z as a backup), in mail, news, HTTP headers, XML
> documents and similar uses which rely on the strings being standards
> compliant, the non-compliant long forms returned by Windows tzname()
> cause real problems which are much more severe than the inconveniences
> that this change has caused.

Isn't that what I said: that _using_ the non-standard results of %Z
caused trouble to other Emacs features?

> One proposal in that thread was to introduce a new format specifier to
> print the long names (on non-Windows platforms it could output the
> commonly used "Continent/City" format). Another proposal was that %EZ
> could be used, which is especially fitting, for the Windows timezone
> names, which are apparently locale sensitive (which was one of the
> reported problems that led to them being removed in the first place).

Are there any problems to produce localized (i.e. non-ASCII) timezone
names in strftime?  Don't Posix systems do that in some locales?

Message #37 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jason Rumney <jasonr <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9794 <at> debbugs.gnu.org, drew.adams <at> oracle.com
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Wed, 19 Oct 2011 18:08:59 +0200

> From: Jason Rumney <jasonr <at> gnu.org>
> Cc: Drew Adams <drew.adams <at> oracle.com>,  9794 <at> debbugs.gnu.org
> Date: Wed, 19 Oct 2011 21:20:06 +0800
> 
> One proposal in that thread was to introduce a new format specifier to
> print the long names (on non-Windows platforms it could output the
> commonly used "Continent/City" format). Another proposal was that %EZ
> could be used, which is especially fitting, for the Windows timezone
> names, which are apparently locale sensitive (which was one of the
> reported problems that led to them being removed in the first place).

I used the %EZ format proposed by Andreas Schwab, and came up with
the Windows-specific patch below.

Paul, would such a change be acceptable by gnulib?

=== modified file 'ChangeLog'
--- ChangeLog	2011-10-18 18:12:53 +0000
+++ ChangeLog	2011-10-19 16:02:14 +0000
@@ -1,3 +1,8 @@
+2011-10-19  Eli Zaretskii  <eliz <at> gnu.org>
+
+	* lib/strftime.c (strftime_case_) [_WIN32 || __WIN32__]: Provide
+	non-empty time-zone string only for the %EZ format specifier.
+
 2011-10-18  Jan Djärv  <jan.h.d <at> swipnet.se>
 
 	* configure.in (GLIB_REQUIRED, GTK_REQUIRED): Set to 2.10 (Bug#9786).

=== modified file 'lib/strftime.c'
--- lib/strftime.c	2011-03-31 04:24:03 +0000
+++ lib/strftime.c	2011-10-19 15:48:42 +0000
@@ -1302,6 +1302,12 @@ strftime_case_ (bool upcase, STREAM_OR_C
             }
 
 #if HAVE_TZNAME
+#if (defined _WIN32 || defined __WIN32__)
+	  /* Microsoft runtime produces time-zone names that are not
+	     RFC822 compliant, and are also localized.  So we only
+	     produce them for %EZ.  */
+	  if (modifier == L_('E'))
+#endif
           /* The tzset() call might have changed the value.  */
           if (!(zone && *zone) && tp->tm_isdst >= 0)
             zone = tzname[tp->tm_isdst != 0];

=== modified file 'nt/ChangeLog'
--- nt/ChangeLog	2011-09-04 21:52:59 +0000
+++ nt/ChangeLog	2011-10-19 16:02:25 +0000
@@ -1,3 +1,7 @@
+2011-10-19  Eli Zaretskii  <eliz <at> gnu.org>
+
+	* config.nt (HAVE_TZNAME, HAVE_DECL_TZNAME): Define.
+
 2011-09-04  Paul Eggert  <eggert <at> cs.ucla.edu>
 
 	* config.nt (HAVE_SNPRINTF): New macro.

=== modified file 'nt/config.nt'
--- nt/config.nt	2011-09-26 03:20:03 +0000
+++ nt/config.nt	2011-10-19 15:57:12 +0000
@@ -187,7 +187,14 @@ along with GNU Emacs.  If not, see <http
 
 #undef TM_IN_SYS_TIME
 #undef HAVE_TM_ZONE
-#undef HAVE_TZNAME
+
+/* Define to 1 if you don't have `tm_zone' but do have the external array
+   `tzname'. */
+#define HAVE_TZNAME 1
+
+/* Define to 1 if you have the declaration of `tzname', and to 0 if you don't.
+   */
+#define HAVE_DECL_TZNAME 1
 
 #undef const
 

Message #40 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9794 <at> debbugs.gnu.org, drew.adams <at> oracle.com, Jason Rumney <jasonr <at> gnu.org>
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 00:48:02 -0700

On 10/19/11 09:08, Eli Zaretskii wrote:
> Paul, would such a change be acceptable by gnulib?

That's a looooong story.  I went through the cited bug reports
and found three problems:

1.  The original problem dates back to my 1999-10-19 patch to
    editfns.c.  This patch fixed several problems by adding
    conversions between Emacs's string encoding and the underlying
    system's encoding for strings.  The patch added conversions for
    several functions, including format-time-string, but it did not
    alter current-time-zone, because I didn't think of the possibility
    that time zone names might be non-ASCII on some platforms.

    Michael Schierl reported the current-time-zone issue to RMS in 2007; see
    <http://lists.gnu.org/archive/html/emacs-devel/2007-06/msg00334.html>.
    But somehow the light bulb didn't go off, and current-time-zone
    wasn't fixed.  Instead a workaround was installed, which basically
    said "On Windows, don't ever generate time zone names, because
    they might contain non-ASCII characters."

    Now that I see this problem, I have changed current-time-zone so
    that it converts non-ASCII characters properly, which is what
    Schierl's bug report was actually about.  I installed the fix in
    the trunk, as bzr 106149.

2.  Because of the 2007 workaround, in the Windows port
    (format-time-string "%Z") always generates an empty string.  This
    is obviously wrong and is not what users expect, and should be
    fixed by reverting the Windows part of the 2007 workaround, which
    we can safely do now that the current-time-zone bug has been
    fixed.  Your patch to nt/config.nt should do this, and I agree
    it should be installed.  (There is no need to change lib/strftime.c.)

3.  There is some talk in the bug reports about time zone names and
    RFC822 compliance.  But time zone names in general do not conform
    to RFC822.  For example, on a POSIX host if you set the TZ
    environment variable as follows:

       TZ="<-8+>0"

    then (format-time-string "%Z") returns "-8+", and that's
    the correct value, even if it's not an RFC822 zone.

    Any code that assumes that (format-time-string "%Z") must generate
    an RFC822 zone is making an unwarranted assumption and should be
    fixed.  I did a quick scan for such code in Emacs and didn't find
    any.  Perhaps there's some out-of-Emacs Lisp code that's making
    the assumption, but if so, that code needs to be fixed because
    in general it does not work and has never worked.

Message #43 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9794 <at> debbugs.gnu.org, drew.adams <at> oracle.com, jasonr <at> gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 11:24:24 +0200

> Date: Thu, 20 Oct 2011 00:48:02 -0700
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> CC: Jason Rumney <jasonr <at> gnu.org>, drew.adams <at> oracle.com, 
>  9794 <at> debbugs.gnu.org
> 
>     Michael Schierl reported the current-time-zone issue to RMS in 2007; see
>     <http://lists.gnu.org/archive/html/emacs-devel/2007-06/msg00334.html>.
>     But somehow the light bulb didn't go off, and current-time-zone
>     wasn't fixed.  Instead a workaround was installed, which basically
>     said "On Windows, don't ever generate time zone names, because
>     they might contain non-ASCII characters."

That's not a fair description of what was done.  In response to my
question of whether this was too drastic a measure, Jason explained
the real reason for the change he did:

  Non-ASCII is a side issue, and treating it as the main issue is what
  caused us to install an incorrect fix originally.

  The original issue reported a number of years ago, was that the timezone
  names for the Japanese locale on Windows are not RFC 822 compliant. So
  we suppressed them in certain conditions, but in fact, the timezone
  names are not RFC 822 compliant in most locales on Windows. Unless we
  use a lookup table to figure out which names are RFC compliant and which
  aren't, then I don't see a way we can leave them enabled.

>     Any code that assumes that (format-time-string "%Z") must generate
>     an RFC822 zone is making an unwarranted assumption and should be
>     fixed.

Fixed how?  Given some arbitrary string, how can Lisp code check
whether it is or isn't compliant?  Non-ASCII characters are easy to
check, but what about time zones that include only ASCII characters?

And anyway, if all bets are off about the strings returned by
current-time-zone, then what exactly is its contract?

If we are going to require Lisp code to check the values and reject
non-compliant ones, we should at the very least provide a utility
function to do that correctly.  Can you suggest such a function?

>            I did a quick scan for such code in Emacs and didn't find
>     any.  Perhaps there's some out-of-Emacs Lisp code that's making
>     the assumption, but if so, that code needs to be fixed because
>     in general it does not work and has never worked.

Jason, can you point out which package(s) needed an RFC822-compliant
time zone name?  In the mail exchange I found, you just say

  [...] since the result of current-time-zone is used for mail
  headers, where non-ASCII characters are not allowed, and the POSIX
  timezone names are expected [...]

Message #46 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 11:46:06 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> Fixed how?  Given some arbitrary string, how can Lisp code check
> whether it is or isn't compliant?

By using %z.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #49 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: eggert <at> cs.ucla.edu, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 12:05:47 +0200

> From: Andreas Schwab <schwab <at> linux-m68k.org>
> Cc: Paul Eggert <eggert <at> cs.ucla.edu>,  9794 <at> debbugs.gnu.org
> Date: Thu, 20 Oct 2011 11:46:06 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > Fixed how?  Given some arbitrary string, how can Lisp code check
> > whether it is or isn't compliant?
> 
> By using %z.

That's _after_ you discover that the string returned by %Z is
non-compliant.  My question was ho to discover that, sorry for being
unclear.

Message #52 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 12:10:52 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> That's _after_ you discover that the string returned by %Z is
> non-compliant.

You know that in advance.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #55 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: eggert <at> cs.ucla.edu, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 12:49:12 +0200

> From: Andreas Schwab <schwab <at> linux-m68k.org>
> Cc: eggert <at> cs.ucla.edu,  9794 <at> debbugs.gnu.org
> Date: Thu, 20 Oct 2011 12:10:52 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > That's _after_ you discover that the string returned by %Z is
> > non-compliant.
> 
> You know that in advance.

No, I don't, because setting TZ=EST or some such in the environment
will produce a compliant string.

Message #58 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 13:22:26 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> No, I don't, because setting TZ=EST or some such in the environment
> will produce a compliant string.

It does not match [+-]NNNN.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #61 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: eggert <at> cs.ucla.edu, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 14:58:51 +0200

> From: Andreas Schwab <schwab <at> linux-m68k.org>
> Cc: eggert <at> cs.ucla.edu,  9794 <at> debbugs.gnu.org
> Date: Thu, 20 Oct 2011 13:22:26 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > No, I don't, because setting TZ=EST or some such in the environment
> > will produce a compliant string.
> 
> It does not match [+-]NNNN.

But the output of %Z or current-time-zone isn't supposed to.  E.g., on
fencepost.gnu.org, I get "EDT".  It's %z that produces [-+]NNNN.

To step back a notch, the original issue was that current-time-zone
can produce strings that are not RFC822 compliant, even on a Posix
system, if the luser sets TZ to some arbitrary string.  Paul says that
any software that uses the output of current-time-zone should be able
to detect this non-compliance and use %z instead.  I'm asking how to
do that.

With a previous "work-around" on Windows, the test was easy: the
output was a blank string, which is definitely not RFC822 compliant.
If we remove that work-around, we can get something like "Pacific
Daylight Time" instead, which is invalid according to RFC822.  I'm
asking how to detect such "invalid" zone names.

Message #64 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 15:06:01 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> It's %z that produces [-+]NNNN.

Which is exactly what is required.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #67 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: eggert <at> cs.ucla.edu, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 15:18:19 +0200

> From: Andreas Schwab <schwab <at> linux-m68k.org>
> Cc: eggert <at> cs.ucla.edu,  9794 <at> debbugs.gnu.org
> Date: Thu, 20 Oct 2011 15:06:01 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > It's %z that produces [-+]NNNN.
> 
> Which is exactly what is required.

But is only tangential to this bug report.

Message #70 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Andreas Schwab <schwab <at> linux-m68k.org>, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 08:23:29 -0700

On 10/20/11 05:58, Eli Zaretskii wrote:
> current-time-zone
> can produce strings that are not RFC822 compliant, even on a Posix
> system, if the luser sets TZ to some arbitrary string.  Paul says that
> any software that uses the output of current-time-zone should be able
> to detect this non-compliance and use %z instead.

Perhaps I wasn't clear enough, as I didn't intend to imply anything
about detecting non-compliance.  Let me try to explain in more detail.

Software cannot simply use %Z and then filter out invalid strings,
because sometimes the %Z output can be a valid RFC822 string and
still be wrong.  For example, in eastern Australia the time zone
name is typically "EST", a valid RFC822 zone, but using "EST" is
wrong because RFC822 says EST is equivalent to 5 hours behind UTC,
whereas Australian EST is equivalent to 10 or 11 hours ahead of UTC.
An inhabitant of Sydney should put "+1000" or "+1100" in outgoing
email, not "EST".  (Australians commonly use the same abbreviation
for both Eastern Standard Time and Eastern Summer Time.)

The simplest way around the problem, which is what Emacs Lisp
code does now, is to follow Andreas's advice use %z.  Any
outside-of-Emacs Lisp code that uses %Z for RFC822 zones should
be fixed to use %z, because %Z does not work and has never worked
in general.

There are more-complicated solutions if you reeeeaaalllly want
the RFC822 symbolic names, but nobody uses them, for good reason.
In the email world, these symbolic names are obsolete, because
of problems like the above.  RFC2822, the successor to RFC822,
describes them in its section 4.3 ("Obsolete Date and Time"),
and says in section 1.3 that obsolete forms such as these
"MUST NOT be generated by creators of conformant messages".
Anybody who uses %Z to generate RFC822 headers is not conforming
to the current email standards, even if they are in the US eastern
time zone and are generating "EDT" and "EST".

Message #73 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: schwab <at> linux-m68k.org, 9794 <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Thu, 20 Oct 2011 18:03:50 +0200

> Date: Thu, 20 Oct 2011 08:23:29 -0700
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> CC: Andreas Schwab <schwab <at> linux-m68k.org>, 9794 <at> debbugs.gnu.org
> 
> The simplest way around the problem, which is what Emacs Lisp
> code does now, is to follow Andreas's advice use %z.  Any
> outside-of-Emacs Lisp code that uses %Z for RFC822 zones should
> be fixed to use %z, because %Z does not work and has never worked
> in general.

OK, thanks for explaining.  I will wait a few days for Jason to agree
or disagree, and if no objections pop up, I will commit the changes to
config.nt.

Message #76 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Jason Rumney <jasonr <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 9794 <at> debbugs.gnu.org,
	drew.adams <at> oracle.com
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Fri, 21 Oct 2011 23:40:20 +0800

Eli Zaretskii <eliz <at> gnu.org> writes:

>>     Any code that assumes that (format-time-string "%Z") must generate
>>     an RFC822 zone is making an unwarranted assumption and should be
>>     fixed.
>
> Fixed how?  Given some arbitrary string, how can Lisp code check
> whether it is or isn't compliant?  Non-ASCII characters are easy to
> check, but what about time zones that include only ASCII characters?

Lisp code that needs RFC822 compliance should just use %z.  Only a small
subset of timezone abbreviations are allowed by RFC822:

zone        =  "UT"  / "GMT"                ; Universal Time
                                            ; North American : UT
            /  "EST" / "EDT"                ;  Eastern:  - 5/ - 4
            /  "CST" / "CDT"                ;  Central:  - 6/ - 5
            /  "MST" / "MDT"                ;  Mountain: - 7/ - 6
            /  "PST" / "PDT"                ;  Pacific:  - 8/ - 7
            /  1ALPHA                       ; Military: Z = UT;
                                            ;  A:-1; (J not used)
                                            ;  M:-12; N:+1; Y:+12
            / ( ("+" / "-") 4DIGIT )        ; Local differential
                                            ;  hours+min. (HHMM)

So Paul is probably correct - we should not worry about RFC / POSIX or
whatever compliance for %Z.

> Jason, can you point out which package(s) needed an RFC822-compliant
> time zone name?  In the mail exchange I found, you just say
>
>   [...] since the result of current-time-zone is used for mail
>   headers, where non-ASCII characters are not allowed, and the POSIX
>   timezone names are expected [...]

A translation of the original report is here:
 http://www.m17n.org/mlarchive/mule-ja/200102/msg00072.html

The original problem leading to that report seems to have been observed
in a beta version of mew:
 http://groups.yahoo.co.jp/group/emacs21-users-ja/message/42

Message #79 received at 9794 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Jason Rumney <jasonr <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Kazu Yamamoto <kazu <at> iij.ad.jp>,
	9794 <at> debbugs.gnu.org, drew.adams <at> oracle.com
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Fri, 21 Oct 2011 10:34:50 -0700

On 10/21/11 08:40, Jason Rumney wrote:

> The original problem leading to that report seems to have been
> observed in a beta version of mew:
> http://groups.yahoo.co.jp/group/emacs21-users-ja/message/42

Thanks, that reinforces the suggestion that we should install the
nt/config.nt patch and then mark this as done.  The current
version of Mew (6.4) uses %z for the time zone, so it should work
fine.  Mew also uses %Z, which it encodes and puts into an RFC822
comment, which should also work.  Mew even contains a workaround
for the Emacs 22 and 23 bug where %Z generates the empty string on
Japanese Windows!  If we install the nt/config.nt patch, that
should make Mew work better now, and once Mew assumes Emacs 24 or
later it can remove that workaround.

I'll CC: this to the Mew maintainer to give him a heads-up
(the the full thread is at <http://debbugs.gnu.org/9794>).

Message #80 received at 9794-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jason Rumney <jasonr <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, 641-done <at> debbugs.gnu.org, drew.adams <at> oracle.com,
	9794-done <at> debbugs.gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Sat, 22 Oct 2011 11:21:57 +0200

> From: Jason Rumney <jasonr <at> gnu.org>
> Cc: Paul Eggert <eggert <at> cs.ucla.edu>,  drew.adams <at> oracle.com,  9794 <at> debbugs.gnu.org
> Date: Fri, 21 Oct 2011 23:40:20 +0800
> 
> So Paul is probably correct - we should not worry about RFC / POSIX or
> whatever compliance for %Z.

Thanks.  So, since everybody agrees, I committed as revision 106162
the changes to nt/config.nt that make current-time-zone and the
strftime's %Z format return a non-empty string on MS-Windows.

Message #81 received at 9794-done <at> debbugs.gnu.org (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Eli Zaretskii'" <eliz <at> gnu.org>, "'Jason Rumney'" <jasonr <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, 641-done <at> debbugs.gnu.org, 9794-done <at> debbugs.gnu.org
Subject: RE: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Sat, 22 Oct 2011 07:27:44 -0700

> Thanks.  So, since everybody agrees,

?

> I committed as revision 106162
> the changes to nt/config.nt that make current-time-zone and the
> strftime's %Z format return a non-empty string on MS-Windows.

Just what is the non-empty string that is returned on Windows?  Is it the
informative name that Windows provides (e.g., "(Pacific Daylight Time)"), or
just the %z numerical offset, or something else again?

What about this?

> I suggest format specifiers that let you alternatively do all 
> of the following for the case of time zone names (i.e. "pretty"
> names, not just %z numbers).
> 
> 1. Use only POSIX-compliant time-zone pretty names, which can 
>    mean "" (empty - no available POSIX name).
> 
> 2. Use any available nonempty time-zone pretty names, with 
>    priority to nonempty POSIX-compliant pretty names.
> 
> 3. Same as #2, but with priority to system-supplied names,
>    even when a corresponding nonempty POSIX name is available.
> 
> 4. Use only nonempty POSIX-compliant pretty names, when 
>    available, and fall back to what %z does in cases where the
>    POSIX name is empty.
> 
> #2 and #3 would also fall back to %z when no nonempty name is 
> available.

Message #82 received at 9794-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: eggert <at> cs.ucla.edu, 641-done <at> debbugs.gnu.org, 9794-done <at> debbugs.gnu.org,
	jasonr <at> gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Sat, 22 Oct 2011 17:24:44 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Cc: <eggert <at> cs.ucla.edu>, <9794-done <at> debbugs.gnu.org>,
>         <641-done <at> debbugs.gnu.org>
> Date: Sat, 22 Oct 2011 07:27:44 -0700
> 
> > Thanks.  So, since everybody agrees,
> 
> ?

!

> Just what is the non-empty string that is returned on Windows?

The one provided by Windows.

> Is it the
> informative name that Windows provides (e.g., "(Pacific Daylight Time)")

Yes.  The strings come from the Registry, you can look them up there
(for the exact key, see the discussions wrt bug #641).

> or just the %z numerical offset

No.  If you want the numerical offset, use %z in format-time-string.

> or something else again?

No.

> What about this?
> 
> > I suggest format specifiers that let you alternatively do all 
> > of the following for the case of time zone names (i.e. "pretty"
> > names, not just %z numbers).
> > 
> > 1. Use only POSIX-compliant time-zone pretty names, which can 
> >    mean "" (empty - no available POSIX name).
> > 
> > 2. Use any available nonempty time-zone pretty names, with 
> >    priority to nonempty POSIX-compliant pretty names.
> > 
> > 3. Same as #2, but with priority to system-supplied names,
> >    even when a corresponding nonempty POSIX name is available.
> > 
> > 4. Use only nonempty POSIX-compliant pretty names, when 
> >    available, and fall back to what %z does in cases where the
> >    POSIX name is empty.
> > 
> > #2 and #3 would also fall back to %z when no nonempty name is 
> > available.

I don't see a need to provide all these features, certainly not as
part of these two bug reports, as we are now back where we were in
Emacs 22.  All the regressions were fixed, some by Paul in revision
106149, and the rest by revision 106162 I committed today.

If you still think you need the other options, please file a wishlist
bug report, because Emacs never had those features.

My take on these options is that #3 is what you have after these last
changes, #4 needs a simple test to decide whether to fall back to %z,
and the rest are impractical, because there's no good way of mapping
an arbitrary (possibly non-ASCII) string returned by Windows to the 1-
or 3-letter zones from the short list allowed by RFC 822.

Message #83 received at 9794-done <at> debbugs.gnu.org (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Eli Zaretskii'" <eliz <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, 641-done <at> debbugs.gnu.org, 9794-done <at> debbugs.gnu.org,
	jasonr <at> gnu.org
Subject: RE: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Sat, 22 Oct 2011 09:17:40 -0700

> > I suggest format specifiers that let you alternatively do all 
> > of the following for the case of time zone names (i.e. "pretty"
> > names, not just %z numbers).
> > 
> > 1. Use only POSIX-compliant time-zone pretty names, which can 
> >    mean "" (empty - no available POSIX name).
> > 
> > 2. Use any available nonempty time-zone pretty names, with 
> >    priority to nonempty POSIX-compliant pretty names.
> > 
> > 3. Same as #2, but with priority to system-supplied names,
> >    even when a corresponding nonempty POSIX name is available.
> > 
> > 4. Use only nonempty POSIX-compliant pretty names, when 
> >    available, and fall back to what %z does in cases where the
> >    POSIX name is empty.
> > 
> > #2 and #3 would also fall back to %z when no nonempty name is 
> > available.
>
>...
> All the regressions were fixed, some by Paul in revision
> 106149, and the rest by revision 106162 I committed today.

Thanks to all for the fixes.

> My take on these options is that #3 is what you have after these last
> changes, #4 needs a simple test to decide whether to fall back to %z,
> and the rest are impractical, because there's no good way of mapping
> an arbitrary (possibly non-ASCII) string returned by Windows to the 1-
> or 3-letter zones from the short list allowed by RFC 822.

#3 was what I requested from the beginning, so I'm fine with that.

You fought #3 vehemently, defending #1 (which you now claim is "impractical"!),
but after 3 years you've apparently come 'round (to #3).  That's progress.  Thx
for the fix.

Message #84 received at 9794-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: eggert <at> cs.ucla.edu, 641-done <at> debbugs.gnu.org, 9794-done <at> debbugs.gnu.org,
	jasonr <at> gnu.org
Subject: Re: bug#9794: 24.0.90; `format-time-string' no good for %Z
Date: Sat, 22 Oct 2011 18:25:26 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Cc: <jasonr <at> gnu.org>, <eggert <at> cs.ucla.edu>, <9794-done <at> debbugs.gnu.org>,
>         <641-done <at> debbugs.gnu.org>
> Date: Sat, 22 Oct 2011 09:17:40 -0700
> 
> You fought #3 vehemently

I just reiterated past decisions, when no one said they were based on
wrong assumptions, and upheld Jason's research on this issue.

> defending #1 (which you now claim is "impractical"!)

That's false, I never defended #1.

> but after 3 years you've apparently come 'round (to #3).

There's new information now, which led to reversal of past decisions.
That's normal.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.