GNU logs - #69266, boring messages

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#69266: 30.0.50; bibtex-parse-entry misreads escaped \}
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Mon, 19 Feb 2024 09:12:02 +0000
Resent-Message-ID: <handler.69266.B.17083338774221 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 69266 <at> debbugs.gnu.org
From: Ihor Radchenko <yantar92@HIDDEN>
Date: Mon, 19 Feb 2024 09:14:24 +0000
Message-ID: <87cysshl3j.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=185.67.36.65; envelope-from=yantar92@HIDDEN;
 helo=mout01.posteo.de
X-Spam_score_int: -43
X-Spam_score: -4.4
X-Spam_bar: ----
X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001,
 T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.2 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.5 (----)

Consider the following bibtex entry:

@InCollection{Geyer2011,
  title           = {Introduction to Markov Chain Monte  \} Carlo},
  pages           = 45,
}

According to https://www.bibtex.org/SpecialSymbols/, characters that
conflict with Bibtex format description can be \-escaped.

In the above, with point at the beginning of the entry, M-:
(bibtex-parse-entry t), yields

("=type=" . "InCollection")
("=key=" . "Geyer2011")
("title" . "Introduction to Markov Chain Monte  \\")

The escaped \} is treated as closing }, which is incorrect.

Expected: escaping is properly processed.

In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.41, cairo version 1.18.0) of 2024-02-18 built on localhost
Repository revision: 951379a0983ea66b1396d07628bb726f033ea24b
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101011
System Description: Gentoo Linux

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Ihor Radchenko <yantar92@HIDDEN>
Subject: bug#69266: Acknowledgement (30.0.50; bibtex-parse-entry misreads
 escaped \})
Message-ID: <handler.69266.B.17083338774221.ack <at> debbugs.gnu.org>
References: <87cysshl3j.fsf@localhost>
X-Gnu-PR-Message: ack 69266
X-Gnu-PR-Package: emacs
Reply-To: 69266 <at> debbugs.gnu.org
Date: Mon, 19 Feb 2024 09:12:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 69266 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
69266: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D69266
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#69266: 30.0.50; bibtex-parse-entry misreads escaped \}
Resent-From: Arash Esbati <arash@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 23 Feb 2024 12:09:01 +0000
Resent-Message-ID: <handler.69266.B69266.170869010932649 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Ihor Radchenko <yantar92@HIDDEN>
Cc: 69266 <at> debbugs.gnu.org, Roland Winkler <winkler@HIDDEN>
From: Arash Esbati <arash@HIDDEN>
In-Reply-To: <87cysshl3j.fsf@localhost> (Ihor Radchenko's message of "Mon, 19
 Feb 2024 09:14:24 +0000")
References: <87cysshl3j.fsf@localhost>
Date: Fri, 23 Feb 2024 13:07:51 +0100
Message-ID: <m28r3bl6y0.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Ihor Radchenko <yantar92@HIDDEN> writes:

> Consider the following bibtex entry:
>
> @InCollection{Geyer2011,
>   title           = {Introduction to Markov Chain Monte  \} Carlo},
>   pages           = 45,
> }
>
> According to https://www.bibtex.org/SpecialSymbols/, characters that
> conflict with Bibtex format description can be \-escaped.
>
> In the above, with point at the beginning of the entry, M-:
> (bibtex-parse-entry t), yields
>
> ("=type=" . "InCollection")
> ("=key=" . "Geyer2011")
> ("title" . "Introduction to Markov Chain Monte  \\")
>
> The escaped \} is treated as closing }, which is incorrect.
>
> Expected: escaping is properly processed.

I think the issue is that `bibtex-parse-entry' calls
`bibtex-text-in-field-bounds' which calls `bibtex-parse-field-string'
which is defined like this:

--8<---------------cut here---------------start------------->8---
(defun bibtex-parse-field-string ()
  "Parse a BibTeX field string enclosed by braces or quotes.
If a syntactically correct string is found, a pair containing the start and
end position of the field string is returned, nil otherwise.
Do not move point."
  (let ((end-point
         (or (and (eq (following-char) ?\")
                  (save-excursion
                    (with-syntax-table bibtex-quoted-string-syntax-table
                      (forward-sexp 1))
                    (point)))
             (and (eq (following-char) ?\{)
                  (save-excursion
                    (with-syntax-table bibtex-braced-string-syntax-table
                      (forward-sexp 1))
                    (point))))))
    (if end-point
        (cons (point) end-point))))
--8<---------------cut here---------------end--------------->8---

`bibtex-braced-string-syntax-table' is defined as:

--8<---------------cut here---------------start------------->8---
(defconst bibtex-braced-string-syntax-table
  (let ((st (make-syntax-table)))
    (modify-syntax-entry ?\{ "(}" st)
    (modify-syntax-entry ?\} "){" st)
    (modify-syntax-entry ?\[ "." st)
    (modify-syntax-entry ?\] "." st)
    (modify-syntax-entry ?\( "." st)
    (modify-syntax-entry ?\) "." st)
    (modify-syntax-entry ?\\ "." st)
    (modify-syntax-entry ?\" "." st)
    st)
  "Syntax-table to parse matched braces.")
--8<---------------cut here---------------end--------------->8---

where the backslash gets the punctuation class.  Hence, the
(forward-sexp 1) call above goes wrong.  You can eval this in scratch

--8<---------------cut here---------------start------------->8---
(defconst bibtex-braced-string-syntax-table
  (let ((st (make-syntax-table)))
    (modify-syntax-entry ?\{ "(}" st)
    (modify-syntax-entry ?\} "){" st)
    (modify-syntax-entry ?\[ "." st)
    (modify-syntax-entry ?\] "." st)
    (modify-syntax-entry ?\( "." st)
    (modify-syntax-entry ?\) "." st)
    ;; "." changed to "\\"
    (modify-syntax-entry ?\\ "\\" st)
    (modify-syntax-entry ?\" "." st)
    st)
  "Syntax-table to parse matched braces.")
--8<---------------cut here---------------end--------------->8---

and try your test case again -- it should give the expected result.  I
can't tell why the backslash doesn't get the escape-char syntax, it
would make sense IMO, but that's something Roland W. (CC'ed) has to
decide.

Best, Arash

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#69266: 30.0.50; bibtex-parse-entry misreads escaped \}
Resent-From: Roland Winkler <winkler@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 23 Feb 2024 15:27:02 +0000
Resent-Message-ID: <handler.69266.B69266.17087019893652 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Arash Esbati <arash@HIDDEN>
Cc: Ihor Radchenko <yantar92@HIDDEN>, 69266 <at> debbugs.gnu.org
From: Roland Winkler <winkler@HIDDEN>
In-Reply-To: <m28r3bl6y0.fsf@HIDDEN> (Arash Esbati's message of
 "Fri, 23 Feb 2024 13:07:51 +0100")
References: <87cysshl3j.fsf@localhost> <m28r3bl6y0.fsf@HIDDEN>
Date: Fri, 23 Feb 2024 09:25:52 -0600
Message-ID: <87a5nr9p8f.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Ihor Radchenko <yantar92@HIDDEN> writes:
> According to https://www.bibtex.org/SpecialSymbols/, characters that
> conflict with Bibtex format description can be \-escaped.

I believe the above webpage is incorrect.  If I put something like \}
into a BibTeX field, BibTeX complains about unbalanced braces.
This is with BibTeX, Version 0.99d (TeX Live 2022/Debian).
The parsing algoritm used by BibTeX is very simple.  Generally,
BibTeX fields should contain valid LaTeX code.  So something
like
     title = "$\}$tex",

should work with BibTeX, but it gives the same error message
"unbalanced braces".

Emacs bibtex mode follows the capabilities of BibTeX itself.  I believe
it would not make sense to try to be smarter than that if, in the end,
this is not compatible anymore with BibTeX itself.

I guess one could submit here a bug report / feature request for BibTeX.
But BibTeX has been around for many decades with this limited feature set.

Nowadays, there is also biblatex.  It's intended as a successor for
BibTeX.  But it is something I know little to nothing about because in
my world (physics), everyone I know still uses BibTeX.  Essentially,
biblatex entries still use the same format as BibTeX entries.  I do not
know whether biblatex would deal with something like the above in a
smarter way.  

These BibTeX fields should contain valid LaTeX code.  But from (La)TeX's
perspective, in the string "$\}$" the backslash is not an escape
character for what follows, but \} is a macro that's defined inside TeX
math mode.  Also, the meaning of the statement "valid LaTeX code" can be
heavily redefined within (La)TeX.  Take a look at texinfo files that TeX
can digest.  So a smart BibTeX parser would require (La)TeX itself working
in the background.

In practical terms (for many years) I have never experienced this to be
a constraint when working with BibTeX.  So changing / improving this may
have a low priority among BibTeX / biblatex maintainers.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#69266: 30.0.50; bibtex-parse-entry misreads escaped \}
Resent-From: Ihor Radchenko <yantar92@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 24 Feb 2024 12:17:02 +0000
Resent-Message-ID: <handler.69266.B69266.17087769723641 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 69266
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Roland Winkler <winkler@HIDDEN>
Cc: Arash Esbati <arash@HIDDEN>, 69266 <at> debbugs.gnu.org
Received: via spool by 69266-submit <at> debbugs.gnu.org id=B69266.17087769723641
          (code B ref 69266); Sat, 24 Feb 2024 12:17:02 +0000
Received: (at 69266) by debbugs.gnu.org; 24 Feb 2024 12:16:12 +0000
Received: from localhost ([127.0.0.1]:51723 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1rdqwt-0000wd-Vg
	for submit <at> debbugs.gnu.org; Sat, 24 Feb 2024 07:16:12 -0500
Received: from mout02.posteo.de ([185.67.36.66]:33477)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <yantar92@HIDDEN>) id 1rdqwq-0000w4-He
 for 69266 <at> debbugs.gnu.org; Sat, 24 Feb 2024 07:16:10 -0500
Received: from submission (posteo.de [185.67.36.169]) 
 by mout02.posteo.de (Postfix) with ESMTPS id 8E862240104
 for <69266 <at> debbugs.gnu.org>; Sat, 24 Feb 2024 13:15:38 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
 t=1708776938; bh=g+u4hjr58WUGTPX8JXKAxDNUjEB8qMR5MzMqQj7uzBM=;
 h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type:
 From;
 b=GzWIRk4KHlmQgxhVjN/J3RHwbKLDcwRiREHtHaS2FC3Fef9/H2jIu86dcG0diQtZN
 I/kVxJJKIRkh+gWgijLVR8GbNyE0pqK4sPfMsUT7g8TDI4sLZoj0pAZLMtt7Q1ckrf
 rVtiWVshKu11WLL4fX3AaJcELU7RZ0jCDK4lgqS5cLqx3XBukz4q4cz6DYsr+wiMcs
 57YYReozVD5b7GklGJpUxJX/t/zNqslAwLSruyth8LBZzTZu7oHOlQeBgY6mTVXZ+Z
 r+vSy9b3Ji8Q1a8n24WMlaiZ4vDlZZeJP67PRQrgqU30QMHDxa5Vvcs2auOmYAoWbs
 Yu/FDUWxOsHvg==
Received: from customer (localhost [127.0.0.1])
 by submission (posteo.de) with ESMTPSA id 4Thm6w51hdz6tvr;
 Sat, 24 Feb 2024 13:15:36 +0100 (CET)
From: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <87a5nr9p8f.fsf@HIDDEN>
References: <87cysshl3j.fsf@localhost> <m28r3bl6y0.fsf@HIDDEN>
 <87a5nr9p8f.fsf@HIDDEN>
Date: Sat, 24 Feb 2024 12:19:22 +0000
Message-ID: <8734tixdf9.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Roland Winkler <winkler@HIDDEN> writes:

> Ihor Radchenko <yantar92@HIDDEN> writes:
>> According to https://www.bibtex.org/SpecialSymbols/, characters that
>> conflict with Bibtex format description can be \-escaped.
>
> I believe the above webpage is incorrect.  If I put something like \}
> into a BibTeX field, BibTeX complains about unbalanced braces.
> This is with BibTeX, Version 0.99d (TeX Live 2022/Debian).
> The parsing algoritm used by BibTeX is very simple.  Generally,
> BibTeX fields should contain valid LaTeX code.  So something
> like
>      title = "$\}$tex",
>
> should work with BibTeX, but it gives the same error message
> "unbalanced braces".

I am wondering if there exists a full Bibtex format description
somewhere. I can see some hints scattered over documentation in
https://ctan.org/pkg/bibtex, but nothing is complete.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Last modified: Sat, 24 Feb 2024 12:30:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.