GNU bug report logs - #79668
30.1; json-serialize rejects strings as object keys

Previous Next

To reply to this bug, email your comments to 79668 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Martyanoff <nicolas <at> n16f.net>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.1; json-serialize rejects strings as object keys
Date: Tue, 21 Oct 2025 14:49:18 +0200

This can be easily tested in IELM:

ELISP> (json-serialize '(("foo" . 1) ("bar" . 2)))
*** Eval error ***  Wrong type argument: symbolp, "foo"

JSON object keys are strings, and serialization should support ELisp
strings.

GNU Emacs 30.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.49, cairo version 1.18.4)

-- 
Nicolas Martyanoff
https://n16f.net
nicolas <at> n16f.net

Message #8 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Nicolas Martyanoff <nicolas <at> n16f.net>
Cc: 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Tue, 21 Oct 2025 17:48:55 +0300

> From: Nicolas Martyanoff <nicolas <at> n16f.net>
> Date: Tue, 21 Oct 2025 14:49:18 +0200
> 
> 
> This can be easily tested in IELM:
> 
> ELISP> (json-serialize '(("foo" . 1) ("bar" . 2)))
> *** Eval error ***  Wrong type argument: symbolp, "foo"
> 
> JSON object keys are strings, and serialization should support ELisp
> strings.

The doc string of json-serialize says:

  alist      -- a JSON object.  Keys must be symbols.

So you must use symbols, not strings.  This is not a bug.

Message #11 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Martyanoff <nicolas <at> n16f.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Nicolas Martyanoff <nicolas <at> n16f.net>, 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Tue, 21 Oct 2025 18:38:10 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> The doc string of json-serialize says:
>
>   alist      -- a JSON object.  Keys must be symbols.
>
> So you must use symbols, not strings.  This is not a bug.

Do you mean that it should not be reported as a bug because the code
matches the documentation, or that the fact that it only support symbols
is the right behaviour and that there is a reason not to allow strings?

-- 
Nicolas Martyanoff
https://n16f.net
nicolas <at> n16f.net

Message #14 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Nicolas Martyanoff <nicolas <at> n16f.net>
Cc: 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Tue, 21 Oct 2025 19:44:27 +0300

> From: Nicolas Martyanoff <nicolas <at> n16f.net>
> Cc: Nicolas Martyanoff <nicolas <at> n16f.net>,  79668 <at> debbugs.gnu.org
> Date: Tue, 21 Oct 2025 18:38:10 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > The doc string of json-serialize says:
> >
> >   alist      -- a JSON object.  Keys must be symbols.
> >
> > So you must use symbols, not strings.  This is not a bug.
> 
> Do you mean that it should not be reported as a bug because the code
> matches the documentation, or that the fact that it only support symbols
> is the right behaviour and that there is a reason not to allow strings?

The latter, of course.

Message #17 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Martyanoff <nicolas <at> n16f.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Nicolas Martyanoff <nicolas <at> n16f.net>, 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Tue, 21 Oct 2025 19:51:11 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Do you mean that it should not be reported as a bug because the code
>> matches the documentation, or that the fact that it only support symbols
>> is the right behaviour and that there is a reason not to allow strings?
>
> The latter, of course.

I'm puzzled, in which world does it make any sense _not_ to handle
strings for JSON object keys which _are_ strings?

It's not costing anything, it does not mean that symbol must not work.

If there is a reason behind this decision so be it, but it's hard to
understand what it could be.

-- 
Nicolas Martyanoff
https://n16f.net
nicolas <at> n16f.net

Message #20 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Nicolas Martyanoff <nicolas <at> n16f.net>,
 Mattias Engdegård <mattiase <at> acm.org>
Cc: 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Tue, 21 Oct 2025 22:18:21 +0300

> From: Nicolas Martyanoff <nicolas <at> n16f.net>
> Cc: Nicolas Martyanoff <nicolas <at> n16f.net>,  79668 <at> debbugs.gnu.org
> Date: Tue, 21 Oct 2025 19:51:11 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> Do you mean that it should not be reported as a bug because the code
> >> matches the documentation, or that the fact that it only support symbols
> >> is the right behaviour and that there is a reason not to allow strings?
> >
> > The latter, of course.
> 
> I'm puzzled, in which world does it make any sense _not_ to handle
> strings for JSON object keys which _are_ strings?

AFAIU, JSON objects are maps, so we use the best data types for that
on the Lisp side.  See the node "Parsing JSON" in the ELisp manual.

> It's not costing anything, it does not mean that symbol must not work.
> 
> If there is a reason behind this decision so be it, but it's hard to
> understand what it could be.

I'll leave it to the experts to answer that, but from my perhaps naïve
POV I don't understand why you must use strings there and not symbols.
You can easily make a symbol from string by using 'intern', if that's
a problem.  Of course, you haven't told in what kind of real-life use
case you bumped into this issue, so maybe I'm missing something.

Message #23 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>,
 Nicolas Martyanoff <nicolas <at> n16f.net>, 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Tue, 21 Oct 2025 21:49:03 +0200

21 okt. 2025 kl. 21.18 skrev Eli Zaretskii <eliz <at> gnu.org>:

> From: Nicolas Martyanoff <nicolas <at> n16f.net>

>> I'm puzzled, in which world does it make any sense _not_ to handle
>> strings for JSON object keys which _are_ strings?

The exact reason why the JSON serialiser doesn't accept strings as keys in alists and plists isn't entirely known to me -- I just made it compatible with the previous, libjansson based encoder/decoder. Maybe Philipp remembers?

I'm just speculating here (and Philipp will correct me) but it could be to enforce the same type scheme that is used by the JSON deserialiser, which presumably used symbols for keys because it's felt as more convenient for the user.

On the other hand the old elisp implementation, `json-encode`, accepts any mix of strings, symbols and keywords.

There's one small implementation advantage in insisting on symbols: `json-serialize` gets away with a cheaper uniqueness check by assuming that keys are same iff they are `eq`, which is true for interned symbols. (Yes, you can defeat it by using uninterned symbols.)

It's not clear that this key uniqueness check is very useful but the old implementation had it so I felt I had to include something similar. After all, an alist can be amended by consing a new entry at the front which overrides the old value and you wouldn't want that old value to leak out to another process. However, it really should be the caller's responsibility not to serialise object with duplicate keys, though.

Message #26 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Martyanoff <nicolas <at> n16f.net>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, Eli Zaretskii <eliz <at> gnu.org>,
 Nicolas Martyanoff <nicolas <at> n16f.net>, 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Tue, 21 Oct 2025 22:34:05 +0200

Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:

> I'm just speculating here (and Philipp will correct me) but it could
> be to enforce the same type scheme that is used by the JSON
> deserialiser, which presumably used symbols for keys because it's felt
> as more convenient for the user.

The fact that the JSON parser produces symbols by default is another
problem on its own. It is a well known issue in other languages (e.g.
Ruby, Erlang); you do not want to intern all symbols coming from a
server you do not control.

I was actually thinking about writing a patch to add an option so that
the parser produces string instead (which would not change the defaults,
for those who prefer it that way).

> There's one small implementation advantage in insisting on symbols:
> `json-serialize` gets away with a cheaper uniqueness check by assuming
> that keys are same iff they are `eq`, which is true for interned
> symbols. (Yes, you can defeat it by using uninterned symbols.)
>
> It's not clear that this key uniqueness check is very useful but the
> old implementation had it so I felt I had to include something
> similar. After all, an alist can be amended by consing a new entry at
> the front which overrides the old value and you wouldn't want that old
> value to leak out to another process. However, it really should be the
> caller's responsibility not to serialise object with duplicate keys,
> though.

I really would not want a JSON serializer to enforce object key
uniqueness for me: it is not absurd to think you may end up in a
situation where you absolutely need to send duplicate keys, either for
an a badly designed API (I had to do much worse than that in the past…),
or because you're actually testing that an API handles it correctly.

Now if it happens to be the case, this is the way it is (at least until
someone add an option to disable it), but in that case the user is
responsible for providing either symbols or strings, meaning that he is
responsible for the impact on performances.

In my current situation, I really don't want to intern dozens of symbols
that have no meaning beyond the API protocol, especially with their ugly
underscores in the middle.

-- 
Nicolas Martyanoff
https://n16f.net
nicolas <at> n16f.net

Message #29 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Nicolas Martyanoff <nicolas <at> n16f.net>
Cc: Mattias Engdegård <mattias.engdegard <at> gmail.com>,
 Eli Zaretskii <eliz <at> gnu.org>, Philipp Stephani <p.stephani2 <at> gmail.com>,
 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Wed, 22 Oct 2025 10:37:37 +0200

>>>>> On Tue, 21 Oct 2025 22:34:05 +0200, Nicolas Martyanoff <nicolas <at> n16f.net> said:
    Nicolas> I really would not want a JSON serializer to enforce object key
    Nicolas> uniqueness for me: it is not absurd to think you may end up in a
    Nicolas> situation where you absolutely need to send duplicate keys, either for
    Nicolas> an a badly designed API (I had to do much worse than that in the past…),
    Nicolas> or because you're actually testing that an API handles it correctly.

    Nicolas> Now if it happens to be the case, this is the way it is (at least until
    Nicolas> someone add an option to disable it), but in that case the user is
    Nicolas> responsible for providing either symbols or strings, meaning that he is
    Nicolas> responsible for the impact on performances.

    Nicolas> In my current situation, I really don't want to intern dozens of symbols
    Nicolas> that have no meaning beyond the API protocol, especially with their ugly
    Nicolas> underscores in the middle.

That means weʼd need something more complicated than my initial
thought for fixing this, which is:

diff --git a/src/json.c b/src/json.c
index 30a22dc8038..94203899660 100644
--- a/src/json.c
+++ b/src/json.c
@@ -443,6 +443,8 @@ json_out_object_cons (json_out_t *jo, Lisp_Object obj)
 	  CHECK_CONS (tail);
 	  value = XCAR (tail);
 	}
+      if (STRINGP (key))
+	key = Fintern (key, Qnil);
       key = maybe_remove_pos_from_symbol (key);
       CHECK_TYPE (BARE_SYMBOL_P (key), Qsymbolp, key);
 
Robert
-- 

Message #32 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Nicolas Martyanoff <nicolas <at> n16f.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Philipp Stephani <p.stephani2 <at> gmail.com>,
 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Wed, 22 Oct 2025 10:48:48 +0200

21 okt. 2025 kl. 22.34 skrev Nicolas Martyanoff <nicolas <at> n16f.net>:

> The fact that the JSON parser produces symbols by default is another
> problem on its own.

The JSON parser produces hash tables with string keys by default.

> I really would not want a JSON serializer to enforce object key
> uniqueness for me

Well if you have special requirements that go a bit outside the agreed JSON standard then I'm afraid you are on your own. The uniqueness check has been there since the API was introduced; if we dropped it, there is a risk that code relying on it would misbehave or leak information.

Message #35 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Martyanoff <nicolas <at> n16f.net>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, Eli Zaretskii <eliz <at> gnu.org>,
 Nicolas Martyanoff <nicolas <at> n16f.net>, 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Wed, 22 Oct 2025 11:06:10 +0200

Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:

> 21 okt. 2025 kl. 22.34 skrev Nicolas Martyanoff <nicolas <at> n16f.net>:
>
>> The fact that the JSON parser produces symbols by default is another
>> problem on its own.
>
> The JSON parser produces hash tables with string keys by default.

Hash tables yes. But with alists, it produces symbols:

ELISP> (json-parse-string "{\"a\":1}" :object-type 'alist)
((a . 1))

Which is inconsistent (and a bad idea). Obviously this cannot change
because backward compatibility, but it would not be ridiculous to have
an :object-key-type key argument.

>> I really would not want a JSON serializer to enforce object key
>> uniqueness for me
>
> Well if you have special requirements that go a bit outside the agreed
> JSON standard then I'm afraid you are on your own.
It is not outside the agreed JSON standard.

As per RFC 8259 section 4, "When the names within an object are not
unique, the behavior of software that receives such an object is
unpredictable". The rest of the section is very clear: behaviour is
implementation defined, not invalid.

> The uniqueness check has been there since the API was introduced; if
> we dropped it, there is a risk that code relying on it would misbehave
> or leak information.

This is what options are for. But my points was not that it should be
changed, it was that there was no reason not to accept strings as object
keys for serialization. If it impacts performances because of this
check, that's on the developer who decided to use strings (as it should
be).

-- 
Nicolas Martyanoff
https://n16f.net
nicolas <at> n16f.net

Message #38 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Nicolas Martyanoff <nicolas <at> n16f.net>
Cc: Mattias Engdegård <mattias.engdegard <at> gmail.com>,
 Eli Zaretskii <eliz <at> gnu.org>, Philipp Stephani <p.stephani2 <at> gmail.com>,
 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Wed, 22 Oct 2025 11:29:25 +0200

>>>>> On Wed, 22 Oct 2025 11:06:10 +0200, Nicolas Martyanoff <nicolas <at> n16f.net> said:
    >> The uniqueness check has been there since the API was introduced; if
    >> we dropped it, there is a risk that code relying on it would misbehave
    >> or leak information.

    Nicolas> This is what options are for. But my points was not that it should be
    Nicolas> changed, it was that there was no reason not to accept strings as object
    Nicolas> keys for serialization. If it impacts performances because of this
    Nicolas> check, that's on the developer who decided to use strings (as it should
    Nicolas> be).

The current behaviour is "first symbol wins". I can see people wanting
one of

- first symbol wins
- last symbol wins
- produce duplicate entries

Allowing strings as keys doesnʼt seem like too much of a
burden. Although we could add a user option for that as well.

Robert
-- 

Message #41 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Nicolas Martyanoff <nicolas <at> n16f.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Philipp Stephani <p.stephani2 <at> gmail.com>,
 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Wed, 22 Oct 2025 17:43:05 +0200

22 okt. 2025 kl. 11.06 skrev Nicolas Martyanoff <nicolas <at> n16f.net>:

>> Well if you have special requirements that go a bit outside the agreed
>> JSON standard then I'm afraid you are on your own.
> It is not outside the agreed JSON standard.

You misunderstand. It isn't just RFC 8259 but how it is being interpreted in practice. JSON-based protocols rarely if ever rely on duplicated object keys because JSON codecs cannot be relied on handling them properly.

Now it could be that we are wrong about this and there are in fact significant JSON protocols where duplicated keys are of importance, but then you need to show us some.

Adding an option for disabling uniqueness checks (and accepting string keys in alists, these issues are linked) wouldn't be impossible but as you understand, we can't just do that on your say-so. We'd want to see a clear need for it.

Message #44 received at 79668 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Martyanoff <nicolas <at> n16f.net>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: Philipp Stephani <p.stephani2 <at> gmail.com>, Eli Zaretskii <eliz <at> gnu.org>,
 Nicolas Martyanoff <nicolas <at> n16f.net>, 79668 <at> debbugs.gnu.org
Subject: Re: bug#79668: 30.1; json-serialize rejects strings as object keys
Date: Wed, 22 Oct 2025 17:49:54 +0200

Mattias Engdegård <mattias.engdegard <at> gmail.com> writes:

> 22 okt. 2025 kl. 11.06 skrev Nicolas Martyanoff <nicolas <at> n16f.net>:
>
>>> Well if you have special requirements that go a bit outside the agreed
>>> JSON standard then I'm afraid you are on your own.
>> It is not outside the agreed JSON standard.
>
> You misunderstand. It isn't just RFC 8259 but how it is being interpreted in practice. JSON-based protocols rarely if ever rely on duplicated object keys because JSON codecs cannot be relied on handling them properly.
>
> Now it could be that we are wrong about this and there are in fact significant JSON protocols where duplicated keys are of importance, but then you need to show us some.
>
> Adding an option for disabling uniqueness checks (and accepting string keys in alists, these issues are linked) wouldn't be impossible but as you understand, we can't just do that on your say-so. We'd want to see a clear need for it.

I am just mentioning the topic just to see if Emacs maintainers might be
open to a patch. If not, I'll simply go write my own parser/serializer
to have correct behaviour by default, as I should have done from the
very beginning.

-- 
Nicolas Martyanoff
https://n16f.net
nicolas <at> n16f.net

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.