GNU bug report logs - #50865
28.0.50; Emoji with emoji modifier in Linux console garbles emacs display

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Aura Kelloniemi <kaura.dev@HIDDEN>; dated Tue, 28 Sep 2021 14:12:02 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 4 Oct 2021 17:06:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 04 13:06:41 2021
Received: from localhost ([127.0.0.1]:38257 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mXRQG-0003hU-P1
	for submit <at> debbugs.gnu.org; Mon, 04 Oct 2021 13:06:40 -0400
Received: from eggs.gnu.org ([209.51.188.92]:59396)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mXRQD-0003hE-MP
 for 50865 <at> debbugs.gnu.org; Mon, 04 Oct 2021 13:06:38 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:43728)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mXRQ7-0003Br-2i; Mon, 04 Oct 2021 13:06:31 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1671
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mXRQ6-00006s-Lp; Mon, 04 Oct 2021 13:06:31 -0400
Date: Mon, 04 Oct 2021 20:06:21 +0300
Message-Id: <83ee9067oy.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Aura Kelloniemi <kaura.dev@HIDDEN>
In-Reply-To: <87k0is68cx.fsf@HIDDEN> (message from Aura Kelloniemi on Mon,
 04 Oct 2021 19:51:58 +0300)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN>
 <87sfxkd976.fsf@HIDDEN> <8335pkeivj.fsf@HIDDEN>
 <831r53d764.fsf@HIDDEN> <87v92d564s.fsf@HIDDEN>
 <837det6ida.fsf@HIDDEN> <83o884694k.fsf@HIDDEN> <87k0is68cx.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Aura Kelloniemi <kaura.dev@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> Date: Mon, 04 Oct 2021 19:51:58 +0300
> 
> I looked at the Linux kernel sources, and found from drivers/tty/vt/vt.c a
> table of characters which will be automatically padded with space when writing
> the glyph to the console. It recognizes two-column characters from Unicode
> 5.0.

And does Emacs work correctly as long as you restrict yourself only to
those double-width characters? or are there still problems?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 4 Oct 2021 16:52:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 04 12:52:16 2021
Received: from localhost ([127.0.0.1]:38228 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mXRCK-0003KR-Fe
	for submit <at> debbugs.gnu.org; Mon, 04 Oct 2021 12:52:16 -0400
Received: from smtp.sange.fi ([185.87.108.151]:48423)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mXRCI-0003KH-6K
 for 50865 <at> debbugs.gnu.org; Mon, 04 Oct 2021 12:52:15 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>)
 id 1mXRCA-00086C-Kw; Mon, 04 Oct 2021 19:52:09 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
In-Reply-To: <83o884694k.fsf@HIDDEN>
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN>
 <87sfxkd976.fsf@HIDDEN> <8335pkeivj.fsf@HIDDEN>
 <831r53d764.fsf@HIDDEN> <87v92d564s.fsf@HIDDEN>
 <837det6ida.fsf@HIDDEN> <83o884694k.fsf@HIDDEN>
Date: Mon, 04 Oct 2021 19:51:58 +0300
Message-ID: <87k0is68cx.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.7 (--)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello,

On 2021-10-04 at 19:35 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
 > Here's a potentially better solution, which uses the display-table
 > feature built into Emacs to display problematic characters as some
 > other characters:

Thank you very much.

I looked at the Linux kernel sources, and found from drivers/tty/vt/vt.c a
table of characters which will be automatically padded with space when writing
the glyph to the console. It recognizes two-column characters from Unicode
5.0. I try to get a patch into Linux itself which would extend support for
automatic padding to include current Unicode multi-column characters.

Whether this succeeds or fails, I'll report it here. This might take some time
though, I'm afraid. It probably would fix this alignment issue for good.

-- 
Aura




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 4 Oct 2021 16:36:06 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 04 12:36:06 2021
Received: from localhost ([127.0.0.1]:38206 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mXQwf-0002sV-Vs
	for submit <at> debbugs.gnu.org; Mon, 04 Oct 2021 12:36:06 -0400
Received: from eggs.gnu.org ([209.51.188.92]:50662)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mXQwa-0002rx-Qy
 for 50865 <at> debbugs.gnu.org; Mon, 04 Oct 2021 12:36:04 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:42518)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mXQwU-0005hp-8r; Mon, 04 Oct 2021 12:35:54 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3755
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mXQwB-00008Z-GI; Mon, 04 Oct 2021 12:35:49 -0400
Date: Mon, 04 Oct 2021 19:35:23 +0300
Message-Id: <83o884694k.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: kaura.dev@HIDDEN
In-Reply-To: <837det6ida.fsf@HIDDEN> (message from Eli Zaretskii on Mon, 04
 Oct 2021 16:15:45 +0300)
Subject: Re: bug#50865: 28.0.50;
 Emoji with emoji modifier in Linux console garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN>
 <87sfxkd976.fsf@HIDDEN> <8335pkeivj.fsf@HIDDEN>
 <831r53d764.fsf@HIDDEN> <87v92d564s.fsf@HIDDEN> <837det6ida.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> Date: Mon, 04 Oct 2021 16:15:45 +0300
> From: Eli Zaretskii <eliz@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> 
> > From: Aura Kelloniemi <kaura.dev@HIDDEN>
> > Cc: 50865 <at> debbugs.gnu.org
> > Date: Mon, 04 Oct 2021 15:25:23 +0300
> > 
> > The solution of modifying char-width-table at least worked very well for me.
> > Of course I am intetrested in the things that will break, if I use it, but
> > most likely those will be smaller annoyances than a garbled display.
> > 
> > I can document this hack on emacs wiki, if nothing else can be done.
> 
> I don't recommend documenting such a "solution", because
> char-width-table affects more than just the display of wide
> characters, it also affects Lisp programs that use string-width and
> similar functions.

Here's a potentially better solution, which uses the display-table
feature built into Emacs to display problematic characters as some
other characters:

  (or standard-display-table
      (setq standard-display-table (make-display-table)))
  (aset standard-display-table
	#x1f64f (vector (make-glyph-code #xFFFD 'escape-glyph)))

This sets Emacs to display the U+01F64F PERSON WITH FOLDED HANDS
character as a diamond with a special face.  If the diamond also
causes trouble, try replacing it with some ASCII character, like '?'.

If this gives good results, you can do the same for any other
problematic character.  The disadvantage is that they all will look
the same on display, and the only way of knowing what is the real
codepoint in the buffer is to go to the character and type "C-u C-x =".





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 4 Oct 2021 13:16:05 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 04 09:16:05 2021
Received: from localhost ([127.0.0.1]:35934 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mXNp7-0002zh-60
	for submit <at> debbugs.gnu.org; Mon, 04 Oct 2021 09:16:05 -0400
Received: from eggs.gnu.org ([209.51.188.92]:53848)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mXNp5-0002yb-G5
 for 50865 <at> debbugs.gnu.org; Mon, 04 Oct 2021 09:16:04 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:53006)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mXNoz-0000Bu-OO; Mon, 04 Oct 2021 09:15:57 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3299
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mXNoz-00033k-83; Mon, 04 Oct 2021 09:15:57 -0400
Date: Mon, 04 Oct 2021 16:15:45 +0300
Message-Id: <837det6ida.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Aura Kelloniemi <kaura.dev@HIDDEN>
In-Reply-To: <87v92d564s.fsf@HIDDEN> (message from Aura Kelloniemi on Mon,
 04 Oct 2021 15:25:23 +0300)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN>
 <87sfxkd976.fsf@HIDDEN> <8335pkeivj.fsf@HIDDEN>
 <831r53d764.fsf@HIDDEN> <87v92d564s.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Aura Kelloniemi <kaura.dev@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> Date: Mon, 04 Oct 2021 15:25:23 +0300
> 
> The solution of modifying char-width-table at least worked very well for me.
> Of course I am intetrested in the things that will break, if I use it, but
> most likely those will be smaller annoyances than a garbled display.
> 
> I can document this hack on emacs wiki, if nothing else can be done.

I don't recommend documenting such a "solution", because
char-width-table affects more than just the display of wide
characters, it also affects Lisp programs that use string-width and
similar functions.

>  > Emacs relies on the terminal to display characters correctly, using 2
>  > columns (with padding by empty space) when the character is
>  > double-width.  If the terminal doesn't live up to these expectations,
>  > the display will become garbled.
> 
> Couldn't emacs add a padding space after every two-column character.

It could, but for which characters and under what conditions?  Who can
produce a full comprehensive description of the problems related to
character width that are inherent in the Linux console?

Anyway, patches to cater to the Linux console will be welcome, if
someone can come up with a method to DTRT.  The problem is that the
changes will need to be on the C level, where we currently simply use
fwrite to output the (UTF-8) encoded text to the device.  Padding
would mean we'd need to write it character by character, or introduce
some logic that looks for the problematic characters and outputs them
specially.

> Emacs already kind of adds a padding space if I type characters one at a time
> (because it repositions the cursor after every command)

No, Emacs doesn't add any padding when it writes to the terminal, at
least AFAICS.  It simply relies on the terminal to produce a 2-column
glyph for a wide character, and positions the cursor accordingly.
Positioning the cursor doesn't in general write a space to the device,
it just outputs a terminfo sequence for cursor addressing.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 4 Oct 2021 12:25:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 04 08:25:33 2021
Received: from localhost ([127.0.0.1]:35850 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mXN2D-0001fk-25
	for submit <at> debbugs.gnu.org; Mon, 04 Oct 2021 08:25:33 -0400
Received: from smtp.sange.fi ([185.87.108.151]:53235)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mXN2A-0001fa-LJ
 for 50865 <at> debbugs.gnu.org; Mon, 04 Oct 2021 08:25:31 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>)
 id 1mXN25-0006wt-N0; Mon, 04 Oct 2021 15:25:28 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
In-Reply-To: <831r53d764.fsf@HIDDEN>
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN>
 <87sfxkd976.fsf@HIDDEN> <8335pkeivj.fsf@HIDDEN>
 <831r53d764.fsf@HIDDEN>
Date: Mon, 04 Oct 2021 15:25:23 +0300
Message-ID: <87v92d564s.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.7 (--)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On 2021-10-02 at 13:58 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
 > > Are you sure they don't? what do the developers say about that?

I am actually a bit confused about the fact that Linux console doesn't seem to
be well known on this list. I am not blaming, just wondering. I would think it
would be very easy for all GNU/Linux users to reproduce this bug any time.

Anyhow, here I provide a proof that Linux really does not understand
two-column characters.

This is again a Bash session in a bare Linu console:

$ echo $'ab\U0001F64Fxy\rabc'
abcxy

This prints letters a and b followed by a wide emoji, followed by letters x
and y. Then it moves the cursor back to the beginning of line with \r and
writes letters a b and c. These should override the first two letters and the
first half of the emoji. This leaves the letters x and y in tact.

But as you see, the c letter here overrides the whole emoji. If the emoji
really was wide, then the output would be

$ echo $'ab\U0001F64Fxy\rabc'
abc xy

Here the space represents the right half of the broken emoji. This later
example is run in a VTE-based terminal that supports Unicode properly.

 > If indeed the Linux console doesn't support double-width characters,
 > or at least enough of them to cause trouble with Emacs display, my
 > suggestion would be to use this setting:

 >   M-x set-terminal-coding-system RET latin-1 RET

As Andreas pointed out, this would not work. Using only ASCII would be a
horrible regression. My native language uses many letters outside the ascii
range. Nowadays even programming becomes difficult without Unicode. This is
not a feasible solution.

 > This will display characters outside the Latin-1 range as \uNNNN or
 > \U0nnnnn (depending on the codepoint), with an underline attribute to
 > make it easier to tell where the character's code ends and the
 > following text begins (in case it begins with a digit).

Linux console does not support the underline attribute. See man 4
console_codes. It talks about simulating the attributes.

 > This should allow you to read the rest of the text without messing up the
 > display. I don't really see a better solution for such problematic
 > terminals.

The solution of modifying char-width-table at least worked very well for me.
Of course I am intetrested in the things that will break, if I use it, but
most likely those will be smaller annoyances than a garbled display.

I can document this hack on emacs wiki, if nothing else can be done.

 > Emacs relies on the terminal to display characters correctly, using 2
 > columns (with padding by empty space) when the character is
 > double-width.  If the terminal doesn't live up to these expectations,
 > the display will become garbled.

Couldn't emacs add a padding space after every two-column character. This
would fix the alignment/garbling issues altogether. This setting could be
controlled by a terminal-local variable and it could be automatically set for
terminals that don't support multi-column characters.

Emacs already kind of adds a padding space if I type characters one at a time
(because it repositions the cursor after every command), but this does not
happen if the text is sent to the terminal in a batch (e.g. when drawing the
contents of a buffer, or when doing a redraw).

-- 
Aura




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 2 Oct 2021 11:56:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 02 07:56:47 2021
Received: from localhost ([127.0.0.1]:58134 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWddH-0008Na-AR
	for submit <at> debbugs.gnu.org; Sat, 02 Oct 2021 07:56:47 -0400
Received: from eggs.gnu.org ([209.51.188.92]:48606)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mWddF-0008NL-In
 for 50865 <at> debbugs.gnu.org; Sat, 02 Oct 2021 07:56:46 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:51044)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mWdd8-0000SO-U2; Sat, 02 Oct 2021 07:56:38 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4126
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mWdd8-0003ob-Do; Sat, 02 Oct 2021 07:56:38 -0400
Date: Sat, 02 Oct 2021 14:56:24 +0300
Message-Id: <83v92fbpxz.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Andreas Schwab <schwab@HIDDEN>
In-Reply-To: <87wnmvwu32.fsf@HIDDEN> (message from Andreas Schwab on Sat,
 02 Oct 2021 13:21:21 +0200)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN>
 <87sfxkd976.fsf@HIDDEN> <8335pkeivj.fsf@HIDDEN>
 <831r53d764.fsf@HIDDEN> <87wnmvwu32.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org, kaura.dev@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Andreas Schwab <schwab@HIDDEN>
> Cc: kaura.dev@HIDDEN,  50865 <at> debbugs.gnu.org
> Date: Sat, 02 Oct 2021 13:21:21 +0200
> 
> On Okt 02 2021, Eli Zaretskii wrote:
> 
> > If indeed the Linux console doesn't support double-width characters,
> > or at least enough of them to cause trouble with Emacs display, my
> > suggestion would be to use this setting:
> >
> >   M-x set-terminal-coding-system RET latin-1 RET
> 
> How can that work?  The terminal's encoding is utf-8, not latin-1.

Then perhaps us-ascii is the best we can do in that case.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 2 Oct 2021 11:21:26 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 02 07:21:26 2021
Received: from localhost ([127.0.0.1]:58068 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWd54-0005Dm-DL
	for submit <at> debbugs.gnu.org; Sat, 02 Oct 2021 07:21:26 -0400
Received: from mail-out.m-online.net ([212.18.0.9]:41066)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <whitebox@HIDDEN>) id 1mWd52-0005Dd-8D
 for 50865 <at> debbugs.gnu.org; Sat, 02 Oct 2021 07:21:25 -0400
Received: from frontend01.mail.m-online.net (unknown [192.168.8.182])
 by mail-out.m-online.net (Postfix) with ESMTP id 4HM4LB5h5cz1qwdP;
 Sat,  2 Oct 2021 13:21:22 +0200 (CEST)
Received: from localhost (dynscan1.mnet-online.de [192.168.6.70])
 by mail.m-online.net (Postfix) with ESMTP id 4HM4LB3vR5z1qqkC;
 Sat,  2 Oct 2021 13:21:22 +0200 (CEST)
X-Virus-Scanned: amavisd-new at mnet-online.de
Received: from mail.mnet-online.de ([192.168.8.182])
 by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new,
 port 10024)
 with ESMTP id RwUs0BHIa3dO; Sat,  2 Oct 2021 13:21:21 +0200 (CEST)
X-Auth-Info: +qQt7/i3BH0i9u6HInQACpBK7VhkLwUfO8Q7UDpKEcq/3VeBKzchV9sme3WFn8e6
Received: from igel.home (ppp-46-244-179-172.dynamic.mnet-online.de
 [46.244.179.172])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by mail.mnet-online.de (Postfix) with ESMTPSA;
 Sat,  2 Oct 2021 13:21:21 +0200 (CEST)
Received: by igel.home (Postfix, from userid 1000)
 id 26B9B2C07C4; Sat,  2 Oct 2021 13:21:21 +0200 (CEST)
From: Andreas Schwab <schwab@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN>
 <87sfxkd976.fsf@HIDDEN> <8335pkeivj.fsf@HIDDEN>
 <831r53d764.fsf@HIDDEN>
X-Yow: What UNIVERSE is this, please??
Date: Sat, 02 Oct 2021 13:21:21 +0200
In-Reply-To: <831r53d764.fsf@HIDDEN> (Eli Zaretskii's message of "Sat, 02 Oct
 2021 13:58:59 +0300")
Message-ID: <87wnmvwu32.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.5 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org, kaura.dev@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.5 (-)

On Okt 02 2021, Eli Zaretskii wrote:

> If indeed the Linux console doesn't support double-width characters,
> or at least enough of them to cause trouble with Emacs display, my
> suggestion would be to use this setting:
>
>   M-x set-terminal-coding-system RET latin-1 RET

How can that work?  The terminal's encoding is utf-8, not latin-1.

Andreas.

-- 
Andreas Schwab, schwab@HIDDEN
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 2 Oct 2021 10:59:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 02 06:59:24 2021
Received: from localhost ([127.0.0.1]:58019 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWcjk-0002Ov-N2
	for submit <at> debbugs.gnu.org; Sat, 02 Oct 2021 06:59:24 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39956)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mWcji-0002Og-Sy
 for 50865 <at> debbugs.gnu.org; Sat, 02 Oct 2021 06:59:23 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:50234)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mWcjc-0003nG-Q3; Sat, 02 Oct 2021 06:59:16 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4596
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mWcjc-0000j1-9P; Sat, 02 Oct 2021 06:59:16 -0400
Date: Sat, 02 Oct 2021 13:58:59 +0300
Message-Id: <831r53d764.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: kaura.dev@HIDDEN
In-Reply-To: <8335pkeivj.fsf@HIDDEN> (message from Eli Zaretskii on Fri, 01
 Oct 2021 20:48:32 +0300)
Subject: Re: bug#50865: 28.0.50;
 Emoji with emoji modifier in Linux console garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN> <87sfxkd976.fsf@HIDDEN>
 <8335pkeivj.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> Resent-From: Eli Zaretskii <eliz@HIDDEN>
> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
> Resent-CC: bug-gnu-emacs@HIDDEN
> Resent-Sender: help-debbugs@HIDDEN
> Date: Fri, 01 Oct 2021 20:48:32 +0300
> From: Eli Zaretskii <eliz@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> 
> > From: Aura Kelloniemi <kaura.dev@HIDDEN>
> > Cc: 50865 <at> debbugs.gnu.org
> > Date: Fri, 01 Oct 2021 19:02:53 +0300
> > 
> > On 2021-10-01 at 18:41 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
> > Hi,
> > 
> >  > Then by default auto-composition-mode should be disabled on that
> >  > console.
> > 
> > Yes, sure. What about Linux's lack of support for 2-column wide characters?
> 
> Are you sure they don't? what do the developers say about that?

If indeed the Linux console doesn't support double-width characters,
or at least enough of them to cause trouble with Emacs display, my
suggestion would be to use this setting:

  M-x set-terminal-coding-system RET latin-1 RET

This will display characters outside the Latin-1 range as \uNNNN or
\U0nnnnn (depending on the codepoint), with an underline attribute to
make it easier to tell where the character's code ends and the
following text begins (in case it begins with a digit).  This should
allow you to read the rest of the text without messing up the display.
I don't really see a better solution for such problematic terminals.
Emacs relies on the terminal to display characters correctly, using 2
columns (with padding by empty space) when the character is
double-width.  If the terminal doesn't live up to these expectations,
the display will become garbled.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 2 Oct 2021 08:55:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 02 04:55:17 2021
Received: from localhost ([127.0.0.1]:57800 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWand-00031S-7N
	for submit <at> debbugs.gnu.org; Sat, 02 Oct 2021 04:55:17 -0400
Received: from eggs.gnu.org ([209.51.188.92]:49350)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mWanX-000312-CI
 for 50865 <at> debbugs.gnu.org; Sat, 02 Oct 2021 04:55:15 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:47936)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mWanP-0004YG-Ni; Sat, 02 Oct 2021 04:55:04 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4953
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mWanP-00025O-2U; Sat, 02 Oct 2021 04:55:03 -0400
Date: Sat, 02 Oct 2021 11:54:48 +0300
Message-Id: <835yufdcx3.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Lars Ingebrigtsen <larsi@HIDDEN>
In-Reply-To: <87wnmvvoaf.fsf@HIDDEN> (message from Lars Ingebrigtsen on Sat, 
 02 Oct 2021 10:11:52 +0200)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN> <87wnmvvoaf.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org, kaura.dev@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Lars Ingebrigtsen <larsi@HIDDEN>
> Cc: Aura Kelloniemi <kaura.dev@HIDDEN>,  50865 <at> debbugs.gnu.org
> Date: Sat, 02 Oct 2021 10:11:52 +0200
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> >> Yes. GNU/Linux distros are configured to set TERM=linux at latest in the
> >> console getty processes.
> >
> > Then by default auto-composition-mode should be disabled on that
> > console.
> 
> I thought this was the case in Emacs 28?
> 
> termp/linux.el has
> 
>   ;; Compositions confuse cursor movement.
>   (setq-default auto-composition-mode "linux")

Yes, that's what I said.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 2 Oct 2021 08:12:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 02 04:12:08 2021
Received: from localhost ([127.0.0.1]:57778 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWa7s-0001v7-Na
	for submit <at> debbugs.gnu.org; Sat, 02 Oct 2021 04:12:08 -0400
Received: from quimby.gnus.org ([95.216.78.240]:34696)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <larsi@HIDDEN>) id 1mWa7q-0001ub-Iy
 for 50865 <at> debbugs.gnu.org; Sat, 02 Oct 2021 04:12:07 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org;
 s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:
 References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:
 Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender:
 Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:
 List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=YbIZnNyaoLIae5bVIWPTdbIQck6M9ItO0BX4pU5Doro=; b=n+D78rlbPMkDGp8P4lQSwHqTxY
 o0xXS6UVd2lcju8vuytWwBYcfa010lBbXrLl3+cpeMW4iKavY/8q1+DItJGNGbLGWVM8aJSw/elG3
 lQYwIjklwoo1BwesT0JHMJ/YQFbY3rRtf25Q3ezpiM93OyH8xkzYjhNx8UnRryHlaH5g=;
Received: from [84.212.220.105] (helo=elva)
 by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.92) (envelope-from <larsi@HIDDEN>)
 id 1mWa7g-00043O-Np; Sat, 02 Oct 2021 10:12:00 +0200
From: Lars Ingebrigtsen <larsi@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN>
Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAMAAABg3Am1AAAABGdBTUEAALGPC/xhBQAAACBj
 SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAM1BMVEVRWy9XYTJmXEXU
 XmfOfoTfmo7nqZjruKTtxbCTbWTKM0n42sX+7NK4sI3+8db96Mz///+hZ1HqAAAAAWJLR0QQlbIN
 LAAAAAd0SU1FB+UKAggLAmkNGeQAAAGASURBVEjHpZWLsoMgDERBRUDA/P/f3vAQgUS0czNTp3X2
 sLtIWyGkkFJ0s6xbmXVdN6X2fdc4Jo4WKB/0eMtW/aoYgOjRJOqPIwPqzSHdseuRgO0jsOgC3Jmm
 kazzHdA5kPWFDCfAVwD11qEenjsMcWTwJ3jGQXMAtjWnjwOkdLYY2wbA5aPeHQTQA5A2My0fL3C8
 OcTNBJSecUag6k2tmjbzzPI5IMuzteZaPgG0QwPIvJlVPgVkirO0y78BuS3MgUqUtr6T8w51W7Et
 DPoG2AgQ8OCM+hkQ8Jz5CUAiGQDO4A0gnbld0hfgeyAfvhZQHFCZ8uEB0DVSP0CA/I3Q/wTOT4Bv
 S9yAmQCei9QBanCgNeD+Edh6iwwQAwbQM2DukDPDk4PiIgHUa4IJcBPpObTjy8t0kRggp7pMRqCN
 VHb1ypLfcQ4F0MDPA7D/AOy/AHsLuGfgIKURcO5TpHzNgPsCWLHYZfkJEAgYGsn7AShHQ+0hYCSd
 HDiTweH669V/MYBJyMqaKjEAAAAldEVYdGRhdGU6Y3JlYXRlADIwMjEtMTAtMDJUMDg6MTE6MDIr
 MDA6MDAao2qtAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDIxLTEwLTAyVDA4OjExOjAyKzAwOjAwa/7S
 EQAAAABJRU5ErkJggg==
X-Now-Playing: Men's _Credit Card Babies_: "Credit Card Babies (L.E.D.
 Version)"
Date: Sat, 02 Oct 2021 10:11:52 +0200
In-Reply-To: <83bl48eoqk.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 01 Oct
 2021 18:41:55 +0300")
Message-ID: <87wnmvvoaf.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 @@CONTACT_ADDRESS@@ for details.
 Content preview: Eli Zaretskii <eliz@HIDDEN> writes: >> Yes. GNU/Linux
 distros
 are configured to set TERM=linux at latest in the >> console getty processes.
 > > Then by default auto-composition-mode should be disabled on that >
 console. Content analysis details:   (-2.9 points, 5.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -1.0 ALL_TRUSTED            Passed through trusted hosts only via SMTP
 -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
 [score: 0.0000]
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org, Aura Kelloniemi <kaura.dev@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Eli Zaretskii <eliz@HIDDEN> writes:

>> Yes. GNU/Linux distros are configured to set TERM=linux at latest in the
>> console getty processes.
>
> Then by default auto-composition-mode should be disabled on that
> console.

I thought this was the case in Emacs 28?

termp/linux.el has

  ;; Compositions confuse cursor movement.
  (setq-default auto-composition-mode "linux")

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 1 Oct 2021 17:49:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 01 13:49:03 2021
Received: from localhost ([127.0.0.1]:57304 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWMed-0004uz-Bj
	for submit <at> debbugs.gnu.org; Fri, 01 Oct 2021 13:49:03 -0400
Received: from eggs.gnu.org ([209.51.188.92]:57512)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mWMeX-0004uP-QU
 for 50865 <at> debbugs.gnu.org; Fri, 01 Oct 2021 13:49:01 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:48160)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mWMeR-0000oA-Iq; Fri, 01 Oct 2021 13:48:51 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4868
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mWMeR-00089R-2j; Fri, 01 Oct 2021 13:48:51 -0400
Date: Fri, 01 Oct 2021 20:48:32 +0300
Message-Id: <8335pkeivj.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Aura Kelloniemi <kaura.dev@HIDDEN>
In-Reply-To: <87sfxkd976.fsf@HIDDEN> (message from Aura Kelloniemi on Fri,
 01 Oct 2021 19:02:53 +0300)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN> <87sfxkd976.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Aura Kelloniemi <kaura.dev@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> Date: Fri, 01 Oct 2021 19:02:53 +0300
> 
> On 2021-10-01 at 18:41 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
> Hi,
> 
>  > Then by default auto-composition-mode should be disabled on that
>  > console.
> 
> Yes, sure. What about Linux's lack of support for 2-column wide characters?

Are you sure they don't? what do the developers say about that?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 1 Oct 2021 16:03:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 01 12:03:03 2021
Received: from localhost ([127.0.0.1]:57181 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWL03-000279-Em
	for submit <at> debbugs.gnu.org; Fri, 01 Oct 2021 12:03:03 -0400
Received: from smtp.sange.fi ([185.87.108.151]:40625)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mWKzz-00026c-D9
 for 50865 <at> debbugs.gnu.org; Fri, 01 Oct 2021 12:03:02 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>)
 id 1mWKzv-0004iG-8p; Fri, 01 Oct 2021 19:02:56 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
In-Reply-To: <83bl48eoqk.fsf@HIDDEN>
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN> <83bl48eoqk.fsf@HIDDEN>
Date: Fri, 01 Oct 2021 19:02:53 +0300
Message-ID: <87sfxkd976.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.7 (--)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On 2021-10-01 at 18:41 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
Hi,

 > Then by default auto-composition-mode should be disabled on that
 > console.

Yes, sure. What about Linux's lack of support for 2-column wide characters?

-- 
Aura




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 1 Oct 2021 15:42:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 01 11:42:35 2021
Received: from localhost ([127.0.0.1]:57162 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWKg8-0001Zm-GL
	for submit <at> debbugs.gnu.org; Fri, 01 Oct 2021 11:42:35 -0400
Received: from eggs.gnu.org ([209.51.188.92]:58376)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mWKg4-0001ZX-NW
 for 50865 <at> debbugs.gnu.org; Fri, 01 Oct 2021 11:42:27 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:43136)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mWKfx-00060M-1D; Fri, 01 Oct 2021 11:42:18 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4983
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mWKfv-0006v6-8s; Fri, 01 Oct 2021 11:42:16 -0400
Date: Fri, 01 Oct 2021 18:41:55 +0300
Message-Id: <83bl48eoqk.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Aura Kelloniemi <kaura.dev@HIDDEN>
In-Reply-To: <87v92gddcg.fsf@HIDDEN> (message from Aura Kelloniemi on Fri,
 01 Oct 2021 17:33:19 +0300)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
 <87v92gddcg.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Aura Kelloniemi <kaura.dev@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> Date: Fri, 01 Oct 2021 17:33:19 +0300
> 
>  > Does term/linux.el get loaded when you run Emacs on that terminal?
> 
> Yes. GNU/Linux distros are configured to set TERM=linux at latest in the
> console getty processes.

Then by default auto-composition-mode should be disabled on that
console.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 1 Oct 2021 14:33:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 01 10:33:27 2021
Received: from localhost ([127.0.0.1]:57051 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWJbL-0005yO-AA
	for submit <at> debbugs.gnu.org; Fri, 01 Oct 2021 10:33:27 -0400
Received: from smtp.sange.fi ([185.87.108.151]:34613)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mWJbJ-0005yE-E6
 for 50865 <at> debbugs.gnu.org; Fri, 01 Oct 2021 10:33:26 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>)
 id 1mWJbF-0002Ui-Je; Fri, 01 Oct 2021 17:33:23 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
In-Reply-To: <83czooeulj.fsf@HIDDEN>
References: <87y27cdglm.fsf@HIDDEN> <83czooeulj.fsf@HIDDEN>
Date: Fri, 01 Oct 2021 17:33:19 +0300
Message-ID: <87v92gddcg.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.7 (--)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On 2021-10-01 at 16:35 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
 > > From: Aura Kelloniemi <kaura.dev@HIDDEN>
 > > Cc: 50865 <at> debbugs.gnu.org
 > > Date: Fri, 01 Oct 2021 16:23:01 +0300
[--]
 > > Here is a sample Bash session for demonstration:
 > > $ echo $'i\u300'
 > > i=E2=97=88
 > > $ echo $'\uEC'
 > > =C3=AC

 > Ouch!  What a terrible misfeature!

Yes, but of course unintentional.

Variation selectors work in a similar way. <HEAVY BLACK HEART> <VARIATION
SELECTOR-16> prints two diamonds which is two 1-column wide characters.

There has been a recent change in emacs after which it no more prints the
variation selectors. This is probably related to the addition of
glyphless-char-display-control.

Anyway, with Linux it is safest to think the console as an old text-mode VGA
display which has been extended to support more codepoitns.

 > Does term/linux.el get loaded when you run Emacs on that terminal?

Yes. GNU/Linux distros are configured to set TERM=3Dlinux at latest in the
console getty processes.

--=20
Aura




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 1 Oct 2021 13:35:50 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 01 09:35:50 2021
Received: from localhost ([127.0.0.1]:54997 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWIha-0003pY-KS
	for submit <at> debbugs.gnu.org; Fri, 01 Oct 2021 09:35:50 -0400
Received: from eggs.gnu.org ([209.51.188.92]:53122)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mWIhX-0003pM-Vk
 for 50865 <at> debbugs.gnu.org; Fri, 01 Oct 2021 09:35:48 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:35886)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mWIhR-0007jm-WE; Fri, 01 Oct 2021 09:35:42 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1239
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mWIhO-0003Fo-2n; Fri, 01 Oct 2021 09:35:41 -0400
Date: Fri, 01 Oct 2021 16:35:20 +0300
Message-Id: <83czooeulj.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Aura Kelloniemi <kaura.dev@HIDDEN>
In-Reply-To: <87y27cdglm.fsf@HIDDEN> (message from Aura Kelloniemi on Fri,
 01 Oct 2021 16:23:01 +0300)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87y27cdglm.fsf@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Aura Kelloniemi <kaura.dev@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> Date: Fri, 01 Oct 2021 16:23:01 +0300
> 
>  > > I noticed, that Linux console does not understand most of the zero-width
>  > > characters either.
> 
>  > It doesn't need to: Emacs displays those characters on a TTY as
>  > spaces.
> 
> Can this be configured ā€“ i.e. can I change the space to something else to ease
> debugging?

Yes, see glyphless-char-display-control.

>  > That's not exception, that's the rule, actually, for true zero-width
>  > characters, not for accents.  Accents exist to combine with preceding
>  > base character, and what you seem to describe means the Linux console
>  > is unable to do even Latin accents?
> 
> Here is a sample Bash session for demonstration:
> $ echo $'i\u300'
> iā—ˆ
> $ echo $'\uEC'
> Ƭ

Ouch!  What a terrible misfeature!

> If I set auto-composition-mode to nil, then Emacs does not print anything (not
> even the space) when I insert a combining character. If I then move the point
> over the invisible combining character, the point moves, but the screen cursor
> does not. This is a very confusing behaviour.

I think it's expected for accents.

> When running in the Linux console emacs's term/linux.el sets
> auto-composition-mode to a special value of "linux". I don't know what this
> means.

That is explained in the doc string of auto-composition-mode.

Does term/linux.el get loaded when you run Emacs on that terminal?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 1 Oct 2021 13:23:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 01 09:23:10 2021
Received: from localhost ([127.0.0.1]:54969 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mWIVJ-0003WT-SX
	for submit <at> debbugs.gnu.org; Fri, 01 Oct 2021 09:23:10 -0400
Received: from smtp.sange.fi ([185.87.108.151]:37003)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mWIVH-0003WK-OG
 for 50865 <at> debbugs.gnu.org; Fri, 01 Oct 2021 09:23:08 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>)
 id 1mWIVD-0004BZ-Rx; Fri, 01 Oct 2021 16:23:05 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
In-Reply-To: <83h7e3iljt.fsf@HIDDEN>
Date: Fri, 01 Oct 2021 16:23:01 +0300
Message-ID: <87y27cdglm.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.7 (--)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On 2021-09-29 at 16:00 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
 > > From: Aura Kelloniemi <kaura.dev@HIDDEN>
 > > Cc: 50865 <at> debbugs.gnu.org
 > > Date: Tue, 28 Sep 2021 23:32:53 +0300
 > >=20
 > > On 2021-09-28 at 21:35 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
 > >  > That should not be that way.  Some characters are double-width, and
 > >  > should take up 2 columns on display.
 > >=20
 > > I noticed, that Linux console does not understand most of the zero-wid=
th
 > > characters either.

 > It doesn't need to: Emacs displays those characters on a TTY as
 > spaces.

Can this be configured =E2=80=93 i.e. can I change the space to something e=
lse to ease
debugging?

 > > It happily prints most of the code points in the list of
 > > zero-width characters. Of course they are printed just as diamonds, be=
cause
 > > Linux cannot store enough glyphs in its 512-glyph font space, but anyw=
ay it
 > > prints a diamond for such characters as <COMBINING GRAVE ACCENT>.

 > COMBINING GRAVE ACCENT (or any other combining codepoint) is not a
 > good example of zero-width characters.

On modern terminal emulators this certainly holds, but Linux is not a modern
terminal emulator and does not support combining characters. It just prints=
 a
diamond for all codepoitns which don't have an associated glyph in the font
(or the kernel knows them to be zero-wide, and this information is out of
date).

 > Try "C-x 8 RET 200c RET"
 > instead.

 > Or FEFF
 > or 1D173 or E007f or 1BCA0.

They print just a single space within emacs. If I print them with echo,
they print a diamond.

 > > The character range \y200B-\u200F seems to be an exception here. When =
I try to
 > > print one of these characters on a Linxu VT, it really prints nothing.

 > That's not exception, that's the rule, actually, for true zero-width
 > characters, not for accents.  Accents exist to combine with preceding
 > base character, and what you seem to describe means the Linux console
 > is unable to do even Latin accents?

Here is a sample Bash session for demonstration:
$ echo $'i\u300'
i=E2=97=88
$ echo $'\uEC'
=C3=AC

 > > When I insert zero-width characters in Emacs, the diamonds representin=
g the
 > > characters are printed interspersed by the padding spaces added by ema=
cs. The
 > > cursor is left behind the extending line of characters as a type, beca=
use
 > > Emacs thinks, that the zero-width characters really do not print anyth=
ing,
 > > even though they do.

 > Is this with or without auto-composition-mode?

Ok, this was with auto-composition-mode set to t. And it only happens with
combining characters. Other zero-wide characters print the single space, as
should be.

If I set auto-composition-mode to nil, then Emacs does not print anything (=
not
even the space) when I insert a combining character. If I then move the poi=
nt
over the invisible combining character, the point moves, but the screen cur=
sor
does not. This is a very confusing behaviour.

Non-combining zero-wide characters print the space (as you said), and there
are no cursor movement issues.

When running in the Linux console emacs's term/linux.el sets
auto-composition-mode to a special value of "linux". I don't know what this
means.

--=20
Aura




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 29 Sep 2021 13:00:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 29 09:00:50 2021
Received: from localhost ([127.0.0.1]:48049 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mVZCc-0001Vq-LM
	for submit <at> debbugs.gnu.org; Wed, 29 Sep 2021 09:00:50 -0400
Received: from eggs.gnu.org ([209.51.188.92]:60342)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mVZCZ-0001Vb-P2
 for 50865 <at> debbugs.gnu.org; Wed, 29 Sep 2021 09:00:50 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:55558)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mVZCT-0002qE-Pm; Wed, 29 Sep 2021 09:00:41 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4847
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mVZC8-0006wo-HP; Wed, 29 Sep 2021 09:00:41 -0400
Date: Wed, 29 Sep 2021 16:00:22 +0300
Message-Id: <83h7e3iljt.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Aura Kelloniemi <kaura.dev@HIDDEN>
In-Reply-To: <87a6jw8mq2.fsf@HIDDEN> (message from Aura Kelloniemi on Tue,
 28 Sep 2021 23:32:53 +0300)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87o88chjsl.fsf@HIDDEN> <83a6jwk7cb.fsf@HIDDEN>
 <87ilyk8wu2.fsf@HIDDEN> <835yukk44q.fsf@HIDDEN>
 <87fsto8unj.fsf@HIDDEN> <8335pok0pv.fsf@HIDDEN> <87a6jw8mq2.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Aura Kelloniemi <kaura.dev@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> Date: Tue, 28 Sep 2021 23:32:53 +0300
> 
> On 2021-09-28 at 21:35 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
>  > That should not be that way.  Some characters are double-width, and
>  > should take up 2 columns on display.
> 
> I noticed, that Linux console does not understand most of the zero-width
> characters either.

It doesn't need to: Emacs displays those characters on a TTY as
spaces.

> It happily prints most of the code points in the list of
> zero-width characters. Of course they are printed just as diamonds, because
> Linux cannot store enough glyphs in its 512-glyph font space, but anyway it
> prints a diamond for such characters as <COMBINING GRAVE ACCENT>.

COMBINING GRAVE ACCENT (or any other combining codepoint) is not a
good example of zero-width characters.  Try "C-x 8 RET 200c RET"
instead.  Or FEFF or 1D173 or E007f or 1BCA0.

> The character range \y200B-\u200F seems to be an exception here. When I try to
> print one of these characters on a Linxu VT, it really prints nothing.

That's not exception, that's the rule, actually, for true zero-width
characters, not for accents.  Accents exist to combine with preceding
base character, and what you seem to describe means the Linux console
is unable to do even Latin accents?

> When I insert zero-width characters in Emacs, the diamonds representing the
> characters are printed interspersed by the padding spaces added by emacs. The
> cursor is left behind the extending line of characters as a type, because
> Emacs thinks, that the zero-width characters really do not print anything,
> even though they do.

Is this with or without auto-composition-mode?

> I believe that the one viable solution is to make char-width-table a terminal
> local variable, so that there can be a simplified version for terminals that
> don't understand Unicode correctly.

That would affect much more than display, because Emacs consults that
table for other purposes.  We need something limited to display alone.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 28 Sep 2021 20:33:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 28 16:33:02 2021
Received: from localhost ([127.0.0.1]:47130 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mVJmg-0007mQ-B4
	for submit <at> debbugs.gnu.org; Tue, 28 Sep 2021 16:33:02 -0400
Received: from smtp.sange.fi ([185.87.108.151]:60339)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mVJmd-0007m7-FK
 for 50865 <at> debbugs.gnu.org; Tue, 28 Sep 2021 16:33:00 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>)
 id 1mVJmZ-000178-R6; Tue, 28 Sep 2021 23:32:57 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
In-Reply-To: <8335pok0pv.fsf@HIDDEN>
References: <87o88chjsl.fsf@HIDDEN> <83a6jwk7cb.fsf@HIDDEN>
 <87ilyk8wu2.fsf@HIDDEN> <835yukk44q.fsf@HIDDEN>
 <87fsto8unj.fsf@HIDDEN> <8335pok0pv.fsf@HIDDEN>
Date: Tue, 28 Sep 2021 23:32:53 +0300
Message-ID: <87a6jw8mq2.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.7 (--)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On 2021-09-28 at 21:35 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
 > That should not be that way.  Some characters are double-width, and
 > should take up 2 columns on display.

I noticed, that Linux console does not understand most of the zero-width
characters either. It happily prints most of the code points in the list of
zero-width characters. Of course they are printed just as diamonds, because
Linux cannot store enough glyphs in its 512-glyph font space, but anyway it
prints a diamond for such characters as <COMBINING GRAVE ACCENT>.

The character range \y200B-\u200F seems to be an exception here. When I try to
print one of these characters on a Linxu VT, it really prints nothing.

When I insert zero-width characters in Emacs, the diamonds representing the
characters are printed interspersed by the padding spaces added by emacs. The
cursor is left behind the extending line of characters as a type, because
Emacs thinks, that the zero-width characters really do not print anything,
even though they do.

I believe that the one viable solution is to make char-width-table a terminal
local variable, so that there can be a simplified version for terminals that
don't understand Unicode correctly.

-- 
Aura




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 28 Sep 2021 19:21:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 28 15:21:14 2021
Received: from localhost ([127.0.0.1]:47040 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mVIf5-0005zY-QP
	for submit <at> debbugs.gnu.org; Tue, 28 Sep 2021 15:21:13 -0400
Received: from smtp.sange.fi ([185.87.108.151]:57045)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mVIf3-0005zP-EW
 for 50865 <at> debbugs.gnu.org; Tue, 28 Sep 2021 15:21:07 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>)
 id 1mVIez-00032l-1i; Tue, 28 Sep 2021 22:21:03 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
In-Reply-To: <8335pok0pv.fsf@HIDDEN>
References: <87o88chjsl.fsf@HIDDEN> <83a6jwk7cb.fsf@HIDDEN>
 <87ilyk8wu2.fsf@HIDDEN> <835yukk44q.fsf@HIDDEN>
 <87fsto8unj.fsf@HIDDEN> <8335pok0pv.fsf@HIDDEN>
Date: Tue, 28 Sep 2021 22:20:58 +0300
Message-ID: <87czos8q1x.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.7 (--)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

On 2021-09-28 at 21:35 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
 > > From: Aura Kelloniemi <kaura.dev@HIDDEN>
 > > Cc: 50865 <at> debbugs.gnu.org
 > > Date: Tue, 28 Sep 2021 20:41:36 +0300
 > >=20
 > > utf-8-unix
 > >=20
 > > And the Linux console is properly in Unicode mode. (Which means Unicod=
e as it
 > > was understood in 2002 or so.)

 > Which could be too long ago?  Are you saying that the Linux terminal
 > doesn't understand Unicode beyond the year 2002?  That could explain a
 > lot.

More or less. The code point range has been extended to allow for code poin=
ts
outside of the 16-bit range. Otherwise I haven't seen much development. The
console font can contain at most 512 glyphs, which is a huge limitation.

 > So it sounds like your terminal cannot handle double-width
 > characters.

Yes, that is correct. Sorry, I probably should have said this earlier. The
Linux console does not support extended-width characters.

This is not a problem just with Emacs, but every single program/library that
supports double-width characters, e.g. readline.

 > Those "space characters" you see are padding glyphs
 > output by Emacs when it displays a double-width character.  On my
 > terminal emulator, the results are satisfactory, and I see no
 > artifacts.  Are you sure the spaces you saw aren't just visual
 > surprises, and otherwise don't present any real problems?  If they do
 > present real problems, can you describe them in more detail, including
 > the exact sequence of characters you typed for that?

Cursor movement gets messed up. If I type:

<PERSON WITH FOLDED HANDS> <DIGIT ONE>

the display is correct:

<PERSON WITH FOLDED HANDS> <DIGIT ONE> _cursor_

If I now do C-l, the display looks like:

<PERSON WITH FOLDED HANDS> <DIGIT ONE> <SPACE> _cursor_

If I type BackSpace to delete the digit 1, only the space disappears, so the
display looks like:

<PERSON WITH FOLDED HANDS> <DIGIT ONE> _cursor_

But really the buffer contains only the emoji.

If I now press BackSpace again, both the emoji and the digit disappear, and
the line becomes empty.

If I use arrow keys to move around in the buffer, the cursor is moved by two
columns every time the point moves over a double-width character. But becau=
se
the terminal does not show the double-width character as double-wide, the
cursor placement is off by the number of double-width characters on the left
side of the point.

 > > Probably there is no other documentation than the kernel source =E2=80=
=93 or
 > > at least the documentation is outdated. Linux console is kind of
 > > deprecated, but it is still sort of maintained as there is no
 > > alternative being developed.

 > Perhaps you should take this up with the developers, then.  But I'd
 > like to understand better what display problems you saw originally,
 > because all I read there now is that you saw those extra spaces.

I could, but many have done this already. The situation is such that it wou=
ld
be easier to rewrite the whole console driver from scratch than to try to
extend it with more features. There does not seem to be interest for doing =
it.

Mostly people use Linux console for emergency maintainance, or then they are
blind and cannot use the graphical desktop, because the accessibility
technology does not support this. I happen to use Linux VT's for both of th=
ese
reasons.

I took a look at Emacs's term/linux.el.gz. It sets auto-composition-mode to
"linux". I don't know what this special value does.

--=20
Aura




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 28 Sep 2021 18:35:52 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 28 14:35:52 2021
Received: from localhost ([127.0.0.1]:46966 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mVHxH-0004ni-Rv
	for submit <at> debbugs.gnu.org; Tue, 28 Sep 2021 14:35:52 -0400
Received: from eggs.gnu.org ([209.51.188.92]:56626)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mVHxE-0004nS-9U
 for 50865 <at> debbugs.gnu.org; Tue, 28 Sep 2021 14:35:50 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:54414)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mVHx5-00056m-2z; Tue, 28 Sep 2021 14:35:39 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1025
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mVHwg-00040t-Bs; Tue, 28 Sep 2021 14:35:33 -0400
Date: Tue, 28 Sep 2021 21:35:08 +0300
Message-Id: <8335pok0pv.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Aura Kelloniemi <kaura.dev@HIDDEN>
In-Reply-To: <87fsto8unj.fsf@HIDDEN> (message from Aura Kelloniemi on Tue,
 28 Sep 2021 20:41:36 +0300)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87o88chjsl.fsf@HIDDEN> <83a6jwk7cb.fsf@HIDDEN>
 <87ilyk8wu2.fsf@HIDDEN> <835yukk44q.fsf@HIDDEN> <87fsto8unj.fsf@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Aura Kelloniemi <kaura.dev@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> Date: Tue, 28 Sep 2021 20:41:36 +0300
> 
> utf-8-unix
> 
> And the Linux console is properly in Unicode mode. (Which means Unicode as it
> was understood in 2002 or so.)

Which could be too long ago?  Are you saying that the Linux terminal
doesn't understand Unicode beyond the year 2002?  That could explain a
lot.

>  > Does typing the below solve the problem?
> 
>  >   M-: (set-char-table-range char-width-table '(#x1f600 . #x1f64f) 1) RET
> 
> No, not alone, but when I tried
> 
> (set-char-table-range char-width-table '(#x1f300 . #x1f64f) 1)
> 
> the problem is gone. Using the same trick to force some other characters (e.g.
> TIMER CLOCK) to width 1, I was able to get rid of the other problem that I
> mentioned, but could not track.

So it sounds like your terminal cannot handle double-width
characters.  Those "space characters" you see are padding glyphs
output by Emacs when it displays a double-width character.  On my
terminal emulator, the results are satisfactory, and I see no
artifacts.  Are you sure the spaces you saw aren't just visual
surprises, and otherwise don't present any real problems?  If they do
present real problems, can you describe them in more detail, including
the exact sequence of characters you typed for that?

> Now the question is: should this be added to the linux terminal setup in
> Emacs, and for what character range.

No.  That is just a kludgey workaround for some problem I don't yet
understand well enough.

> If I am right, all Unicode code points above 0x20 print just one
> character to the terminal, except if they don't print anything.

That should not be that way.  Some characters are double-width, and
should take up 2 columns on display.

> Probably there is no other documentation than the kernel source ā€“ or
> at least the documentation is outdated. Linux console is kind of
> deprecated, but it is still sort of maintained as there is no
> alternative being developed.

Perhaps you should take this up with the developers, then.  But I'd
like to understand better what display problems you saw originally,
because all I read there now is that you saw those extra spaces.

> Is there a way to get a list of code points that Emacs thinks have a width of
> something else than 1?

You can use map-char-table to display all characters that have width
of 2 columns.  Or you can look in lisp/international/characters.el,
around line 1250, where double-width characters are listed.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 28 Sep 2021 17:41:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 28 13:41:47 2021
Received: from localhost ([127.0.0.1]:46945 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mVH6w-0001MI-Qb
	for submit <at> debbugs.gnu.org; Tue, 28 Sep 2021 13:41:47 -0400
Received: from smtp.sange.fi ([185.87.108.151]:58027)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mVH6t-0001M8-6z
 for 50865 <at> debbugs.gnu.org; Tue, 28 Sep 2021 13:41:44 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>)
 id 1mVH6o-0000H1-AJ; Tue, 28 Sep 2021 20:41:40 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
In-Reply-To: <835yukk44q.fsf@HIDDEN>
References: <87o88chjsl.fsf@HIDDEN> <83a6jwk7cb.fsf@HIDDEN>
 <87ilyk8wu2.fsf@HIDDEN> <835yukk44q.fsf@HIDDEN>
Date: Tue, 28 Sep 2021 20:41:36 +0300
Message-ID: <87fsto8unj.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.7 (--)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On 2021-09-28 at 20:21 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
 > What is your terminal's encoding? what does the following show in the
 > echo-area?

utf-8-unix

And the Linux console is properly in Unicode mode. (Which means Unicode as =
it
was understood in 2002 or so.)

 > Does typing the below solve the problem?

 >   M-: (set-char-table-range char-width-table '(#x1f600 . #x1f64f) 1) RET

No, not alone, but when I tried

(set-char-table-range char-width-table '(#x1f300 . #x1f64f) 1)

the problem is gone. Using the same trick to force some other characters (e=
.g.
TIMER CLOCK) to width 1, I was able to get rid of the other problem that I
mentioned, but could not track.

Now the question is: should this be added to the linux terminal setup in
Emacs, and for what character range. If I am right, all Unicode code points
above 0x20 print just one character to the terminal, except if they don't
print anything. Probably there is no other documentation than the kernel
source =E2=80=93 or at least the documentation is outdated. Linux console i=
s kind of
deprecated, but it is still sort of maintained as there is no alternative
being developed.

Is there a way to get a list of code points that Emacs thinks have a width =
of
something else than 1?

Thank you for helping me!

--=20
Aura




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 28 Sep 2021 17:21:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 28 13:21:32 2021
Received: from localhost ([127.0.0.1]:46893 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mVGnL-0000p7-RE
	for submit <at> debbugs.gnu.org; Tue, 28 Sep 2021 13:21:32 -0400
Received: from eggs.gnu.org ([209.51.188.92]:43050)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mVGnK-0000os-FI
 for 50865 <at> debbugs.gnu.org; Tue, 28 Sep 2021 13:21:30 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:52772)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mVGnE-00020T-Fx; Tue, 28 Sep 2021 13:21:24 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4500
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mVGnD-0006F0-56; Tue, 28 Sep 2021 13:21:24 -0400
Date: Tue, 28 Sep 2021 20:21:25 +0300
Message-Id: <835yukk44q.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Aura Kelloniemi <kaura.dev@HIDDEN>
In-Reply-To: <87ilyk8wu2.fsf@HIDDEN> (message from Aura Kelloniemi on Tue,
 28 Sep 2021 19:54:29 +0300)
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
References: <87o88chjsl.fsf@HIDDEN> <83a6jwk7cb.fsf@HIDDEN>
 <87ilyk8wu2.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Aura Kelloniemi <kaura.dev@HIDDEN>
> Cc: 50865 <at> debbugs.gnu.org
> Date: Tue, 28 Sep 2021 19:54:29 +0300
> 
>  > If your terminal doesn't understand character composition, the best
>  > solution for you is to turn off auto-composition-mode when using Emacs
>  > on that terminal.  Please try that and tell if that resolves the issue
>  > for you: type "M-x auto-composition-mode RET" and make sure Emacs says
>  > that the mode is disabled.
> 
> Unfortunately it does not seem to do anything. The problem stays exactly as I
> described.

What is your terminal's encoding? what does the following show in the
echo-area?

  M-: (terminal-coding-system) RET

> For some reason, when I re-enable auto-composition-mode, the output appears
> correct (no extra spaces at the end of line) for a while, but if I start to
> add characters to the line containing the emojis, the display gets garbled
> again.

Does typing the below solve the problem?

  M-: (set-char-table-range char-width-table '(#x1f600 . #x1f64f) 1) RET

(You will need to redraw the display after evaluating it, e.g. with
"C-l" or "M-x redraw-display RET".)




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 28 Sep 2021 16:54:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 28 12:54:37 2021
Received: from localhost ([127.0.0.1]:46855 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mVGNJ-00008H-FB
	for submit <at> debbugs.gnu.org; Tue, 28 Sep 2021 12:54:37 -0400
Received: from smtp.sange.fi ([185.87.108.151]:45033)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mVGNH-000088-Hq
 for 50865 <at> debbugs.gnu.org; Tue, 28 Sep 2021 12:54:36 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>)
 id 1mVGND-00036L-HK; Tue, 28 Sep 2021 19:54:33 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#50865: 28.0.50; Emoji with emoji modifier in Linux console
 garbles emacs display
In-Reply-To: <83a6jwk7cb.fsf@HIDDEN>
References: <87o88chjsl.fsf@HIDDEN> <83a6jwk7cb.fsf@HIDDEN>
Date: Tue, 28 Sep 2021 19:54:29 +0300
Message-ID: <87ilyk8wu2.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.7 (--)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On 2021-09-28 at 19:12 +0300, Eli Zaretskii <eliz@HIDDEN> wrote:
 > If your terminal doesn't understand character composition, the best
 > solution for you is to turn off auto-composition-mode when using Emacs
 > on that terminal.  Please try that and tell if that resolves the issue
 > for you: type "M-x auto-composition-mode RET" and make sure Emacs says
 > that the mode is disabled.

Unfortunately it does not seem to do anything. The problem stays exactly as I
described.

Running again with emacs -Q, I tried both turning off the mode locally and
globally. I got the message that the mode was disabled.

For some reason, when I re-enable auto-composition-mode, the output appears
correct (no extra spaces at the end of line) for a while, but if I start to
add characters to the line containing the emojis, the display gets garbled
again.

-- 
Aura




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at 50865 <at> debbugs.gnu.org:


Received: (at 50865) by debbugs.gnu.org; 28 Sep 2021 16:12:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 28 12:12:08 2021
Received: from localhost ([127.0.0.1]:46768 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mVFiB-0005Br-S6
	for submit <at> debbugs.gnu.org; Tue, 28 Sep 2021 12:12:08 -0400
Received: from eggs.gnu.org ([209.51.188.92]:55104)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1mVFi9-0005BN-NT
 for 50865 <at> debbugs.gnu.org; Tue, 28 Sep 2021 12:12:06 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:49762)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <eliz@HIDDEN>)
 id 1mVFi3-0002S1-Rb; Tue, 28 Sep 2021 12:11:59 -0400
Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4193
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1mVFi3-0004Jh-Et; Tue, 28 Sep 2021 12:11:59 -0400
Date: Tue, 28 Sep 2021 19:12:04 +0300
Message-Id: <83a6jwk7cb.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Aura Kelloniemi <kaura.dev@HIDDEN>
In-Reply-To: <87o88chjsl.fsf@HIDDEN> (message from Aura Kelloniemi on Tue,
 28 Sep 2021 17:11:22 +0300)
Subject: Re: bug#50865: 28.0.50;
 Emoji with emoji modifier in Linux console garbles emacs display
References: <87o88chjsl.fsf@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 50865
Cc: 50865 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Aura Kelloniemi <kaura.dev@HIDDEN>
> Date: Tue, 28 Sep 2021 17:11:22 +0300
> 
> I'm running emacs in the Linux virtual console. Showing text that contains
> Unicode emoji characters with modifiers causes wrong output. The problem may
> be related to the fact that Linux basically does not understand Unicode
> composition, multi-column charactes or anything fancy. Anyhow, here are the
> instructions to reproduce this:
> 
> Run emacs -Q
> 
> In the scratch buffer on an empty line:
> Type C-x 8 RET PERSON WITH FOLDED HANDS RET
> šŸ™ appears followed by the cursor. This is what I expect. (In the TTY screen
> only a black diamond is shown, but I can verify from /de/vcsu that the
> character really is an emoji.)
> 
> Then type C-x 8 RET EMOJI MODIFIER FITZPATRICK TYPE-3 RET
> Now the emoji modifier appears (again as a diamond), but it is followed by a
> space character.
> 
> So now the line looks like:
> <PERSON WITH FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <SPACE>
> _cursor_

If your terminal doesn't understand character composition, the best
solution for you is to turn off auto-composition-mode when using Emacs
on that terminal.  Please try that and tell if that resolves the issue
for you: type "M-x auto-composition-mode RET" and make sure Emacs says
that the mode is disabled.

Thanks.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 28 Sep 2021 14:11:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 28 10:11:38 2021
Received: from localhost ([127.0.0.1]:46594 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mVDpZ-00024h-T4
	for submit <at> debbugs.gnu.org; Tue, 28 Sep 2021 10:11:38 -0400
Received: from lists.gnu.org ([209.51.188.17]:53380)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <kaura.dev@HIDDEN>) id 1mVDpY-00024Y-Nn
 for submit <at> debbugs.gnu.org; Tue, 28 Sep 2021 10:11:37 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:56530)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <kaura.dev@HIDDEN>)
 id 1mVDpY-0005lU-JG
 for bug-gnu-emacs@HIDDEN; Tue, 28 Sep 2021 10:11:36 -0400
Received: from oiva.sange.fi ([2a03:e581:4:200:aa51::150]:34408
 helo=smtp.sange.fi)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <kaura.dev@HIDDEN>)
 id 1mVDpV-0002bK-OU
 for bug-gnu-emacs@HIDDEN; Tue, 28 Sep 2021 10:11:36 -0400
Received: from 88-114-110-12.elisa-laajakaista.fi ([88.114.110.12]
 helo=solaria)
 by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.89) (envelope-from <kaura.dev@HIDDEN>) id 1mVDpM-0005mE-Op
 for bug-gnu-emacs@HIDDEN; Tue, 28 Sep 2021 17:11:27 +0300
From: Aura Kelloniemi <kaura.dev@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 28.0.50; Emoji with emoji modifier in Linux console garbles emacs
 display
Date: Tue, 28 Sep 2021 17:11:22 +0300
Message-ID: <87o88chjsl.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.7 (--)
Received-SPF: pass client-ip=2a03:e581:4:200:aa51::150;
 envelope-from=kaura.dev@HIDDEN; helo=smtp.sange.fi
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.6 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

Hello,

I'm running emacs in the Linux virtual console. Showing text that contains
Unicode emoji characters with modifiers causes wrong output. The problem may
be related to the fact that Linux basically does not understand Unicode
composition, multi-column charactes or anything fancy. Anyhow, here are the
instructions to reproduce this:

Run emacs -Q

In the scratch buffer on an empty line:
Type C-x 8 RET PERSON WITH FOLDED HANDS RET
=F0=9F=99=8F appears followed by the cursor. This is what I expect. (In the=
 TTY screen
only a black diamond is shown, but I can verify from /de/vcsu that the
character really is an emoji.)

Then type C-x 8 RET EMOJI MODIFIER FITZPATRICK TYPE-3 RET
Now the emoji modifier appears (again as a diamond), but it is followed by a
space character.

So now the line looks like:
<PERSON WITH FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <SPACE>
_cursor_

I think that the space there should not be added. My guess is that the space
is actually not written to the terminal, but that emacs misplaces the curso=
r.

Now if a press backspace to delete the last typed character (the emoji
modifier), only the space disappears. If I then press backspace again, both=
 the
emoji and the modifier disapper at the same time.

Interestingly, if I repeat the process of adding the above mentioned emoji =
and
the modifier characters on a line two times (starting again from an empty
line), the line looks like this:

<PERSON WITH FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <SPACE> <PER=
SON
WITH FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <SPACE> _cursor_

If I now run M-x redraw-display RET, the line looks like this:

<PERSON WITH FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <PERSON WITH
FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <SPACE> <SPACE> <SPACE>
_cursor_

So for some reason there are now three spaces before the cursor.

I tried this on Debian version of Emacs 26.1, and the results were similar,
but not exactly identical.

Why do I care? If I use a Linux console, and it cannot display emoji, why d=
oes
this matter? Because if there is any other text on the same line, it often
gets very garbled, especially if Emacs decides to only update the line
partially.

I cannot stop other people from using emojis nowadays, and that's why I'd l=
ike
Emacs to tolerate them.

I will gladly provide more detail. I'm also interested in any (dirty) hacks
that could be used to work around this issue, as it disturbs my emacs usage
all the time (I use Telegram from within Emacs).

If I run emacs within GNU screen (that itself runs in a Linux VT), this
problem does not seem to appear. There are other issues with some Unicode
characters in screen, but I haven't yet found a clear way to reproduce these
issues.

Below is the data produced by M-x report-emacs-bug RET






In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30, c=
airo version 1.17.4)
 of 2021-09-26 built on solaria
Repository revision: 43ae8c828d853382bbc2a27b9e14b9fff6ba18b6
Repository branch: makepkg
System Description: Arch Linux

Configured using:
 'configure --prefix=3D/usr --sysconfdir=3D/etc --libexecdir=3D/usr/lib
 --localstatedir=3D/var --with-native-compilation --with-x-toolkit=3Dgtk3
 --with-xft --with-wide-int --with-modules --with-gameuser=3D:games
 --with-sound=3Dalsa --with-cairo --with-harfbuzz
 --enable-link-time-optimization 'CFLAGS=3D-march=3Dnative -mtune=3Dnative =
-O2 -pipe
 -fno-plt -fuse-ld=3Dgold -flto -fuse-ld=3Dgold -flto'
 LDFLAGS=3D-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
 CPPFLAGS=3D-D_FORTIFY_SOURCE=3D2'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM
XPM GTK3 ZLIB

Important settings:
  value of $LANG: fi_FI.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/d

Minor modes in effect:
  gpm-mouse-mode: t
  leaf-key-override-global-mode: t
  shell-dirtrack-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  icomplete-mode: t
  desktop-save-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: linux
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t

Load-path shadows:
/home/aura/.config/emacs/elpa/magit-20210925.1143/magit-section-pkg hides /=
home/aura/.config/emacs/elpa/magit-section-20210829.1849/magit-section-pkg
/home/aura/.config/emacs/elpa/transient-20210919.1006/transient hides /usr/=
share/emacs/28.0.50/lisp/transient

Features:
(shadow sort mail-extr emacsbug sendmail cursor-sensor t-mouse term/linux
desktop+ haskell-mode haskell-cabal haskell-utils haskell-font-lock
haskell-indentation haskell-string haskell-sort-imports haskell-lexeme
haskell-align-imports haskell-complete-module haskell-ghc-support flymake-p=
roc
flymake warnings dabbrev haskell-customize python tramp-sh autorevert
filenotify conf-mode cargo cargo-process markdown-mode color racer f s dash
company-oddmuse company-keywords company-etags etags fileloop xref project
company-gtags company-dabbrev-code company-dabbrev company-files company-cl=
ang
company-capf company-cmake company-semantic company-template company-bbdb
vc-git vc-dispatcher rust-utils rust-mode rx rust-rustfmt rust-playpen
rust-compile compile rust-cargo yaml-mode org-element avl-tree generator
ol-eww eww xdg url-queue mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus
nnselect gnus-search eieio-opt cl-extra help-mode speedbar ezimage dframe
gnus-art mm-uu mml2015 gnus-sum shr kinsoku svg dom gnus-group gnus-undo
gnus-start gnus-dbus dbus xml gnus-cloud nnimap nnmail mail-source utf7 net=
rc
nnoo gnus-spec gnus-int gnus-range gnus-win gnus nnheader ol-docview doc-vi=
ew
jka-compr image-mode exif ol-bibtex bibtex ol-bbdb ol-w3m org ob ob-tangle
ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint
org-pcomplete org-list org-faces org-entities noutline outline org-version
ob-emacs-lisp ob-core ob-eval org-table ol org-keys org-compat advice org-m=
acs
org-loaddefs find-func recentf tree-widget notmuch notmuch-tree notmuch-jump
notmuch-hello notmuch-show notmuch-print notmuch-crypto notmuch-mua
notmuch-message notmuch-draft notmuch-maildir-fcc notmuch-address
notmuch-company notmuch-parser notmuch-wash diff-mode easy-mmode coolj
notmuch-query goto-addr thingatpt icalendar diary-lib diary-loaddefs cal-me=
nu
calendar cal-loaddefs notmuch-tag crm notmuch-lib notmuch-version
notmuch-compat hl-line message rmc puny dired dired-loaddefs rfc822 mml
mailabbrev gmm-utils mailheader mm-view mml-smime mml-sec epa derived epg
rfc6068 epg-config gnus-util rmail rmail-loaddefs mail-utils
text-property-search smime dig mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr server leaf-keywords
leaf finder-inf package browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap url-handlers
url-parse url-vars cus-edit pp wid-edit tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat shell pcomplete comint ansi-color ri=
ng
parse-time iso8601 time-date ls-lisp format-spec auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json map savehist
minibuf-eldef keypad ido seq byte-opt gv bytecomp byte-compile cconv icompl=
ete
company edmacro kmacro pcase subr-x desktop frameset cl-loaddefs cl-lib
cus-load info iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomm=
ent
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-l=
ang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button loadde=
fs
faces cus-face macroexp files window text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 541107 17966)
 (symbols 48 48459 2)
 (strings 32 220875 3286)
 (string-bytes 1 6213483)
 (vectors 16 57360)
 (vector-slots 8 1387450 51457)
 (floats 8 484 304)
 (intervals 56 910 2)
 (buffers 992 39))

--=20
Aura




Acknowledgement sent to Aura Kelloniemi <kaura.dev@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#50865; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 4 Oct 2021 17:15:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.