GNU bug report logs - #34873
27.0.50; Creating unibyte string in emacs modules

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 34873 in the body.
You can then email your comments to 34873 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Evgeny Zajcev <lg.zevlg <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.0.50; Creating unibyte string in emacs modules
Date: Fri, 15 Mar 2019 23:52:17 +0300

[Message part 1 (text/plain, inline)]

There is no possibility to create unibyte strings from emacs modules in
order to create some binary data representation

The only solution for now is to call `make_string` method and then apply
`encode-coding-string' with utf-8 encoding to the result.  But on large
data this became very slow

I used this patch to bypass the conversion steps and it works pretty nice

Ideally I would like to use user-ptr objects as `:data' in image
specification, but solution with `make_data' module method will be suitable
enough

Thanks

Here is the patch I'm currently using:

diff --git a/src/emacs-module.c b/src/emacs-module.c
index 4e2411cb29..500079c257 100644
--- a/src/emacs-module.c
+++ b/src/emacs-module.c
@@ -591,6 +591,19 @@ module_make_string (emacs_env *env, const char *str,
ptrdiff_t length)
                         code_convert_string_norecord (lstr, Qutf_8,
false));
 }

+static emacs_value
+module_make_data (emacs_env *env, const char *str, ptrdiff_t length)
+{
+  MODULE_FUNCTION_BEGIN (module_nil);
+  if (! (0 <= length && length <= STRING_BYTES_BOUND))
+    overflow_error ();
+
+  Lisp_Object lstr = make_uninit_string (length + 1);
+  memcpy (SDATA (lstr), str, length);
+  SDATA (lstr)[length] = 0;
+  return lisp_to_value (env, lstr);
+}
+
 static emacs_value
 module_make_user_ptr (emacs_env *env, emacs_finalizer_function fin, void
*ptr)
 {
@@ -1081,6 +1094,7 @@ initialize_environment (emacs_env *env, struct
emacs_env_private *priv)
   env->make_float = module_make_float;
   env->copy_string_contents = module_copy_string_contents;
   env->make_string = module_make_string;
+  env->make_data = module_make_data;
   env->make_user_ptr = module_make_user_ptr;
   env->get_user_ptr = module_get_user_ptr;
   env->set_user_ptr = module_set_user_ptr;


Configuration:

In GNU Emacs 27.0.50 (build 19, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2019-03-15 built on XPS
Repository revision: 020e69d992c98fd852e835c9bd707a8d137090f2
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.1 LTS

Recent messages:
t
Mark set [2 times]
Auto-saving...
Mark set [2 times]
t
Saving file /home/lg/.emacs.d/*scratch-file*...
Wrote /home/lg/.emacs.d/*scratch-file*
Finding changes in /home/lg/dev/emacs/src/emacs-module.c...done
Saved text until "nv->set_user_ptr = module_set_user_ptr;
"

Configured using:
 'configure --without-makeinfo --with-xwidgets --with-modules'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF
XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS XWIDGETS
PDUMPER LCMS2 GMP

Important settings:
  value of $LC_MONETARY: ru_RU.UTF-8
  value of $LC_NUMERIC: ru_RU.UTF-8
  value of $LC_TIME: ru_RU.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: C/*

Minor modes in effect:
  icomplete-mode: t
  save-place-mode: t
  pyvenv-mode: t
  shell-dirtrack-mode: t
  dot-mode: t
  display-time-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
~/github/telega.el/test hides ~/github/emacs-ffi/test
/home/lg/.emacs.d/elpa/flim-20180328.2324/md4 hides
/usr/local/share/emacs/27.0.50/lisp/md4
/home/lg/.emacs.d/elpa/flim-20180328.2324/hex-util hides
/usr/local/share/emacs/27.0.50/lisp/hex-util
/home/lg/.emacs.d/elpa/flim-20180328.2324/sasl-digest hides
/usr/local/share/emacs/27.0.50/lisp/net/sasl-digest
/home/lg/.emacs.d/elpa/flim-20180328.2324/sasl-ntlm hides
/usr/local/share/emacs/27.0.50/lisp/net/sasl-ntlm
/home/lg/.emacs.d/elpa/flim-20180328.2324/hmac-md5 hides
/usr/local/share/emacs/27.0.50/lisp/net/hmac-md5
/home/lg/.emacs.d/elpa/flim-20180328.2324/sasl hides
/usr/local/share/emacs/27.0.50/lisp/net/sasl
/home/lg/.emacs.d/elpa/flim-20180328.2324/ntlm hides
/usr/local/share/emacs/27.0.50/lisp/net/ntlm
/home/lg/.emacs.d/elpa/flim-20180328.2324/hmac-def hides
/usr/local/share/emacs/27.0.50/lisp/net/hmac-def
/home/lg/.emacs.d/elpa/flim-20180328.2324/sasl-cram hides
/usr/local/share/emacs/27.0.50/lisp/net/sasl-cram

Features:
(shadow sort mail-extr emacsbug sendmail smerge-mode vc vc-dispatcher
vc-git cc-mode cc-fonts cc-guess cc-menus cc-styles cc-align ffi-wand
ffi ffi-module home desktop frameset gnus-demon nntp gnus-group
gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source utf7 netrc
gnus-spec gnus-win nnoo gnus-int gnus-range message rmc puny dired
dired-loaddefs rfc822 mml mml-sec epa epg mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader gnus nnheader
gnus-util rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums time-date
mail-utils mm-util mail-prsvr cal-menu calendar cal-loaddefs icomplete
saveplace cython-mode help-fns radix-tree elpy find-file-in-project ivy
delsel colir color ivy-overlay ffap windmove diff-mode elpy-shell pyvenv
esh-var esh-cmd esh-opt esh-io esh-ext esh-proc esh-arg esh-groups
eshell esh-module esh-mode esh-util elpy-profile elpy-django s
elpy-refactor python tramp-sh tramp tramp-loaddefs trampver
tramp-integration tramp-compat ucs-normalize parse-time format-spec grep
files-x etags fileloop generator xref project cus-edit cus-start
cus-load wid-edit python-mode pcase info-look which-func imenu shell
pcomplete hippie-exp flymake-proc flymake warnings thingatpt compile
cc-cmds cc-engine cc-vars cc-defs rx dot-mode server time elec-pair
google-translate google-translate-default-ui google-translate-core-ui
google-translate-core google-translate-tk url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap whitespace undo-tree diff ido comint ansi-color ring avoid
edmacro kmacro browse-kill-ring advice derived cl-extra help-mode
use-package use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core mule-util
autoinsert tex-site gh-common marshal eieio-compat cl info package
easymenu epg-config url-handlers url-parse auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting xwidget-internal move-toolbar
gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 457704 32112)
 (symbols 48 29656 2)
 (strings 32 92291 5700)
 (string-bytes 1 15482759)
 (vectors 16 62874)
 (vector-slots 8 1478873 216844)
 (floats 8 425 132)
 (intervals 56 879 164)
 (buffers 992 16))

-- 
lg


-- 
lg

[Message part 2 (text/html, inline)]

Message #8 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Evgeny Zajcev <lg.zevlg <at> gmail.com>
Cc: 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Sat, 16 Mar 2019 09:30:26 +0200

> From: Evgeny Zajcev <lg.zevlg <at> gmail.com>
> Date: Fri, 15 Mar 2019 23:52:17 +0300
> 
> There is no possibility to create unibyte strings from emacs modules in order to create some binary data
> representation
> 
> The only solution for now is to call `make_string` method and then apply `encode-coding-string' with utf-8
> encoding to the result.  But on large data this became very slow
> 
> I used this patch to bypass the conversion steps and it works pretty nice
> 
> Ideally I would like to use user-ptr objects as `:data' in image specification, but solution with `make_data'
> module method will be suitable enough

Please explain why you need to create a unibyte string for that.  I
guess what's missing is more details regarding what you are trying to
do (the single sentence above about :data wasn't enough for me), and
how is your module involved in that, in particular why do you need a
module to display an image in Emacs.

Message #11 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: lg.zevlg <at> gmail.com
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Sat, 16 Mar 2019 11:32:47 +0100

[Message part 1 (text/plain, inline)]

16 марта 2019 г., в 8:30, Eli Zaretskii <eliz <at> gnu.org> написал(а):

>> From: Evgeny Zajcev <lg.zevlg <at> gmail.com>
>> Date: Fri, 15 Mar 2019 23:52:17 +0300
>> 
>> There is no possibility to create unibyte strings from emacs modules in order to create some binary data
>> representation
>> 
>> The only solution for now is to call `make_string` method and then apply `encode-coding-string' with utf-8
>> encoding to the result.  But on large data this became very slow
>> 
>> I used this patch to bypass the conversion steps and it works pretty nice
>> 
>> Ideally I would like to use user-ptr objects as `:data' in image specification, but solution with `make_data'
>> module method will be suitable enough
> 
> Please explain why you need to create a unibyte string for that.  I
> guess what's missing is more details regarding what you are trying to
> do (the single sentence above about :data wasn't enough for me), and
> how is your module involved in that, in particular why do you need a
> module to display an image in Emacs.

See this thread https://lists.gnu.org/archive/html/emacs-devel/2019-03/msg00465.html

I need this module, because all existing packages to manipulate images are not capable enough for me

—
lg

[Message part 2 (text/html, inline)]

Message #14 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: lg.zevlg <at> gmail.com
Cc: 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Sat, 16 Mar 2019 14:20:41 +0200

> From: lg.zevlg <at> gmail.com
> Date: Sat, 16 Mar 2019 11:32:47 +0100
> Cc: 34873 <at> debbugs.gnu.org
> 
>  Please explain why you need to create a unibyte string for that.  I
>  guess what's missing is more details regarding what you are trying to
>  do (the single sentence above about :data wasn't enough for me), and
>  how is your module involved in that, in particular why do you need a
>  module to display an image in Emacs.
> 
> See this thread https://lists.gnu.org/archive/html/emacs-devel/2019-03/msg00465.html

I've read that (I read everything on emacs-devel).  It still doesn't
tell me enough to answer my questions, unfortunately.

> I need this module, because all existing packages to manipulate images are not capable enough for me

Please tell more, if at all possible.  I don't think I understand the
context yet.

Message #17 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: lg.zevlg <at> gmail.com
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Sat, 16 Mar 2019 16:28:05 +0100

16 марта 2019 г., в 13:20, Eli Zaretskii <eliz <at> gnu.org> написал(а):

>> From: lg.zevlg <at> gmail.com
>> Date: Sat, 16 Mar 2019 11:32:47 +0100
>> Cc: 34873 <at> debbugs.gnu.org
>> 
>> Please explain why you need to create a unibyte string for that.  I
>> guess what's missing is more details regarding what you are trying to
>> do (the single sentence above about :data wasn't enough for me), and
>> how is your module involved in that, in particular why do you need a
>> module to display an image in Emacs.
>> 
>> See this thread https://lists.gnu.org/archive/html/emacs-devel/2019-03/msg00465.html
> 
> I've read that (I read everything on emacs-devel).  It still doesn't
> tell me enough to answer my questions, unfortunately.

Ok, emacs-ffi is emacs module that implements FFI to shared libraries. wand-mode is package that implements ffi bindings (using emacs-ffi module) to libMagickWand and on top of them powerful image editor is written

wand-mode does many trickery on images in very fast fasion, and it needs method to display previews/results in Emacs buffer. Result of image manipulation is raw rgb(a) data that seats in user-ptr available from imagemagick without any copying. I want `create-image’ with :format ‘image/x-rgb to accept this user-ptr as data to make image creation fast

Method proposed by Stefan with make_string/encode-coding-string works, but its speed is far from being usable on big images.  There is no method just to use data as-is to make things fast


—
lg

Message #20 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: lg.zevlg <at> gmail.com
Cc: 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Sat, 16 Mar 2019 18:17:08 +0200

> From: lg.zevlg <at> gmail.com
> Date: Sat, 16 Mar 2019 16:28:05 +0100
> Cc: 34873 <at> debbugs.gnu.org
> 
> wand-mode does many trickery on images in very fast fasion, and it needs method to display previews/results in Emacs buffer. Result of image manipulation is raw rgb(a) data that seats in user-ptr available from imagemagick without any copying. I want `create-image’ with :format ‘image/x-rgb to accept this user-ptr as data to make image creation fast
> 
> Method proposed by Stefan with make_string/encode-coding-string works, but its speed is far from being usable on big images.  There is no method just to use data as-is to make things fast

I'd prefer to provide a way to use user-ptr objects for this purpose,
rather than allow unibyte strings to be created by modules.  The
latter is a slippery slope which I think we should rather avoid.

Can you propose a patch that uses user-ptr instead?

Thanks.

Message #25 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Tue, 07 May 2019 17:55:56 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: lg.zevlg <at> gmail.com
>> Date: Sat, 16 Mar 2019 16:28:05 +0100
>> Cc: 34873 <at> debbugs.gnu.org
>> 
>> wand-mode does many trickery on images in very fast fasion, and it needs
>> method to display previews/results in Emacs buffer. Result of image
>> manipulation is raw rgb(a) data that seats in user-ptr available from
>> imagemagick without any copying. I want `create-image’ with :format
>> ‘image/x-rgb to accept this user-ptr as data to make image creation fast
>> 
>> Method proposed by Stefan with make_string/encode-coding-string works, but its
>> speed is far from being usable on big images.  There is no method just to use
>> data as-is to make things fast
>
> I'd prefer to provide a way to use user-ptr objects for this purpose,
> rather than allow unibyte strings to be created by modules.  The
> latter is a slippery slope which I think we should rather avoid.
>
> Can you propose a patch that uses user-ptr instead?

Sorry if this is a dumb question, but would the functions
unibyte-string, string-to-unibyte, string-as-unibyte et al. be of any
use here?

-- 
Basil

Message #28 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Tue, 07 May 2019 21:33:40 +0300

> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
> Cc: <lg.zevlg <at> gmail.com>,  <34873 <at> debbugs.gnu.org>
> Date: Tue, 07 May 2019 17:55:56 +0100
> 
> Sorry if this is a dumb question, but would the functions
> unibyte-string, string-to-unibyte, string-as-unibyte et al. be of any
> use here?

I'm not sure I follow: how can these functions help writing code in a
module?  A module cannot be written in Lisp, and you cannot call these
functions on a C string, only on a Lisp string.

What am I missing?

Message #31 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Tue, 07 May 2019 20:24:48 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
>> Cc: <lg.zevlg <at> gmail.com>,  <34873 <at> debbugs.gnu.org>
>> Date: Tue, 07 May 2019 17:55:56 +0100
>> 
>> Sorry if this is a dumb question, but would the functions
>> unibyte-string, string-to-unibyte, string-as-unibyte et al. be of any
>> use here?
>
> I'm not sure I follow: how can these functions help writing code in a
> module?  A module cannot be written in Lisp, and you cannot call these
> functions on a C string, only on a Lisp string.
>
> What am I missing?

It could be I that is missing something, but AIUI the module API already
provides module_make_string for creating a multibyte Lisp string from a
C string, which can then be passed to one of the aforementioned
functions via module_funcall, no?

In other words, I'm asking whether the aforementioned functions are in
any way a better alternative to the encode-coding-string approach
previously proposed.

Thanks,

-- 
Basil

Message #34 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Tue, 07 May 2019 22:48:03 +0300

> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
> Cc: <lg.zevlg <at> gmail.com>,  <34873 <at> debbugs.gnu.org>
> Date: Tue, 07 May 2019 20:24:48 +0100
> 
> It could be I that is missing something, but AIUI the module API already
> provides module_make_string for creating a multibyte Lisp string from a
> C string, which can then be passed to one of the aforementioned
> functions via module_funcall, no?
> 
> In other words, I'm asking whether the aforementioned functions are in
> any way a better alternative to the encode-coding-string approach
> previously proposed.

Not sure how it would be better, but I don't think I understand the
use case in enough detail.

Message #37 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Wed, 26 Aug 2020 12:43:59 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> I'd prefer to provide a way to use user-ptr objects for this purpose,
> rather than allow unibyte strings to be created by modules.  The
> latter is a slippery slope which I think we should rather avoid.

In what way are unibyte strings slippery slopes?  They're a standard
part of the Emacs type vocabulary.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #40 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Wed, 26 Aug 2020 13:52:58 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: lg.zevlg <at> gmail.com,  34873 <at> debbugs.gnu.org
> Date: Wed, 26 Aug 2020 12:43:59 +0200
> 
> In what way are unibyte strings slippery slopes?  They're a standard
> part of the Emacs type vocabulary.

They produce incorrect display of non-ASCII characters, for starters.
They also produce unexpected effects when concatenated with multibyte
strings.  They are in general something to avoid because of these and
other subtleties, since many Lisp programmers are unaware of them, and
can easily fall into one of these pitfalls.

In this case AFAIR the request was to use unibyte strings for binary
data, i.e. stuff that is not a string at all, which is one more reason
not to base the implementation on unibyte strings.

Message #43 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Thu, 27 Aug 2020 15:04:32 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> In what way are unibyte strings slippery slopes?  They're a standard
>> part of the Emacs type vocabulary.
>
> They produce incorrect display of non-ASCII characters, for starters.
> They also produce unexpected effects when concatenated with multibyte
> strings.  They are in general something to avoid because of these and
> other subtleties, since many Lisp programmers are unaware of them, and
> can easily fall into one of these pitfalls.
>
> In this case AFAIR the request was to use unibyte strings for binary
> data, i.e. stuff that is not a string at all, which is one more reason
> not to base the implementation on unibyte strings.

Yes, the only real reason to use unibyte strings is for binary data, I
thought?  That's the use case in the proposed patch, too.

Just like unibyte buffers are for data, not text, unibyte strings are
for data, not text.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #46 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Thu, 27 Aug 2020 16:40:31 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: lg.zevlg <at> gmail.com,  34873 <at> debbugs.gnu.org
> Date: Thu, 27 Aug 2020 15:04:32 +0200
> 
> > In this case AFAIR the request was to use unibyte strings for binary
> > data, i.e. stuff that is not a string at all, which is one more reason
> > not to base the implementation on unibyte strings.
> 
> Yes, the only real reason to use unibyte strings is for binary data, I
> thought?

No, that's not the use case for unibyte strings.  We have bindat.el
for that.  Unibyte strings is for handling text that was not yet
decoded into the internal representation, or which was encoded before
sending it to an external program or I/O port.

> Just like unibyte buffers are for data, not text, unibyte strings are
> for data, not text.

If that's what you really think, we disagree.  About the only use case
which comes close (though not close enough) to what you describe is
when we hold compressed data in a buffer.

Message #49 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Lars Ingebrigtsen <larsi <at> gnus.org>, lg.zevlg <at> gmail.com,
 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Thu, 27 Aug 2020 10:33:31 -0400

On Thu, 27 Aug 2020 at 09:43, Eli Zaretskii <eliz <at> gnu.org> wrote:

> > Yes, the only real reason to use unibyte strings is for binary data, I
> > thought?
>
> No, that's not the use case for unibyte strings.  We have bindat.el
> for that.

Quoting from bindat.el commentary: "Encoded (binary) data is stored in
a unibyte string or vector,"
Does that mean that you think binary data should be stored only in vectors?

Message #52 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: larsi <at> gnus.org, lg.zevlg <at> gmail.com, 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Thu, 27 Aug 2020 19:57:14 +0300

> From: Noam Postavsky <npostavs <at> gmail.com>
> Date: Thu, 27 Aug 2020 10:33:31 -0400
> Cc: Lars Ingebrigtsen <larsi <at> gnus.org>, 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
> 
> Does that mean that you think binary data should be stored only in vectors?

Why would it mean that, I wonder?

Message #55 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Lars Magne Ingebrigtsen <larsi <at> gnus.org>, lg.zevlg <at> gmail.com,
 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Thu, 27 Aug 2020 13:13:36 -0400

On Thu, 27 Aug 2020 at 12:57, Eli Zaretskii <eliz <at> gnu.org> wrote:

> > Does that mean that you think binary data should be stored only in vectors?
>
> Why would it mean that, I wonder?

You said binary data is not the use case for unibyte strings and
pointed to bindat.el instead. bindat.el uses unibyte strings or
vectors for binary data.

(NOT unibyte strings) AND (unibyte strings OR vectors) => vectors

Message #58 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: larsi <at> gnus.org, lg.zevlg <at> gmail.com, 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Thu, 27 Aug 2020 20:18:55 +0300

> From: Noam Postavsky <npostavs <at> gmail.com>
> Date: Thu, 27 Aug 2020 13:13:36 -0400
> Cc: Lars Magne Ingebrigtsen <larsi <at> gnus.org>, 34873 <at> debbugs.gnu.org, lg.zevlg <at> gmail.com
> 
> On Thu, 27 Aug 2020 at 12:57, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> > > Does that mean that you think binary data should be stored only in vectors?
> >
> > Why would it mean that, I wonder?
> 
> You said binary data is not the use case for unibyte strings and
> pointed to bindat.el instead. bindat.el uses unibyte strings or
> vectors for binary data.
> 
> (NOT unibyte strings) AND (unibyte strings OR vectors) => vectors

I don't see how it follows.  You are reading into what I wrote
something I didn't mean.  The underlying implementation of bindat.el
doesn't matter much as long as bindat.el is used.

Message #61 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Evgeny Zajcev <lg.zevlg <at> gmail.com>
Cc: 34873 <at> debbugs.gnu.org
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Tue, 13 Oct 2020 06:52:32 +0200

Evgeny Zajcev <lg.zevlg <at> gmail.com> writes:

> There is no possibility to create unibyte strings from emacs modules
> in order to create some binary data representation

There was some discussion about what the meaning of "is" is that
followed, but the general idea seems sensible -- if modules are to be
able to pass binary data to Emacs efficiently, something like this is
needed, so I've now added this to Emacs 28 in the form of
make_unibyte_string.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #68 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 34873 <at> debbugs.gnu.org, Evgeny Zajcev <lg.zevlg <at> gmail.com>
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Tue, 13 Oct 2020 16:31:43 +0200

Am Di., 13. Okt. 2020 um 06:53 Uhr schrieb Lars Ingebrigtsen <larsi <at> gnus.org>:
>
> Evgeny Zajcev <lg.zevlg <at> gmail.com> writes:
>
> > There is no possibility to create unibyte strings from emacs modules
> > in order to create some binary data representation
>
> There was some discussion about what the meaning of "is" is that
> followed, but the general idea seems sensible -- if modules are to be
> able to pass binary data to Emacs efficiently, something like this is
> needed, so I've now added this to Emacs 28 in the form of
> make_unibyte_string.
>


Thanks. However, you need to add it properly to maintain backward
compatibility; see the comments at the top of emacs-module.c.

Message #71 received at 34873 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 34873 <at> debbugs.gnu.org, Evgeny Zajcev <lg.zevlg <at> gmail.com>
Subject: Re: bug#34873: 27.0.50; Creating unibyte string in emacs modules
Date: Wed, 14 Oct 2020 05:59:01 +0200

Philipp Stephani <p.stephani2 <at> gmail.com> writes:

> Thanks. However, you need to add it properly to maintain backward
> compatibility; see the comments at the top of emacs-module.c.

OK; now moved to the right .h file.  (The comments referred to
emacs-env-27.h etc, which doesn't exist, so I assumed that it was
supposed to be module-env-27.h etc, and updated the comments.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.