GNU bug report logs - #61932
28.2; [csv-mode] Comma in quoted field mistaken as field delimiter

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 3 Mar 2023 11:02:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 03 06:02:00 2023
Received: from localhost ([127.0.0.1]:59764 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1pY3Al-0001lA-Vv
	for submit <at> debbugs.gnu.org; Fri, 03 Mar 2023 06:02:00 -0500
Received: from lists.gnu.org ([209.51.188.17]:59766)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <joostkremers@HIDDEN>) id 1pY3Aj-0001l1-1H
 for submit <at> debbugs.gnu.org; Fri, 03 Mar 2023 06:01:57 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <joostkremers@HIDDEN>)
 id 1pY3Ai-0004UB-QT
 for bug-gnu-emacs@HIDDEN; Fri, 03 Mar 2023 06:01:56 -0500
Received: from out1-smtp.messagingengine.com ([66.111.4.25])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <joostkremers@HIDDEN>)
 id 1pY3Af-0006QP-9h
 for bug-gnu-emacs@HIDDEN; Fri, 03 Mar 2023 06:01:56 -0500
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
 by mailout.nyi.internal (Postfix) with ESMTP id 6209E5C0074
 for <bug-gnu-emacs@HIDDEN>; Fri,  3 Mar 2023 06:01:49 -0500 (EST)
Received: from imap44 ([10.202.2.94])
 by compute3.internal (MEProxy); Fri, 03 Mar 2023 06:01:49 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.fm; h=
 cc:content-type:content-type:date:date:from:from:in-reply-to
 :message-id:mime-version:reply-to:sender:subject:subject:to:to;
 s=fm1; t=1677841309; x=1677927709; bh=Q2LM4YK5e4wjURbOTSK7u9EZD
 Ax187ULnXUAwMWvAn0=; b=x+z/yrGBYdESpQf0vNzhwMHH8DVeHnM/LI+RTysp+
 aUsFCBF6afiiH+iyd7RM7S0Wy2JnmXWkqj0f4se8eupBAt92lCJiptwgoQA/DTXl
 meg0XMVOJ8yK+UD19bol0BTUZ8KbTaHEoJXGH08tMj3zCWhrOYTtEDrARrChxa2c
 vdjpRC/O5VGKV/UIWKzf/Vx6suPSEKUaYGxatwFdofq59+2uWIPmAO8MhaEo55Yc
 j2a9KcKAmi8HfVm8Pw0GQnMYHQ7h91NPh9xDcvhzBvCvJ/QvyHdpJp2AivagCvWR
 uMSatlcB+2cE0MS69vXlKMeRGI10jibugRtP61PS/QU/w==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:content-type:content-type:date:date
 :feedback-id:feedback-id:from:from:in-reply-to:message-id
 :mime-version:reply-to:sender:subject:subject:to:to:x-me-proxy
 :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
 1677841309; x=1677927709; bh=Q2LM4YK5e4wjURbOTSK7u9EZDAx187ULnXU
 AwMWvAn0=; b=B7mCtRqh+6Qb1J1nyP3JpdIgMXji62zBDp1sPgeR7FT8eZisqdo
 9jmtxzxRnjQogJM0OmqnOMnUsyaI+9M5Geu6xRnA9I5vxg7ZaCicwAmWwZ8EtyOk
 KJswssTuLk8hkwvZU2MW1SGwjIy+uQBhTD7VOpHrQEpB1MQPwvsGxmA9H+3seAMX
 MeVtiud1HQN6Br+9RmnGCMNZ+L9dPBqw+rfnFSrQNujwX3RkRKFGDNrEfXYyXgDD
 epXTKWRURygkz1mZFt5SiL+Lphnlm9seXW4mqmDC40XxtIB0+y8ZD2mRop4/4XuC
 7dFdt+jUPiA/Xwq6PZ3E1mizwp1dRVIq0iA==
X-ME-Sender: <xms:ndMBZH_N2C8Jm-2TE-O2pvfOzAn8f0udlK-8M9YFIKIhFAEU70ReJQ>
 <xme:ndMBZDu1-QIqwf046d2to7AbonZBWiIBt9lCiS8X_LaMdMhHdSCTp5kczJp6xvpfH
 jxRvkRXB5lYCw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrudelledgudekucetufdoteggodetrfdotf
 fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
 uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfffhffvufgtsehttdertd
 erredtnecuhfhrohhmpeflohhoshhtuceojhhoohhsthhkrhgvmhgvrhhssehfrghsthhm
 rghilhdrfhhmqeenucggtffrrghtthgvrhhnpeeihffgkeejjedufeevtdetteehveffie
 ehjedtvdehveffgfeggfejffekieekjeenucevlhhushhtvghrufhiiigvpedtnecurfgr
 rhgrmhepmhgrihhlfhhrohhmpehjohhoshhtkhhrvghmvghrshesfhgrshhtmhgrihhlrd
 hfmh
X-ME-Proxy: <xmx:ndMBZFCE6dBqQE9l8ug65srW3ARsxWbhl8cfGK_47a-tdpiSsHu2Ig>
 <xmx:ndMBZDei8F4UCZNjW-k5a5C0tdp2x99DQ6AQPRKgVPtAodtDKEtb7g>
 <xmx:ndMBZMM_KhcrfH6vdvlqr3ZNPjRVkwbZfGOIR6lqoRcA-buo0fndcA>
 <xmx:ndMBZCbJP_MkkQly6vybYeTVExL3rBZRTW4TQHBC6IexvVfHDUcZSw>
Feedback-ID: ie15541ac:Fastmail
Received: by mailuser.nyi.internal (Postfix, from userid 501)
 id 399DB36A0073; Fri,  3 Mar 2023 06:01:49 -0500 (EST)
X-Mailer: MessagingEngine.com Webmail Interface
User-Agent: Cyrus-JMAP/3.9.0-alpha0-183-gbf7d00f500-fm-20230220.001-gbf7d00f5
Mime-Version: 1.0
Message-Id: <f983f8eb-ad0f-4a99-be91-d36bb900912f@HIDDEN>
Date: Fri, 03 Mar 2023 12:01:28 +0100
From: Joost <joostkremers@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 28.2; [csv-mode] Comma in quoted field mistaken as field delimiter
Content-Type: text/plain
Received-SPF: pass client-ip=66.111.4.25;
 envelope-from=joostkremers@HIDDEN; helo=out1-smtp.messagingengine.com
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001,
 SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

It seems that csv-mode mistakes a comma in a quoted field for a field delimiter
in certain cases. Starting with `emacs -Q', then installing and loading
`csv-mode' from GNU ELPA, create a small csv file with two columns:

==================================================
one,1
two,2
"three,four",34
five,5
==================================================

Now, when I use TAB to move from one field to the next, point stops at the comma
between "three" and "four".

Also, hen sorting, the third line is sorted on "four", not on "34", both when sorting
numerically and alphabetically. This is the result when sorting numerically on
column 2 (C-c C-n):

==================================================
"three,four",34
one,1
two,2
five,5
==================================================

Line 3 ends up as line one, (I assume because "four" is not a number and sorts
as zero). When sorting alphabetically (C-c C-s), I get:

==================================================
one,1
two,2
five,5
"three,four",34
==================================================

Where the line <<"three,four",34>> is sorted on "four", not on "34".

`csv-separators` is set to the default value `("\"")'.

I've also occasionally experiences problems deleting a column (C-c C-k) if there
is a column with text before the column being deleted and some of the fields in
this column contain quoted commas. I haven't constructed a minimal example for
this, since I assume it's the same problem, but I could if desired.

Note that font lock does seem to understand that the comma in the field is not a
separator, because it doesn't have `csv-separator-face'.

Thanks,

Joost


In GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.36, cairo version 1.17.6)
 of 2023-01-03 built on 2
Windowing system distributor 'The X.Org Foundation', version 11.0.12101007
System Description: Arch Linux

Configured using:
 'configure --with-x-toolkit=gtk3 --with-native-compilation --sysconfdir=/etc
 --prefix=/usr --libexecdir=/usr/lib --localstatedir=/var --with-cairo
 --with-harfbuzz --with-libsystemd --with-modules 'CFLAGS=-march=x86-64
 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat
 -Werror=format-security -fstack-clash-protection -fcf-protection -g
 -ffile-prefix-map=/build/emacs/src=/usr/src/debug/emacs -flto=auto'
 'LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto''
Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM
GTK3 ZLIB
Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: CSV

Minor modes in effect:
  cursor-sensor-mode: t
  subword-mode: t
  magit-auto-revert-mode: t
  company-box-mode: t
  csv-align-mode: t
  csv-field-index-mode: t
  minions-mode: t
  doom-modeline-mode: t
  which-key-mode: t
  global-atomic-chrome-edit-mode: t
  marginalia-mode: t
  all-the-icons-completion-mode: t
  company-prescient-mode: t
  prescient-persist-mode: t
  vertico-multiform-mode: t
  vertico-mode: t
  projectile-mode: t
  global-diff-hl-mode: t
  diff-hl-mode: t
  global-git-commit-mode: t
  shell-dirtrack-mode: t
  hl-line-mode: t
  goggles-mode: t
  company-quickhelp-mode: t
  company-quickhelp-local-mode: t
  global-company-mode: t
  company-mode: t
  electric-pair-mode: t
  recentf-mode: t
  delete-selection-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-save-visited-mode: t

Load-path shadows:
~/.emacs.d/lisp/custom hides /usr/share/emacs/28.2/lisp/custom
/home/joost/.emacs.d/elpa/transient-20230220.1425/transient hides /usr/share/emacs/28.2/lisp/transient

Features:
(shadow mail-extr emacsbug mc-edit-lines magit-extras display-line-numbers
vc-mtn vc-hg vc-bzr vc-src vc-sccs vc-cvs vc-rcs bug-reference mixed-pitch
pandoc-mode pandoc-mode-utils consult-vertico consult ace-window avy misearch
multi-isearch cursor-sensor vc-git cal-move tabify cap-words superword subword
visual-fill-column org-autolist org-indent ol-eww eww mm-url ol-rmail ol-mhe
ol-irc ol-info ol-gnus nnselect gnus-search eieio-opt speedbar ezimage dframe
gnus-art mm-uu mml2015 mm-view mml-smime smime dig gnus-sum gnus-group gnus-undo
gnus-start gnus-dbus gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo
gnus-spec gnus-int gnus-range gnus-win gnus nnheader ol-docview doc-view
jka-compr image-mode exif ol-bibtex ol-bbdb ol-w3m ol-doi org-link-doi
org-capture org-refile magit-bookmark magit-submodule magit-obsolete magit-blame
magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch
magit-clone magit-remote magit-commit magit-sequence magit-notes magit-worktree
magit-tag magit-merge magit-branch magit-reset magit-files magit-refs
magit-status magit magit-repos magit-apply magit-wip magit-log which-func
magit-diff smerge-mode diff magit-core magit-autorevert magit-margin
magit-transient magit-process magit-mode mm-archive mule-util gnutls url-cache
jk-input-methods quail company-box company-box-doc frame-local company-box-icons
company-amelia amelia parse-csv csv-mode sort hi-lock go-translate
gts-engine-youdao gts-engine-stardict gts-engine-deepl gts-engine-google-rpc
gts-engine-google gts-engine-bing gts-implements gts-faces gts-core wgrep-ag
wgrep deadgrep flycheck-ledger vterm bookmark term disp-table ehelp vterm-module
term/xterm xterm ielm minions doom-modeline doom-modeline-segments
doom-modeline-env doom-modeline-core shrink-path which-key atomic-chrome iimage
image+ image-file image-converter marginalia all-the-icons-completion
company-prescient prescient char-fold orderless vertico-multiform vertico
groovy-mode dockerfile-mode sh-script smie executable impatient-mode htmlize
tide tide-lv indium indium-list-sources indium-scratch indium-interaction
indium-chrome indium-nodejs indium-repl indium-debugger indium-debugger-litable
js2-refactor js2r-paredit js2r-conveniences js2r-conditionals js2r-wrapping
js2r-functions js2r-vars mc-hide-unmatched-lines-mode mc-mark-more
mc-cycle-cursors multiple-cursors-core rect js2r-iife js2r-formatting
js2r-helpers indium-debugger-locals indium-breakpoint indium-inspector
indium-render indium-faces indium-seq-fix indium-client indium-structs
json-process-client js2-mode js cc-mode cc-fonts cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine cc-vars cc-defs jupyter python-pytest lsp-pyright
dap-python paredit clojure-mode align dap-mode dap-tasks dap-launch lsp-docker
dap-overlays eldoc-box jsonrpc ert debug backtrace lsp-ui lsp-ui-flycheck
lsp-ui-doc goto-addr lsp-ui-imenu lsp-ui-peek lsp-ui-sideline flycheck-posframe
flycheck lsp-mode network-stream markdown-mode inline imenu f f-shortdoc
shortdoc lsp-ui-util face-remap lsp-protocol ht ag vc-svn find-dired s
projectile lisp-mnt grep compile ibuf-ext ibuffer ibuffer-loaddefs thingatpt
diff-hl log-view vc-dir ewoc vc vc-dispatcher diff-mode git-commit magit-git
magit-base magit-section crm with-editor transient log-edit message rfc822 mml
mml-sec epa epg rfc6068 epg-config gnus-util rmail rmail-loaddefs mm-decode
mm-bodies mm-encode mailabbrev gmm-utils mailheader pcvs-util add-log compat
compat-29 autorevert filenotify aggressive-indent sendmail mail-utils nswbuff
finder-inf yaml-mode yaml guess-language flyspell ispell reftex reftex-loaddefs
reftex-vars tab-jump-out yasnippet-snippets yasnippet company-org-block
org-settings org-clock ob-jupyter jupyter-tramp jupyter-server derived
jupyter-server-ioloop jupyter-ioloop-comm jupyter-ioloop zmq zmq-core
jupyter-rest-api url-http url-auth mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr url-gw nsm rmc jupyter-org-extensions jupyter-org-client
jupyter-repl jupyter-kernel-manager jupyter-channel jupyter-widget-client
websocket bindat simple-httpd jupyter-client jupyter-comm-layer jupyter-messages
hmac-def jupyter-mime shr kinsoku text-property-search puny svg
jupyter-kernelspec jupyter-env jupyter-base eieio-base ob-shell ob-scheme
ob-clojure ob-python python tramp-sh org-element avl-tree dom org ob ob-tangle
ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint
org-pcomplete org-list org-faces org-entities noutline outline org-version
ob-emacs-lisp ob-core ob-eval org-table oc-basic ol org-keys oc org-compat
org-macs org-loaddefs find-func cal-menu calendar cal-loaddefs denote xdg dired
dired-loaddefs docker-tramp tramp-cache tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat shell pcomplete comint ansi-color
parse-time ls-lisp format-spec biblio biblio-download biblio-dissemin
biblio-ieee biblio-hal biblio-dblp biblio-crossref biblio-arxiv timezone
biblio-doi biblio-core let-alist url-queue ido hl-line bibtex iso8601 time-date
adaptive-wrap goggles comp comp-cstr warnings pulse posframe bm
company-quickhelp pos-tip all-the-icons all-the-icons-faces data-material
data-weathericons data-octicons data-fileicons data-faicons data-alltheicons
company-keywords company-etags etags fileloop generator xref project
company-gtags company-dabbrev-code company-dabbrev company-ipa company-files
company-clang company-cmake company-semantic company-template company-css
company-capf company solarized-light-theme solarized-theme solarized
solarized-faces color paradox paradox-menu paradox-commit-list hydra ring lv
cus-edit pp cus-load paradox-execute paradox-github paradox-core spinner
whitespace jk-functions advice dash eshell esh-cmd esh-ext esh-opt esh-proc
esh-io esh-arg esh-module esh-groups esh-util notifications dbus xml cl-extra
help-mode edmacro kmacro server elec-pair recentf tree-widget wid-edit delsel
use-package use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key use-package-core easy-mmode tex-site rx info
package browse-url url url-proxy url-privacy url-expand url-methods url-history
url-cookie url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map url-vars
seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib pcase iso-transl
tooltip eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer
select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj
charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk
x-toolkit x multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 2772328 367045)
 (symbols 48 74856 49)
 (strings 32 422440 48171)
 (string-bytes 1 14715498)
 (vectors 16 153295)
 (vector-slots 8 3375385 310936)
 (floats 8 1991 2084)
 (intervals 56 273127 52213)
 (buffers 992 34))