X-Loop: help-debbugs@HIDDEN
Subject: bug#58929: 29.0.50; Calc: finding roots utpn doesn't work as advertised
Resent-From: Matt Armstrong <matt@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Mon, 31 Oct 2022 18:11:02 +0000
Resent-Message-ID: <handler.58929.B.166723984721247 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: report 58929
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords:
To: 58929 <at> debbugs.gnu.org
X-Debbugs-Original-To: bug-gnu-emacs@HIDDEN
Received: via spool by submit <at> debbugs.gnu.org id=B.166723984721247
(code B ref -1); Mon, 31 Oct 2022 18:11:02 +0000
Received: (at submit) by debbugs.gnu.org; 31 Oct 2022 18:10:47 +0000
Received: from localhost ([127.0.0.1]:41738 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1opZFG-0005Wa-9p
for submit <at> debbugs.gnu.org; Mon, 31 Oct 2022 14:10:47 -0400
Received: from lists.gnu.org ([209.51.188.17]:58926)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <matt@HIDDEN>) id 1opZFD-0005WS-SH
for submit <at> debbugs.gnu.org; Mon, 31 Oct 2022 14:10:44 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <matt@HIDDEN>) id 1opZF9-0002Nk-35
for bug-gnu-emacs@HIDDEN; Mon, 31 Oct 2022 14:10:40 -0400
Received: from relay10.mail.gandi.net ([2001:4b98:dc4:8::230])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <matt@HIDDEN>) id 1opZF4-0004kn-JD
for bug-gnu-emacs@HIDDEN; Mon, 31 Oct 2022 14:10:38 -0400
Received: (Authenticated sender: matt@HIDDEN)
by mail.gandi.net (Postfix) with ESMTPSA id B13C2240008
for <bug-gnu-emacs@HIDDEN>; Mon, 31 Oct 2022 18:10:25 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rfc20.org; s=gm1;
t=1667239828;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:mime-version:mime-version:content-type:content-type:
content-transfer-encoding:content-transfer-encoding;
bh=nF+rbCMEQ+oAcL3lSmcga/qqowH/q1YBKSyvjbyJiaY=;
b=e6EeXEONNK0rEjYAEYYDLHWX59gAZhABqWifvn0DwkGyoURj2sipoUllwOCB3qiJ5fLcR1
5I6iq4cPS0zOpHkfk7sKak3qTdCh7MKxT0G7kzyK6EwiqFJvsPG3w1pr9th6l1bKFEs6zS
BRi0fnjl3PZuVBZibr79ReWBOXFltdZmVwjZla92ddDcGiAKhKLREdWD8cMOvmZ8kDSlMi
NLvIRpyR9cKkkYulooAXZcX8VLRalzKoAvtRRBmzO+ekqUtZSvHxn7MRD9Z666PeXj1s3F
uYUgw0fOItSV/0GE/6ACgfwt+T0h3wLoiEbQdNIEE6x2xAvXQW2t2kJgnAI0zg==
Received: from matt by naz with local (Exim 4.96)
(envelope-from <matt@HIDDEN>) id 1opZEs-000Gs0-1P
for bug-gnu-emacs@HIDDEN; Mon, 31 Oct 2022 11:10:22 -0700
From: Matt Armstrong <matt@HIDDEN>
Date: Mon, 31 Oct 2022 11:10:22 -0700
Message-ID: <87fsf3uanl.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=2001:4b98:dc4:8::230; envelope-from=matt@HIDDEN;
helo=relay10.mail.gandi.net
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001,
SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.6 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)
Quick facts:
- all of these examples work with emacs -Q
- reproduces in Emacs 29, 27, 25
I think I have found that the calc package's "root" finding command does
not work as documented with calc's probability distribution functions.
Or, at least, not with "utpn", the probability distribution function for
the normal (gaussian) distribution. I have found similar issues with
"utpt" (the Student's t-distribution), but will focus on "utpn" here.
The Emacs Calc package info page for Probability Distribution Functions
(https://www.gnu.org/software/emacs/manual/html_node/calc/Probability-Distr=
ibution-Functions.html)
has this to say about the normal distribution function:
> The =E2=80=98utpn(x,m,s)=E2=80=99 function uses a normal (Gaussian) distr=
ibution with
> mean =E2=80=98m=E2=80=99 and standard deviation =E2=80=98s=E2=80=99. It i=
s the probability that such a
> normal-distributed random variable would exceed =E2=80=98x=E2=80=99.
I am interested in solving `utpn(x,m,s) =3D p` for `p` given a fixed `m`
and `s`. The same info page has this to say about that:
> While Calc does not provide inverses of the probability distribution
> functions, the a R command can be used to solve for the inverse. Since
> the distribution functions are monotonic, a R is guaranteed to be able
> to find a solution given any initial guess.
SUPPOSITION: the above is true, especially this: "a R is guaranteed to be
able to find a solution given any initial guess."
The Root Finding info page
(https://www.gnu.org/software/emacs/manual/html_node/calc/Root-Finding.html)
has this to say about the a R command:
> The a R (calc-find-root) [root] command finds a numerical solution (or
> root) of an equation. (This command treats inequalities the same as
> equations. If the input is any other kind of formula, it is interpreted
> as an equation of the form =E2=80=98X =3D 0=E2=80=99.)
>
> The a R command requires an initial guess on the top of the stack, and a
> formula in the second-to-top position. It prompts for a solution
> variable, which must appear in the formula. All other variables that
> appear in the formula must have assigned values, i.e., when a value is
> assigned to the solution variable and the formula is evaluated with =3D, =
it
> should evaluate to a number. Any assigned value for the solution variable
> itself is ignored and unaffected by this command.
>
> When the command completes, the initial guess is replaced on the stack by
> a vector of two numbers: The value of the solution variable that solves
> the equation, and the difference between the lefthand and righthand sides
> of the equation at that value. Ordinarily, the second number will be zero
> or very nearly zero. (Note that Calc uses a slightly higher precision
> while finding the root, and thus the second number may be slightly
> different from the value you would compute from the equation yourself.)
WHAT I HAVE TRIED
I have tried entering the formula `utpn(x, 10, 1) =3D 0.5` and invoking `a
R` with an initial guess of `10`. I then solve for `x`, expecting
`x =3D 10`.
Note that the correct answer of `x =3D 10` is "obviously" correct. I am
passing `m =3D 10` for the "mean" arg to `utpn`, and by definition a
random value from the normal distribution has an `0.5` chance of being
larger than the mean.
To do this I type "C-x * *" to enter the *Calculator* buffer and then
type:
` utpn(x, 10, 1) =3D 0.5 RET
10 RET <-- initial guess
a R RET <-- invoke "root" function
x RET <-- answer to prompt; solve for "x"
and I end up with this in the `*Calculator*` buffer:
2: utpn(x, 10, 1) =3D 0.5
1: [10., 0.]
This is correct. The 'a R' command has returned a vector whose first
element is `x` and the second is an error delta (not important here).
Alternatively, evaluate this lisp to do the same thing:
ELISP> (calc-eval '("root(utpn(x, 10, 1) =3D 0.5, x, 10)"))
"[10., 0.]"
While correct, here I have *given* the answer to calc in the initial
guess (the root function's third arg). The problem is that the function
doesn't do well when given other guesses.
```
` utpn(x,10,1)=3D0.5 RET
1.0 RET <---- a different guess
a R RET
x RET
```
results in this:
```
2: utpn(x, 10, 1) =3D 0.5
1: root(utpn(x, 10, 1) =3D 0.5, x, 1)
```
With this in the minibuffer:
> *Newton's method failed to converge: 4.8639203627839e17*
I'm finding that if the initial guess is close enough to the right answer,
in this case `10`, then `a R` works, otherwise not. E.g. guessing `8.6`
fails but `8.7` works.
This behavior makes `a R` much less useful. Also, documentation
suggests that any guess is expected to work (re-quoting from above with
my emphasis):
> Since the distribution functions are monotonic, 'a R' is guaranteed to
> be able to find a solution *given any initial guess*.
Have I found a bug here, or am I doing something wrong?
ADDENDUM
This shows the function failing to find a root for guesses not all that
far away from the actual answer:
ELISP> (cl-loop for x from 8.0 to 12.0 by 0.1 collect (list (format "%.1f" =
x) (calc-eval '("root(utpn(x,10,1)=3D0.5,x,$)") nil (prin1-to-string x))))
(("8.0" "root(utpn(x, 10, 1) =3D 0.5, x, 8.)")
("8.1" "root(utpn(x, 10, 1) =3D 0.5, x, 8.1)")
("8.2" "root(utpn(x, 10, 1) =3D 0.5, x, 8.2)")
("8.3" "root(utpn(x, 10, 1) =3D 0.5, x, 8.3)")
("8.4" "root(utpn(x, 10, 1) =3D 0.5, x, 8.4)")
("8.5" "root(utpn(x, 10, 1) =3D 0.5, x, 8.5)")
("8.6" "root(utpn(x, 10, 1) =3D 0.5, x, 8.6)")
("8.7" "[10., 0.]")
("8.8" "[10., 0.]")
("8.9" "[10., 0.]")
("9.0" "[10., 0.]")
("9.1" "[10., 0.]")
("9.2" "[10., 0.]")
("9.3" "[10., 0.]")
("9.4" "[10., 0.]")
("9.5" "[10., 0.]")
("9.6" "[10., 0.]")
("9.7" "[10., 0.]")
("9.8" "[10., 0.]")
("9.9" "[10., 0.]")
("10.0" "[10., 0.]")
("10.1" "[10., 0.]")
("10.2" "[10., 0.]")
("10.3" "[10., 0.]")
("10.4" "[10., 0.]")
("10.5" "[10., 0.]")
("10.6" "[10., 0.]")
("10.7" "[10., 0.]")
("10.8" "[10., 0.]")
("10.9" "[10., 0.]")
("11.0" "[10., 0.]")
("11.1" "[10., 0.]")
("11.2" "[10., 0.]")
("11.3" "[10., 0.]")
("11.4" "root(utpn(x, 10, 1) =3D 0.5, x, 11.4)")
("11.5" "root(utpn(x, 10, 1) =3D 0.5, x, 11.5)")
("11.6" "root(utpn(x, 10, 1) =3D 0.5, x, 11.6)")
("11.7" "root(utpn(x, 10, 1) =3D 0.5, x, 11.7)")
("11.8" "root(utpn(x, 10, 1) =3D 0.5, x, 11.8)")
("11.9" "root(utpn(x, 10, 1) =3D 0.5, x, 11.9)")
("12.0" "root(utpn(x, 10, 1) =3D 0.5, x, 12.)"))
In GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version
3.24.34, cairo version 1.16.0) of 2022-10-30 built on naz
Repository revision: 73953b23aad5af80275838cd3b1c88e045e1856e
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12201003
System Description: Debian GNU/Linux bookworm/sid
Configured using:
'configure 'CFLAGS=3D-O2 -g3''
Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB
Important settings:
value of $LANG: en_US.UTF-8
value of $XMODIFIERS: @im=3Dibus
locale-coding-system: utf-8-unix
Major mode: Text
Minor modes in effect:
msb-mode: t
display-time-mode: t
flyspell-mode: t
global-tab-line-mode: t
tab-line-mode: t
envrc-global-mode: t
envrc-mode: t
shell-dirtrack-mode: t
auto-insert-mode: t
keyfreq-autosave-mode: t
keyfreq-mode: t
savehist-mode: t
icomplete-vertical-mode: t
icomplete-mode: t
editorconfig-mode: t
which-key-mode: t
electric-pair-mode: t
override-global-mode: t
tooltip-mode: t
global-eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tab-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
column-number-mode: t
line-number-mode: t
auto-fill-function: do-auto-fill
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
temp-buffer-resize-mode: t
abbrev-mode: t
Load-path shadows:
~/env/elisp/ol-notmuch hides /home/matt/.config/emacs/elpa/ol-notmuch-20220=
428.1337/ol-notmuch
/home/matt/.config/emacs/elpa/transient-20220918.2101/transient hides /home=
/matt/git/e/daily-driver/lisp/transient
/home/matt/.config/emacs/elpa/eglot-20221005.1955/eglot hides /home/matt/gi=
t/e/daily-driver/lisp/progmodes/eglot
Features:
(shadow emacsbug ielm calc-yank shortdoc calc-undo edebug dired-aux
calcalg3 imenu cl-print calc-stuff calc-funcs dabbrev calcalg2 calc-poly
calc-vec calc-mtx calc-lang calc-cplx calccomp calc-menu calc-frac
calc-comb calc-aent calc-math calc-arith calc-alg calc-bin calc-tests
calc-misc calc-prog calc-forms calc-units calc-ext calc calc-loaddefs
rect calc-macs ert pp ewoc debug backtrace mule-util magit-base crm
misearch multi-isearch copyright help-fns radix-tree pcase smerge-mode
diff qp sort smiley gnus-cite mail-extr textsec uni-scripts idna-mapping
ucs-normalize uni-confusable textsec-check gnus-async gnus-bcklg
gnus-agent gnus-srvr gnus-score score-mode nnvirtual nntp gnus-ml
gnus-msg nndoc gnus-cache gnus-dup mm-archive network-stream url-cache
display-line-numbers debbugs-gnu add-log debbugs-compat debbugs
soap-client url-http url-auth url-gw nsm rng-xsd rng-dt rng-util
xsd-regexp debbugs-browse vc-git diff-mode vc-dispatcher bug-reference
editorconfig-core editorconfig-core-handle editorconfig-fnmatch protbuf
msb time flyspell ispell org-element avl-tree generator ol-w3m ol-rmail
ol-mhe ol-irc ol-info org-habit org-agenda org-refile ol-gnus nnselect
gnus-art mm-uu mml2015 mm-view mml-smime smime gnutls dig gnus-sum
gnus-group gnus-undo gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail
mail-source utf7 nnoo parse-time gnus-spec gnus-int gnus-range message
sendmail yank-media rfc822 mml mml-sec epa derived epg rfc6068
epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047
rfc2045 ietf-drums mailabbrev gmm-utils mailheader gnus-win ol-eww eww
xdg url-queue shr pixel-fill kinsoku url-file svg xml dom puny mm-url
gnus nnheader gnus-util text-property-search mail-utils range wid-edit
mm-util mail-prsvr ol-doi org-link-doi ol-docview doc-view filenotify
jka-compr image-mode exif dired dired-loaddefs ol-bibtex ol-bbdb
tab-line server envrc inheritenv web-mode disp-table nix-mode ffap
thingatpt smie nix-repl nix-shell nix-store magit-section dash compat-27
compat-26 nix-instantiate nix-shebang nix-format nix dirtrack ob-shell
shell ob-ruby ob-python python compat compat-macs ob-dot org-protocol
org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote
org-src ob-comint org-pcomplete pcomplete comint ansi-osc ansi-color
ring org-list org-faces org-entities noutline outline org-version
ob-emacs-lisp ob-core ob-eval org-table oc-basic bibtex iso8601
time-date org-keys oc org-loaddefs find-func cal-menu calendar
cal-loaddefs finder-inf ol-notmuch ol rx org-compat org-macs format-spec
skeleton autoinsert advice keyfreq project edmacro kmacro warnings icons
savehist icomplete editorconfig which-key package browse-url url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source eieio eieio-core password-cache json subr-x map byte-opt
url-vars cl-extra help-mode cl-macs gv cl-seq elec-pair use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core cl-loaddefs
cl-lib bytecomp byte-compile info bazel-autoloads
clang-format+-autoloads clang-format-autoloads cmake-mode-autoloads
d-mode-autoloads debbugs-autoloads editorconfig-autoloads
eglot-autoloads elpy-autoloads company-autoloads envrc-autoloads
exec-path-from-shell-autoloads flymake-ruby-autoloads
flymake-easy-autoloads flymake-yamllint-autoloads go-mode-autoloads
google-c-style-autoloads graphviz-dot-mode-autoloads
highlight-indentation-autoloads inheritenv-autoloads magit-autoloads
git-commit-autoloads markdown-mode-autoloads meson-mode-autoloads
nix-mode-autoloads magit-section-autoloads dash-autoloads
nixpkgs-fmt-autoloads ol-notmuch-autoloads notmuch-autoloads
orderless-autoloads org-drill-autoloads ox-hugo-autoloads
persist-autoloads pylint-autoloads pyvenv-autoloads s-autoloads
shfmt-autoloads reformatter-autoloads tomelr-autoloads
transient-autoloads use-package-autoloads bind-key-autoloads
vertico-autoloads web-mode-autoloads which-key-autoloads
with-editor-autoloads compat-autoloads yaml-mode-autoloads
yasnippet-autoloads rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
emacs)
Memory information:
((conses 16 977488 266858)
(symbols 48 43332 287)
(strings 32 225511 12569)
(string-bytes 1 5996788)
(vectors 16 103426)
(vector-slots 8 1994918 188498)
(floats 8 517 940)
(intervals 56 9190 882)
(buffers 984 38))
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: Matt Armstrong <matt@HIDDEN> Subject: bug#58929: Acknowledgement (29.0.50; Calc: finding roots utpn doesn't work as advertised) Message-ID: <handler.58929.B.166723984721247.ack <at> debbugs.gnu.org> References: <87fsf3uanl.fsf@HIDDEN> X-Gnu-PR-Message: ack 58929 X-Gnu-PR-Package: emacs Reply-To: 58929 <at> debbugs.gnu.org Date: Mon, 31 Oct 2022 18:11:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. Your message has been sent to the package maintainer(s): bug-gnu-emacs@HIDDEN If you wish to submit further information on this problem, please send it to 58929 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 58929: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D58929 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
X-Loop: help-debbugs@HIDDEN
Subject: bug#58929: 29.0.50; Calc: finding roots utpn doesn't work as advertised
References: <87fsf3uanl.fsf@HIDDEN>
In-Reply-To: <87fsf3uanl.fsf@HIDDEN>
Resent-From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= <mattiase@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 01 Nov 2022 16:52:01 +0000
Resent-Message-ID: <handler.58929.B58929.16673214824908 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 58929
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords:
To: Matt Armstrong <matt@HIDDEN>
Cc: 58929 <at> debbugs.gnu.org
Received: via spool by 58929-submit <at> debbugs.gnu.org id=B58929.16673214824908
(code B ref 58929); Tue, 01 Nov 2022 16:52:01 +0000
Received: (at 58929) by debbugs.gnu.org; 1 Nov 2022 16:51:22 +0000
Received: from localhost ([127.0.0.1]:44097 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1opuTx-0001H6-Qp
for submit <at> debbugs.gnu.org; Tue, 01 Nov 2022 12:51:22 -0400
Received: from mail33c50.megamailservers.eu ([91.136.10.43]:60832)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mattiase@HIDDEN>) id 1opuTu-0001Gt-U2
for 58929 <at> debbugs.gnu.org; Tue, 01 Nov 2022 12:51:20 -0400
X-Authenticated-User: mattiase@HIDDEN
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=megamailservers.eu;
s=maildub; t=1667321476;
bh=fGninPJ7WSkzwK/Cx8D9Mgy2WcZhovqpwZGN0rc9suE=;
h=From:Subject:Date:Cc:To:From;
b=FochP5xN76hxSYcJ8i8zv7DSOt82sEzj8gbn5mjFy+vmOsp9tmmE70Ew7pwtjgbaz
5KZrOtn95BBotdRD70V4lBRLN3Q6uc09TRlVhUwi59d2orVELGly1Oo136ANtjiqY8
N17iCgZnWbFlK2FOeq451Dbyn83UeYTgw1VOS0M4=
Feedback-ID: mattiase@HIDDEN
Received: from smtpclient.apple (c188-150-171-209.bredband.tele2.se
[188.150.171.209]) (authenticated bits=0)
by mail33c50.megamailservers.eu (8.14.9/8.13.1) with ESMTP id 2A1GpC91099113;
Tue, 1 Nov 2022 16:51:15 +0000
From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= <mattiase@HIDDEN>
Content-Type: text/plain;
charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\))
Message-Id: <8773828B-3A5F-4B15-8786-91F6FCB6D2E2@HIDDEN>
Date: Tue, 1 Nov 2022 17:51:11 +0100
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-CTCH-RefID: str=0001.0A782F25.63614E84.003F, ss=1, re=0.000, recu=0.000,
reip=0.000, cl=1, cld=1, fgs=0
X-CTCH-VOD: Unknown
X-CTCH-Spam: Unknown
X-CTCH-Score: 0.000
X-CTCH-Rules:
X-CTCH-Flags: 0
X-CTCH-ScoreCust: 0.000
X-Origin-Country: SE
X-Spam-Score: 0.3 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)
Unfortunately straight Newton won't converge for this function (erf, =
essentially) if you stray too far from the origin (ie, the mean) so I =
suggest you always use that as a starting guess no matter what point you =
are trying to find the inverse for.
Another way is to use an interval as starting guess instead of a single =
point; that seems to make Calc switch tactics and persevere.
One thing that can happen is running out of stack, because many of the =
implementations like math-newton-root are recursive. This should be easy =
to remedy as they all seem to be tail calls.
Obviously the claim to guarantee a solution for "any initial guess" is =
fanciful.
X-Loop: help-debbugs@HIDDEN
Subject: bug#58929: 29.0.50; Calc: finding roots utpn doesn't work as advertised
Resent-From: Matt Armstrong <matt@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 01 Nov 2022 22:01:01 +0000
Resent-Message-ID: <handler.58929.B58929.166734005311003 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 58929
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords:
To: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= <mattiase@HIDDEN>
Cc: 58929 <at> debbugs.gnu.org
Received: via spool by 58929-submit <at> debbugs.gnu.org id=B58929.166734005311003
(code B ref 58929); Tue, 01 Nov 2022 22:01:01 +0000
Received: (at 58929) by debbugs.gnu.org; 1 Nov 2022 22:00:53 +0000
Received: from localhost ([127.0.0.1]:44356 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1opzJV-0002rP-7I
for submit <at> debbugs.gnu.org; Tue, 01 Nov 2022 18:00:53 -0400
Received: from relay6-d.mail.gandi.net ([217.70.183.198]:42097)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <matt@HIDDEN>) id 1opzJR-0002r7-VO
for 58929 <at> debbugs.gnu.org; Tue, 01 Nov 2022 18:00:51 -0400
Received: (Authenticated sender: matt@HIDDEN)
by mail.gandi.net (Postfix) with ESMTPSA id 58C50C0004;
Tue, 1 Nov 2022 22:00:41 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rfc20.org; s=gm1;
t=1667340041;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:cc:mime-version:mime-version:content-type:content-type:
in-reply-to:in-reply-to:references:references;
bh=DM1d21dzbEsTqYy1ImjhVBWjR9Wkh3+PH38MdbHNne4=;
b=c12GDDgxL0XwlDaxqJZaY8CAh3fWga2cNeTGn5PgPTjanB4KhFRbvtZAzBcXonMjgoekwH
N76fqOi2ZdEd1y334Xzc7ESTUTmxwcQb27Y1kjm0PkOcZAh2Gun67KXjRTdDUSPg2wudjH
E5uf9M37TLqpYGHKhYDFtVLB9FxceuNrV2P1j09gV/PgZCGDvdCbEGEqhXInQvcOgSDcg3
KSBA2Fe6BvBwwNtA16bthgu6sU0w2gv6IYihR+rPRMS5Js0+NSfS3OS30qkXH6Vr7LT2VL
3xBKsRI/c4OiN6O3kn7XtQPZ3wgYYiGXfo1xyM9jvpTFud2Am6sKlDZy+bPvoA==
Received: from matt by naz with local (Exim 4.96)
(envelope-from <matt@HIDDEN>) id 1opzJG-000H2O-1Z;
Tue, 01 Nov 2022 15:00:38 -0700
From: Matt Armstrong <matt@HIDDEN>
In-Reply-To: <8773828B-3A5F-4B15-8786-91F6FCB6D2E2@HIDDEN>
References: <8773828B-3A5F-4B15-8786-91F6FCB6D2E2@HIDDEN>
Date: Tue, 01 Nov 2022 15:00:38 -0700
Message-ID: <87k04es5bt.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mattias Engdeg=C3=A5rd <mattiase@HIDDEN> writes:
> Obviously the claim to guarantee a solution for "any initial guess" is
> fanciful.
Yes. See attached patch for what I wish the docs had said all along
(would have saved me a lot of time).
> Unfortunately straight Newton won't converge for this function (erf,
> essentially) if you stray too far from the origin (ie, the mean) so I
> suggest you always use that as a starting guess no matter what point
> you are trying to find the inverse for.
>
> Another way is to use an interval as starting guess instead of a
> single point; that seems to make Calc switch tactics and persevere.
I wonder if the implementation can be improved.
https://www.gnu.org/software/emacs/manual/html_node/calc/Root-Finding.html
suggests that there are some cases where calc will switch over to the
bisection method. Perhaps the heuristics can be tweaked?
> One thing that can happen is running out of stack, because many of the
> implementations like math-newton-root are recursive. This should be
> easy to remedy as they all seem to be tail calls.
Good to know.
--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
filename=0002-In-calc.texi-provide-guidance-for-taking-the-inverse.patch
From ba68ca5671f9772994bc5c0b2b0b9216a750b692 Mon Sep 17 00:00:00 2001
From: Matt Armstrong <matt@HIDDEN>
Date: Tue, 1 Nov 2022 14:51:36 -0700
Subject: [PATCH 2/2] In calc.texi provide guidance for taking the inverse of
PDFs.
* doc/misc/calc.texi (Probability Distribution Functions): Stop
claiming that the 'a R' function will "always work" for any guess.
Explain the problem with the PDF functions being very flat. Provide
recommended guess values for each function. bug#58929
---
doc/misc/calc.texi | 36 +++++++++++++++++++++++++++++++++---
1 file changed, 33 insertions(+), 3 deletions(-)
diff --git a/doc/misc/calc.texi b/doc/misc/calc.texi
index 89a340e7343..e6b416a447a 100644
--- a/doc/misc/calc.texi
+++ b/doc/misc/calc.texi
@@ -19456,9 +19456,39 @@ Probability Distribution Functions
While Calc does not provide inverses of the probability distribution
functions, the @kbd{a R} command can be used to solve for the inverse.
-Since the distribution functions are monotonic, @kbd{a R} is guaranteed
-to be able to find a solution given any initial guess.
-@xref{Numerical Solutions}.
+@xref{Root Finding}.
+
+The distribution functions, while monotonic, are also be very flat,
+which can cause problems for the Newton's root-finding algorithm used
+by @kbd{a R}. The problem can be avoided by providing a good starting
+guess. Prefer guesses that occur where the slope of the plotted
+function is not flat. Alternatively, use an interval range for the
+guess. The table below provides a recommendation for each of the
+probability distribution functions.
+
+@multitable @columnfractions .25 .25 .25
+@headitem Function
+@tab When solving for
+@tab Recommended initial guess
+@item @samp{utpb(x,n,p)}
+@tab @expr{x}
+@tab @code{0.5}
+@item @samp{utpc(x,c)}
+@tab @expr{x}
+@tab @code{1.0}
+@item @samp{utpf(F,v1,v2)}
+@tab @expr{F}
+@tab @code{1.0}
+@item @samp{utpn(x,m,s)}
+@tab @expr{x}
+@tab @expr{m}
+@item @samp{utpp(n,x)}
+@tab @expr{n}
+@tab @expr{x}
+@item @samp{utpt(t,v)}
+@tab @expr{t}
+@tab @code{[0..1e4]}
+@end multitable
@node Matrix Functions
@chapter Vector/Matrix Functions
--
2.35.1
--=-=-=--
Received: (at control) by debbugs.gnu.org; 1 Nov 2022 22:01:59 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Nov 01 18:01:59 2022 Received: from localhost ([127.0.0.1]:44360 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1opzKY-0002tU-OW for submit <at> debbugs.gnu.org; Tue, 01 Nov 2022 18:01:58 -0400 Received: from relay2-d.mail.gandi.net ([217.70.183.194]:50533) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <matt@HIDDEN>) id 1opzKX-0002tG-LO for control <at> debbugs.gnu.org; Tue, 01 Nov 2022 18:01:58 -0400 Received: (Authenticated sender: matt@HIDDEN) by mail.gandi.net (Postfix) with ESMTPSA id 3DD7640005 for <control <at> debbugs.gnu.org>; Tue, 1 Nov 2022 22:01:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rfc20.org; s=gm1; t=1667340111; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc; bh=aR/o76RZJrriCzK/o0YOiUH8g7VYi4eVY0IxbRtzN0g=; b=gGunY/u/7/QA2xP1vKMH8MpblTxKRh0IVM2z3JI/rPet/cCv5lun7kFVoYI/N3N7oOdzQs e3zT0yolg5j05e+WuTPj05UK40+bfJMHuQexa34svu9FYEuwN6kUjcC3A7dZRJ1WcNBu01 Fvl2329MZZgepTwEnS1W9EUUgGdrTa53ROmsfFyhq1ztaRxs+u7jjvqMmQP4zoLp4pm/pz EzJ3K+Sa91Ep+5W8o9oj5phovAs8Qjl1QFw/xCcE9n55BfRN2gTavfavKdrpbYZ6K+gwm/ E0bEqFNfBEiQi8oJ8RvhbNPqk2/7RlZXJ0cu4HCLmVVW0FZeg0wb0ob3a2dYMg== Received: from matt by naz with local (Exim 4.96) (envelope-from <matt@HIDDEN>) id 1opzKO-000H38-1j for control <at> debbugs.gnu.org; Tue, 01 Nov 2022 15:01:48 -0700 To: control <at> debbugs.gnu.org From: Matt Armstrong <matt@HIDDEN> Subject: control message for bug #58929 Message-Id: <E1opzKO-000H38-1j@naz> Date: Tue, 01 Nov 2022 15:01:48 -0700 X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) tags 58929 + patch quit
X-Loop: help-debbugs@HIDDEN
Subject: bug#58929: 29.0.50; Calc: finding roots utpn doesn't work as advertised
Resent-From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= <mattiase@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 03 Nov 2022 08:54:01 +0000
Resent-Message-ID: <handler.58929.B58929.166746559532501 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 58929
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: patch
To: Matt Armstrong <matt@HIDDEN>
Cc: 58929 <at> debbugs.gnu.org
Received: via spool by 58929-submit <at> debbugs.gnu.org id=B58929.166746559532501
(code B ref 58929); Thu, 03 Nov 2022 08:54:01 +0000
Received: (at 58929) by debbugs.gnu.org; 3 Nov 2022 08:53:15 +0000
Received: from localhost ([127.0.0.1]:47814 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1oqVyM-0008S9-Gd
for submit <at> debbugs.gnu.org; Thu, 03 Nov 2022 04:53:14 -0400
Received: from mail1475c50.megamailservers.eu ([91.136.14.75]:53854
helo=mail118c50.megamailservers.eu)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mattiase@HIDDEN>) id 1oqVyK-0008Rn-Qp
for 58929 <at> debbugs.gnu.org; Thu, 03 Nov 2022 04:53:14 -0400
X-Authenticated-User: mattiase@HIDDEN
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=megamailservers.eu;
s=maildub; t=1667465585;
bh=3H/TRIx62B1I2//tAhGCdpp1QYXDCtYh7SOzW7bKErQ=;
h=Subject:From:In-Reply-To:Date:Cc:References:To:From;
b=Wrv5K7DkIgDE8vUefniwhpK1jpaXtzo5i9mA9xsM1bjdVXxo9e12HUuJ5oDqC2r1N
QFW7//euAd4hJVqLyNCxByVpVg6wm2yGYsC7OxsfaX3iMf+1JT6ArodLSP0SF3aTSa
SyXwi4ZcxTtLEPh7dUzuqWqAXzTVFsWP/9L6++4A=
Feedback-ID: mattiase@HIDDEN
Received: from smtpclient.apple (c188-150-171-209.bredband.tele2.se
[188.150.171.209]) (authenticated bits=0)
by mail118c50.megamailservers.eu (8.14.9/8.13.1) with ESMTP id 2A38r3Bi059627;
Thu, 3 Nov 2022 08:53:05 +0000
Content-Type: text/plain;
charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\))
From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= <mattiase@HIDDEN>
In-Reply-To: <87k04es5bt.fsf@HIDDEN>
Date: Thu, 3 Nov 2022 09:53:02 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <EA909128-24C5-412B-B609-19C200D15E6D@HIDDEN>
References: <8773828B-3A5F-4B15-8786-91F6FCB6D2E2@HIDDEN>
<87k04es5bt.fsf@HIDDEN>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-CTCH-RefID: str=0001.0A782F18.63638171.006B, ss=1, re=0.000, recu=0.000,
reip=0.000, cl=1, cld=1, fgs=0
X-CTCH-VOD: Unknown
X-CTCH-Spam: Unknown
X-CTCH-Score: 0.000
X-CTCH-Rules:
X-CTCH-Flags: 0
X-CTCH-ScoreCust: 0.000
X-Origin-Country: SE
X-Spam-Score: 0.3 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)
1 nov. 2022 kl. 23.00 skrev Matt Armstrong <matt@HIDDEN>:
> See attached patch for what I wish the docs had said all along
> (would have saved me a lot of time).
Thank you, I have not worked through your table to confirm all the =
details but it looks fine on the whole.
One detail: it's not just that the functions are "flat" (ie, all =
derivatives rapidly vanishing as we stray from the mean) but at least =
for the normal distribution, Newton actively diverges with a bad =
starting guess even if computed with infinite precision (draw the =
graph).
> I wonder if the implementation can be improved.
> =
https://www.gnu.org/software/emacs/manual/html_node/calc/Root-Finding.html=
> suggests that there are some cases where calc will switch over to the
> bisection method. Perhaps the heuristics can be tweaked?
=46rom what I can tell, bisection is used if there is an interval =
bracketing the root and either no derivative could be computed or Newton =
fails to converge. Clearly Newton is preferable when it converges since =
it's much faster than bisection.
We could employ special tactics by pattern-matching the function =
expression (perhaps according to your table); not sure if it would be =
worth the trouble.
Of course an actual numerical analyst would know immediately what to do. =
Is there one in the audience?
X-Loop: help-debbugs@HIDDEN
Subject: bug#58929: 29.0.50; Calc: finding roots utpn doesn't work as advertised
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 03 Nov 2022 09:15:01 +0000
Resent-Message-ID: <handler.58929.B58929.16674668442526 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 58929
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: patch
To: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= <mattiase@HIDDEN>
Cc: matt@HIDDEN, 58929 <at> debbugs.gnu.org
Received: via spool by 58929-submit <at> debbugs.gnu.org id=B58929.16674668442526
(code B ref 58929); Thu, 03 Nov 2022 09:15:01 +0000
Received: (at 58929) by debbugs.gnu.org; 3 Nov 2022 09:14:04 +0000
Received: from localhost ([127.0.0.1]:47971 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1oqWIW-0000eg-6N
for submit <at> debbugs.gnu.org; Thu, 03 Nov 2022 05:14:04 -0400
Received: from eggs.gnu.org ([209.51.188.92]:48190)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1oqWIU-0000e5-0j
for 58929 <at> debbugs.gnu.org; Thu, 03 Nov 2022 05:14:02 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1oqWIO-0000kM-Dp; Thu, 03 Nov 2022 05:13:56 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From:
Date; bh=HmRX4UTcU6ZyhHcPxK4Ovk7boOJ74vw5QpUFOdXI+RY=; b=JuyfATZgfWr9FtF572k/
ihoJP5otF6LO5ZFJbbVvEn2zBIGQWLh35TKDpBugYZ6h1raAJOZat5mM7v4GSM6nCKhyMSE4J5z9i
GxfWscxDe8dZsIzwrzuy1zhnPpEgwiZx7WY0UfAnFJFuzak+WW6JPY0QRB7h/imKqJMRXC6mX2s92
APznLosKFuF0R+z1TiiUh8yl7Qa/cTsmn5qaE2G73itFjFUwTuCKAAntVHczvyisUFptITlIdcSSP
EMpdYJijitR/oRhOdYRkKAijoC72xaRVoKT3qu0TB7Zx3mTyl69jawpZPS8X3h7IaWc0t1XQIT49f
6tYqbBs2c0wOHg==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1oqWIN-0004me-Ln; Thu, 03 Nov 2022 05:13:56 -0400
Date: Thu, 03 Nov 2022 11:13:46 +0200
Message-Id: <83a6588kol.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <EA909128-24C5-412B-B609-19C200D15E6D@HIDDEN> (message from
Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= on Thu, 3 Nov 2022 09:53:02 +0100)
References: <8773828B-3A5F-4B15-8786-91F6FCB6D2E2@HIDDEN>
<87k04es5bt.fsf@HIDDEN> <EA909128-24C5-412B-B609-19C200D15E6D@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Cc: 58929 <at> debbugs.gnu.org
> From: Mattias Engdegård <mattiase@HIDDEN>
> Date: Thu, 3 Nov 2022 09:53:02 +0100
>
> Of course an actual numerical analyst would know immediately what to do. Is there one in the audience?
I have good experience from using this technique:
https://www.researchgate.net/publication/226830173_On_the_Structure_of_Zero_Finders
It employs a hybrid method that doesn't need derivatives (but is
almost as fast as Newton). However, its main advantage (which is a
huge one in some real-life situations) is that the structure of the
algorithm allows the caller to make arbitrary modifications to the
approximations that the algorithm produces, and in general have
complete control on the iterations' process. In particular, one can
easily deal with situations where the root approximation goes out of
the expected range, or lands in the area where the function doesn't
behave well enough.
X-Loop: help-debbugs@HIDDEN
Subject: bug#58929: 29.0.50; Calc: finding roots utpn doesn't work as advertised
Resent-From: Matt Armstrong <matt@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 05 Nov 2022 19:30:01 +0000
Resent-Message-ID: <handler.58929.B58929.166767656822916 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 58929
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: patch
To: Eli Zaretskii <eliz@HIDDEN>, Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= <mattiase@HIDDEN>
Cc: 58929 <at> debbugs.gnu.org
Received: via spool by 58929-submit <at> debbugs.gnu.org id=B58929.166767656822916
(code B ref 58929); Sat, 05 Nov 2022 19:30:01 +0000
Received: (at 58929) by debbugs.gnu.org; 5 Nov 2022 19:29:28 +0000
Received: from localhost ([127.0.0.1]:58089 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1orOr7-0005xV-I5
for submit <at> debbugs.gnu.org; Sat, 05 Nov 2022 15:29:28 -0400
Received: from relay7-d.mail.gandi.net ([217.70.183.200]:56631)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <matt@HIDDEN>) id 1orOr2-0005xE-E4
for 58929 <at> debbugs.gnu.org; Sat, 05 Nov 2022 15:29:23 -0400
Received: (Authenticated sender: matt@HIDDEN)
by mail.gandi.net (Postfix) with ESMTPSA id 2168820004;
Sat, 5 Nov 2022 19:29:12 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rfc20.org; s=gm1;
t=1667676554;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:cc:mime-version:mime-version:content-type:content-type:
content-transfer-encoding:content-transfer-encoding:
in-reply-to:in-reply-to:references:references;
bh=iN37Y3UB4CqKgDh04h4PIpLlwmKJsahGuqffc25ef0c=;
b=il1Wj/JDPjIlBHsXKssXdMfpOAxtRAls+ld6WVUZHQSK8I09MdrxRD/CKgblckKiMSxc6Y
1Mzdaxmm3oUWGFRwiKVufpiyx4zXe4aYDrk33BlTZ/KUgBm35F2QRibacCNfGfKlJQTFZc
OeNWygE9sGHkJSrHWUP7WTE/jqqbQHgOBk3+vMBQ2LZK6n5GBpLLTFW4Z5cwCgL7Z7w+gB
V3YovRVipOvVXATZNSjkdjMOFWFVf0rAEMNjpEQ3APg4HnTcMjReZsLVX6n+RprPcw08at
rb/5Z6IILWFuLwuy2Q2dWkfnTiUb3pc+s5xnC+1e+Y/Z6GsMGXnYDB2TUQofyA==
Received: from matt by naz with local (Exim 4.96)
(envelope-from <matt@HIDDEN>) id 1orOqs-000E9o-2t;
Sat, 05 Nov 2022 12:29:10 -0700
From: Matt Armstrong <matt@HIDDEN>
In-Reply-To: <83a6588kol.fsf@HIDDEN>
References: <8773828B-3A5F-4B15-8786-91F6FCB6D2E2@HIDDEN>
<87k04es5bt.fsf@HIDDEN> <EA909128-24C5-412B-B609-19C200D15E6D@HIDDEN>
<83a6588kol.fsf@HIDDEN>
Date: Sat, 05 Nov 2022 12:29:10 -0700
Message-ID: <878rkpci9l.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
Eli Zaretskii <eliz@HIDDEN> writes:
>> Cc: 58929 <at> debbugs.gnu.org
>> From: Mattias Engdeg=C3=A5rd <mattiase@HIDDEN>
>> Date: Thu, 3 Nov 2022 09:53:02 +0100
>>=20
>> Of course an actual numerical analyst would know immediately what to do.=
Is there one in the audience?
>
> I have good experience from using this technique:
>
> https://www.researchgate.net/publication/226830173_On_the_Structure_of_=
Zero_Finders
>
> It employs a hybrid method that doesn't need derivatives (but is
> almost as fast as Newton). However, its main advantage (which is a
> huge one in some real-life situations) is that the structure of the
> algorithm allows the caller to make arbitrary modifications to the
> approximations that the algorithm produces, and in general have
> complete control on the iterations' process. In particular, one can
> easily deal with situations where the root approximation goes out of
> the expected range, or lands in the area where the function doesn't
> behave well enough.
Interesting. I like how that algorithm attempts to "do its best" when
given a single guess point, then adapts to bisection if it happens to
find an opportunity and need. Calc's manual suggests that its "root"
function attempts to do the same, opening a window to use this other
approach.
I'm not sure how to expose the root finding "loop" in calc's UI, but
exposing it calc's lisp level interface would be a natural thing to do.
https://en.wikipedia.org/wiki/ITP_method also looks like an interesting
twist on the bisection/secant method. I have no personal experience
with it, and on Wikipedia I always wonder if pages like this are
reflective of a true utility or merely the ambitions of the paper's
authors! This function needs function values that are opposite in sign
before it is useful. It might be possible to incorporate it into the
approach Eli linked to, but that method claims to be almost as fast as
Newton's method in most cases so it may not need the improvement.
I arrived here because I wanted to use calc to compute 95th and 99th
percentile confidence intervals of some benchmarking results using the
Student's t-distribution. I have figured this out now, but it has been
more difficult than I anticipated, and the manual's over-promise with
respect to "find the root, any guess works" sent me on a long tangent
trying to figure out what I had done wrong. That said, my background in
this area is weak, so I don't think I am the right person to improve
Emacs' root finding implementation.
The patch I sent previously, with my (hopefully) useful suggestions in
the user manual, is probably the end of my useful contribution here.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.