GNU logs - #42669, boring messages

Message sent to bug-guile@HIDDEN:

Subject: bug#42669: [PATCH 0/4] GNU Mcron and the (ice-9 getopt-long) module
Resent-From: Dale Mellor <guile-qf1qmg@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guile@HIDDEN
Resent-Date: Sun, 02 Aug 2020 09:20:02 +0000
Resent-Message-ID: <handler.42669.B.159635995313310 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 42669 <at> debbugs.gnu.org
Message-ID: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
From: Dale Mellor <guile-qf1qmg@HIDDEN>
Date: Sun, 02 Aug 2020 10:19:03 +0100
Organization: DM Bespoke Computer Solutions Ltd
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.30.5-1.1 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Received-SPF: softfail client-ip=52.19.174.175;
 envelope-from=guile-qf1qmg@HIDDEN; helo=rdmp.org
X-Spam_score_int: -4
X-Spam_score: -0.5
X-Spam_bar: /
X-Spam_report: (-0.5 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_SOFTFAIL=0.732,
 SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

/Mcron/ is a GNU package which runs unattended jobs in the operating
system at dynamically computed times; it is 99% Guile but currently
shrouded in a thin veneer of C code for historical reasons, which
have by now vanished.

The Guile /getopt-long/ module parses a command lineʼs arguments for
options and their values according to a provided grammar.

In the process of removing the thin veneer of C code from around the
/GNU Mcron/ package, I am running up against niggles in the
implementation of the /(ice-9 getopt-long)/ module.  The intention
with /mcron/ has always been that a command-line argument be
provided which allows the user to request the display of the next
eight jobs to run, or allows the user to specify the number of such
jobs.  Thus the intention was that command-lines like ‘mcron -s4
file’, ‘mcron -s 4 file’, and ‘mcron -s file’ would all work; alas,
the last one, actually the most important case, doesnʼt with the
current module, which issues a fatal exit on the grounds that ‘file’
fails to meet predicated requirements of the option for ‘-s’ that it
should represent a decimal number.

It is clear that /getopt-long/ can do better than this, especially
if the consumer of the module provides predicates on values which
options can take, e.g. value should be numerical.  It can then
objectively decide that an argument should be taken to be a value,
an option itself, or a ‘loose’ argument.

There are other problems which can be cleared up with the enhanced
logic, as outlined in Point 2 below.

The following patches clear up the situation.

1) The first patch introduces some 28 new tests of the existing
   /getopt-long/ module; these are non-controversial and the current
   code passes all the tests, but they exercise more of the corner
   cases and provide confidence that a new implementation does not
   break existing behaviour.

2) The second patch inverts one test which I disagree with (see
   Point 3, below), and introduces 18 more tests which represent
   currently indeterminate and unsupported behaviour, some
   nevertheless desired by /mcron/; all of these create either test
   FAIL cases with the current code-base, or total panic-escape from
   the calling application.

   Some specific test failures:

   1. A command-line like ‘foo --test=’ produces a /test/ result with
      the empty string as value; I would expect /#t/ as the value
      (which indicates that the option is there but has no given
      value).

   2. A command-line with a negative number always errors.  According
      to the in-line documentation negative numbers canʼt ever appear
      loose on the command-line, but this seems like a case which
      might be realistic in real life and there is no reason to
      reject them.

   3. A command like ‘foo -abc d’ in which /b/ takes a mandatory
      argument and /c/ is an allowed option, errors out, but in my
      opinion in this case /b/ should take “c” as its value and the
      command-line as a whole is *not* erroneous.  If /b/ takes an
      optional value things are more tricky to deal with, but if
      there is a predicate on the values which /b/ can take, then the
      parser can make a clearer decision on taking /c/ as a value or
      another option.

      This might seem picky, but the problem is that command-lines
      are supplied by (possibly hostile) end-users, *not* by the
      /getopt-long/ module, and not by the application which consumes
      the module, either.  Thus this might be regarded as a security
      issue.

   4. The command ‘mcron -s file’, where /s/ takes an optional
      numeric value, errors out.

3) The third patch fixes up the /getopt-long/ module to pass all the
   new tests, as well as all of the existing ones (with the single
   exception outlined in Point 2.3 above).  Considering that the
   entire Guile build also depends on /getopt-long/, we can have
   some confidence that the changes do not bring any incompatibility
   with existing code.

4) The final patch fixes up various commentary and doc-strings in
   the code to emphasise the importance of predicates on optional
   values, and generally make things more concrete.

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Dale Mellor <guile-qf1qmg@HIDDEN>
Subject: bug#42669: Acknowledgement ([PATCH 0/4] GNU Mcron and the (ice-9
 getopt-long) module)
Message-ID: <handler.42669.B.159635995313310.ack <at> debbugs.gnu.org>
References: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
X-Gnu-PR-Message: ack 42669
X-Gnu-PR-Package: guile
X-Gnu-PR-Keywords: patch
Reply-To: 42669 <at> debbugs.gnu.org
Date: Sun, 02 Aug 2020 09:20:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-guile@HIDDEN

If you wish to submit further information on this problem, please
send it to 42669 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
42669: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D42669
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-guile@HIDDEN:

Subject: bug#42669: [PATCH 1/4] test: augment testing of (ice-9 getopt-long) module
References: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
In-Reply-To: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
Resent-From: Dale Mellor <guile-qf1qmg@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guile@HIDDEN
Resent-Date: Sun, 02 Aug 2020 10:33:02 +0000
Resent-Message-ID: <handler.42669.B42669.15963643754772 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 42669 <at> debbugs.gnu.org
Message-ID: <915523a92e868b15282b3cbfa9d6d84a8698a043.camel@HIDDEN>
From: Dale Mellor <guile-qf1qmg@HIDDEN>
Date: Sun, 02 Aug 2020 11:32:45 +0100
Organization: DM Bespoke Computer Solutions Ltd
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.30.5-1.1 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Adding some 28 new tests which explore some undefined (or at least
implied) behaviour of the module.  These are all non-controversial, and
the existing module passes all of the tests.

* test-suite/tests/getopt-long.test: new code added, some slight
  re-arrangement of existing code but nothing which changes the original
  set of tests.
---
 test-suite/tests/getopt-long.test | 184 +++++++++++++++++++++++++++++-
 1 file changed, 182 insertions(+), 2 deletions(-)

diff --git a/test-suite/tests/getopt-long.test b/test-suite/tests/getopt-long.test
index 4ae604883..d66de0e56 100644
--- a/test-suite/tests/getopt-long.test
+++ b/test-suite/tests/getopt-long.test
@@ -1,5 +1,4 @@
 ;;;; getopt-long.test --- long options processing -*- scheme -*-
-;;;; Thien-Thi Nguyen <ttn@HIDDEN> --- August 2001
 ;;;;
 ;;;; 	Copyright (C) 2001, 2006, 2011 Free Software Foundation, Inc.
 ;;;;
@@ -17,10 +16,17 @@
 ;;;; License along with this library; if not, write to the Free Software
 ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
 
+;;; Author:  Thien-Thi Nguyen <ttn@HIDDEN> --- August 2001
+;;;          Dale Mellor --- April 2020
+
+
 (use-modules (test-suite lib)
              (ice-9 getopt-long)
              (ice-9 regex))
 
+
+;;********  Test infrastructure   *********************
+
 (define-syntax pass-if-fatal-exception
   (syntax-rules ()
     ((_ name exn exp)
@@ -49,6 +55,44 @@
 (deferr option-must-be-specified    "option must be specified")
 (deferr option-must-have-arg        "option must be specified with argument")
 
+
+
+;;*************  Newer test infrastructure  ***********************
+
+;; Many tests here are somewhat flakey as they depend on a precise
+;; internal representation of the options analysis, which isn't really
+;; defined or necessary.  In the newer tests below we sort that
+;; structure into alphabetical order, so we know exactly in advance how
+;; to specify the expected results.  We also make the test inputs
+;; strings of command-line options, rather than lists, as these are
+;; clearer and easier for us and closer to the real world.
+
+(define* (A-TEST args option-specs expectation
+                 #:key stop-at-first-non-option)
+
+  (define  (symbol/>string a)
+    (if (symbol? a) (symbol->string a) ""))
+
+  (define (output-sort out)
+    (sort out (λ (a b) (string<? (symbol/>string (car a))
+                                 (symbol/>string (car b))))))
+  
+  (let ((answer
+         (output-sort
+          (getopt-long
+           (cons "foo" (string-split args #\space))
+           option-specs
+           #:stop-at-first-non-option  stop-at-first-non-option))))
+    (cond ((equal? answer (output-sort expectation)) #t)
+          (else (format (current-output-port)
+                        "Test result was \n‘~s’ --VS-- \n‘~s’.\n"
+                        answer (output-sort expectation))
+                #f))))
+
+
+
+;;************   The tests  ******************************
+
 (with-test-prefix "exported procs"
   (pass-if "`option-ref' defined"  (defined? 'option-ref))
   (pass-if "`getopt-long' defined" (defined? 'getopt-long)))
@@ -120,7 +164,12 @@
            (equal? (test3 "prg" "--bar" "--foo")
                    '((()) (foo . #t) (bar . #t))))
 
-  )
+  (pass-if "--="
+           (equal? (test3 "prg" "--=")
+                   '((() "--="))))
+
+ )
+
 
 (with-test-prefix "option-ref"
 
@@ -299,4 +348,135 @@
 
   )
 
+
+
+(with-test-prefix "stop at end-of-options marker"
+
+  (define* (test args expectation #:key stop-at-first-non-option)
+    (A-TEST args
+            '((abby) (ben) (charles))
+            expectation
+            #:stop-at-first-non-option stop-at-first-non-option))
+
+  (pass-if "stop at start" (test "-- --abby" '((() "--abby"))))
+
+  (pass-if "stop in middle" (test "--abby dave -- --ben"
+                                  '((() "dave" "--ben") (abby . #t))))
+
+  (pass-if "stop at end" (test "--abby dave --ben --"
+                               '((() "dave") (abby . #t) (ben . #t))))
+
+  (pass-if "marker before first non-option"
+    (test "--abby -- --ben dave --charles"
+          '((() "--ben" "dave" "--charles") (abby . #t))     
+          #:stop-at-first-non-option #t))
+
+  (pass-if "double end marker"
+    (test "--abby -- -- --ben"
+          '((() "--" "--ben") (abby . #t))))
+
+  (pass-if "separated double end markers"
+    (test "--abby dave -- --ben -- --charles"
+          '((() "dave" "--ben" "--" "--charles")
+            (abby . #t))))
+  )
+
+
+
+(with-test-prefix "negative numbers for option values"
+
+  (define (test args expectation)
+    (A-TEST args
+            `((arthur (single-char #\a) (value optional)
+                      (predicate ,string->number))
+              (beth (single-char #\b) (value #t)
+                    (predicate ,string->number))
+              (charles (single-char #\c) (value optional))
+              (dave (single-char #\d) (value #t)))
+            expectation))
+  
+  (pass-if "predicated --optional=-1"
+    (test "--arthur=-1" '((()) (arthur . "-1"))))
+
+  (pass-if "predicated -o-1"
+    (test "-a-1" '((()) (arthur . "-1"))))
+
+  (pass-if "predicated --optional -1"
+    (test "--arthur -1" '((()) (arthur . "-1"))))
+
+  (pass-if "predicated -o -1"
+    (test "-a -1" '((()) (arthur . "-1"))))
+
+  (pass-if "predicated --mandatory=-1"
+    (test "--beth=-1" '((()) (beth . "-1"))))
+
+  (pass-if "predicated -m-1"
+    (test "-b-1" '((()) (beth . "-1"))))
+
+  (pass-if "predicated --mandatory -1"
+    (test "--beth -1" '((()) (beth . "-1"))))
+
+  (pass-if "predicated -m -1"
+    (test "-b -1" '((()) (beth . "-1"))))
+  
+  (pass-if "non-predicated --optional=-1"
+    (test "--charles=-1" '((()) (charles . "-1"))))
+
+  (pass-if "non-predicated -o-1"
+    (test "-c-1" '((()) (charles . "-1"))))
+
+  (pass-if "non-predicated --mandatory=-1"
+    (test "--dave=-1" '((()) (dave . "-1"))))
+
+  (pass-if "non-predicated -m-1"
+    (test "-d-1" '((()) (dave . "-1"))))
+
+  (pass-if "non-predicated --mandatory -1"
+    (test "--dave -1" '((()) (dave . "-1"))))
+
+  (pass-if "non-predicated -m -1"
+    (test "-d -1" '((()) (dave . "-1"))))
+
+  )
+
+
+
+(with-test-prefix "mcron backwards compatibility"
+
+  (define (test args expectation)
+    (A-TEST args
+            `((daemon   (single-char #\d) (value #f))
+              (stdin    (single-char #\i) (value #t)
+                        (predicate ,(λ (in) (or (string=? in "guile")
+                                                (string=? in "vixie")))))
+              (schedule (single-char #\s) (value optional)
+                        (predicate ,(λ (in) (or (eq? in #t)
+                                                (and (string? in)
+                                                     (string->number in))))))
+              (help     (single-char #\?))
+              (version  (single-char #\V)))
+            expectation))
+  
+  (pass-if  "-s8"   (test  "-s8 file"  '((() "file") (schedule . "8"))))
+
+  (pass-if  "-s 8"  (test "-s 8 file"  '((() "file") (schedule . "8"))))
+
+  (pass-if  "-sd file"
+    (test  "-sd file"   '((() "file") (daemon . #t) (schedule . #t))))
+
+  (pass-if  "--schedule=8"  (test  "--schedule=8 file"
+                                   '((() "file") (schedule . "8"))))
+
+  (pass-if  "--schedule 8"  (test "--schedule 8 file"
+                                  '((() "file") (schedule . "8"))))
+
+  (pass-if  "-ds8"   (test  "-ds8 file"
+                            '((() "file") (daemon . #t) (schedule . "8"))))
+
+  (pass-if  "-ds 8"  (test "-ds 8 file"
+                           '((() "file") (daemon . #t) (schedule . "8"))))
+
+  )
+                 
+
 ;;; getopt-long.test ends here
-- 
2.27.0

Message sent to bug-guile@HIDDEN:

Subject: bug#42669: [PATCH 2/4 (v2)] test *broken*: augmented tests of (ice-9 getopt-long)
References: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
In-Reply-To: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
Resent-From: Dale Mellor <guile-qf1qmg@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guile@HIDDEN
Resent-Date: Sun, 02 Aug 2020 10:35:01 +0000
Resent-Message-ID: <handler.42669.B42669.15963644584908 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 42669 <at> debbugs.gnu.org
Message-ID: <842cf4149fa52b03ebcfd3d202968f873e920efe.camel@HIDDEN>
From: Dale Mellor <guile-qf1qmg@HIDDEN>
Date: Sun, 02 Aug 2020 11:34:10 +0100
Organization: DM Bespoke Computer Solutions Ltd
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.30.5-1.1 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

This is to prepare the ground for some test-driven development mainly to
make the module satisfy the needs of the GNU Mcron project.  The main
requirement is for the module to be more intelligent when dealing with
optional values to command-line options: if the following argument looks
like a new option then treat it as such, otherwise treat it as the value
of the current option.  The particular case is mcronʼs -s option which
needs to assume a default value of “8” if there is not one on the
command line, but currently ‘mcron -s input_file’ fails badly.

Other tests introduced involve allowing negative numbers as option
values, and dealing with various cases of option-processing termination.

* test-suite/tests/getopt-long.test: new code added.
---
 test-suite/tests/getopt-long.test | 114 ++++++++++++++++++++++++++++--
 1 file changed, 109 insertions(+), 5 deletions(-)

diff --git a/test-suite/tests/getopt-long.test b/test-suite/tests/getopt-long.test
index d66de0e56..589982381 100644
--- a/test-suite/tests/getopt-long.test
+++ b/test-suite/tests/getopt-long.test
@@ -164,6 +164,14 @@
            (equal? (test3 "prg" "--bar" "--foo")
                    '((()) (foo . #t) (bar . #t))))
 
+  (pass-if "long option with equals and space"
+           (equal? (test3 "prg" "--foo=" "test")
+                   '((() "test")  (foo . #t))))
+
+  (pass-if "long option with equals and space, not allowed a value"
+           (equal? (test3 "prg" "--foo=" "test")
+                   '((() "test") (foo . #t))))
+  
   (pass-if "--="
            (equal? (test3 "prg" "--=")
                    '((() "--="))))
@@ -295,9 +303,40 @@
   (pass-if "normal 2" (test8 "-ab" "bang" "-c" "couth"))
   (pass-if "normal 3" (test8 "-ac" "couth" "-b" "bang"))
 
-  (pass-if-fatal-exception "bad ordering causes missing option"
-                     exception:option-must-have-arg
-                     (test8 "-abc" "couth" "bang"))
+
+  ;;;; Dale Mellor 2020-04-14
+  ;;;;
+  ;;;;  I disagree with this test: to my mind 'c' is 'b's argument, and
+  ;;;;  the other two arguments are non-options which get passed
+  ;;;;  through; there should not be an exception.
+
+  ;; (pass-if-fatal-exception "bad ordering causes missing option"
+  ;;                          exception:option-must-have-arg
+  ;;                          (test8 "-abc" "couth" "bang"))
+
+  (pass-if "clumped options with trailing mandatory value"
+           (A-TEST "-abc couth bang"
+                   '((apples    (single-char #\a))
+                     (blimps    (single-char #\b) (value #t))
+                     (catalexis (single-char #\c) (value #t)))
+                   '((() "couth" "bang") (apples . #t) (blimps . "c"))))
+  
+  (pass-if  "clumped options with trailing optional value"
+            (A-TEST "-abc couth bang"
+                    '((apples (single-char #\a))
+                      (blimps (single-char #\b)
+                              (value optional)))
+                    '((() "couth" "bang") (apples . #t) (blimps . "c"))))
+
+  (pass-if  "clumped options with trailing optional value"
+            (A-TEST "-abc couth bang"
+                    '((apples (single-char #\a))
+                      (blimps (single-char #\b)
+                              (value optional))
+                      (catalexis (single-char #\c)
+                                 (value #t)))
+                    '((() "bang")
+                      (apples . #t) (blimps . #t) (catalexis . "couth"))))
 
   )
 
@@ -346,6 +385,12 @@
                          #:stop-at-first-non-option #t)
             '((() "compile" "-Wformat" "eval.scm" "-o" "eval.go"))))
 
+  (pass-if "stop after option"
+    (equal? (getopt-long '("foo" "-a" "3" "4" "-b" "4")
+                         '((about (single-char #\a) (value #t))
+                           (breathe (single-char #\b) (value #t)))
+                         #:stop-at-first-non-option #t)
+            '((() "4" "-b" "4") (about . "3"))))
   )
 
 
@@ -371,6 +416,11 @@
           '((() "--ben" "dave" "--charles") (abby . #t))     
           #:stop-at-first-non-option #t))
 
+  (pass-if "first non-option before marker"
+           (test "--abby dave --ben -- --charles"
+                 '((() "dave" "--ben" "--" "--charles")  (abby . #t))
+                 #:stop-at-first-non-option #t))
+
   (pass-if "double end marker"
     (test "--abby -- -- --ben"
           '((() "--" "--ben") (abby . #t))))
@@ -431,16 +481,64 @@
   (pass-if "non-predicated -m-1"
     (test "-d-1" '((()) (dave . "-1"))))
 
+(pass-if-fatal-exception  "non-predicated --optional -1"
+                            exception:no-such-option
+                            (test  "--charles -1"  '((()) (charles . "-1"))))
+
+  (pass-if-fatal-exception  "non-predicated -o -1"
+                            exception:no-such-option
+                            (test  "-c -1"  '((()) (charles . "-1"))))
+
+  (pass-if  "non-predicated --mandatory=-1"
+    (test  "--dave=-1"   '((()) (dave . "-1"))))
+
   (pass-if "non-predicated --mandatory -1"
     (test "--dave -1" '((()) (dave . "-1"))))
-
+  
   (pass-if "non-predicated -m -1"
     (test "-d -1" '((()) (dave . "-1"))))
-
+  
   )
 
 
 
+(with-test-prefix  "negative numbers as short options"
+
+  (define  (test  args  expectation)
+    (A-TEST  args
+             `((zero (single-char #\0) (value #f))
+               (one  (single-char #\1) (value #t)
+                     (predicate ,string->number))
+               (two  (single-char #\2) (value optional)
+                     (predicate ,string->number))
+               (three (single-char #\3) (value #t)
+                      (predicate ,(λ (in) (not (string->number in)))))
+               (four (single-char #\4) (value optional)
+                     (predicate ,(λ (in) (not (string->number in)))))
+               )
+             expectation))
+
+  (pass-if  "-0 -2"  (test "-0 -2" '((()) (zero . #t) (two . #t))))
+  (pass-if  "-1 -2"  (test "-1 -2"  '((()) (one . "-2"))))
+  (pass-if  "-2 -3"  (test "-2 -3"  '((()) (two . "-3"))))
+  (pass-if  "-0 -4 test"  (test "-0 -4 test"
+                                '((()) (zero . #t) (four . "test"))))
+  (pass-if  "-4 -2"  (test "-4 -2"  '((()) (four . #t) (two . #t))))
+  (pass-if-fatal-exception  "-4 -3"  exception:option-must-have-arg
+                            (test "-4 -3" '((()))))
+  (pass-if  "-3a"   (test "-3a"  '((()) (three . "a"))))
+  (pass-if  "-13"   (test "-13"  '((()) (one . "3"))))
+  (pass-if  "-03a"  (test "-03a" '((()) (zero . #t) (three . "a"))))
+  (pass-if  "-023"  (test "-023" '((()) (zero . #t) (two . "3"))))
+  (pass-if  "-025"  (test "-025" '((()) (zero . #t) (two . "5"))))
+  (pass-if-fatal-exception  "-025a"  exception:no-such-option
+                            (test "-025a" '((()) (zero . #t) (two . "5"))))
+  (pass-if  "-02 a" (test "-02 a" '((() "a") (zero . #t) (two . #t))))
+  (pass-if-fatal-exception  "-02a"  exception:no-such-option
+                            (test "-02a" '((()))))
+   )
+
+
 (with-test-prefix "mcron backwards compatibility"
 
   (define (test args expectation)
@@ -461,9 +559,15 @@
 
   (pass-if  "-s 8"  (test "-s 8 file"  '((() "file") (schedule . "8"))))
 
+  (pass-if  "-s file"
+            (test  "-s file"   '((() "file") (schedule . #t))))
+
   (pass-if  "-sd file"
     (test  "-sd file"   '((() "file") (daemon . #t) (schedule . #t))))
 
+  (pass-if  "-ds file"
+            (test  "-ds file"   '((() "file") (daemon . #t) (schedule . #t))))
+
   (pass-if  "--schedule=8"  (test  "--schedule=8 file"
                                    '((() "file") (schedule . "8"))))
 
-- 
2.27.0

Message sent to bug-guile@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#42669: [PATCH 3/4 (v2)] (ice-9 getopt-long): substantially re-written to pass all the new tests
References: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
In-Reply-To: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
Resent-From: Dale Mellor <guile-qf1qmg@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guile@HIDDEN
Resent-Date: Sun, 02 Aug 2020 10:36:02 +0000
Resent-Message-ID: <handler.42669.B42669.15963645395044 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 42669
X-GNU-PR-Package: guile
X-GNU-PR-Keywords: patch
To: 42669 <at> debbugs.gnu.org
Received: via spool by 42669-submit <at> debbugs.gnu.org id=B42669.15963645395044
          (code B ref 42669); Sun, 02 Aug 2020 10:36:02 +0000
Received: (at 42669) by debbugs.gnu.org; 2 Aug 2020 10:35:39 +0000
Received: from localhost ([127.0.0.1]:41767 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1k2BL8-0001JI-R9
	for submit <at> debbugs.gnu.org; Sun, 02 Aug 2020 06:35:39 -0400
Received: from rdmp.org ([52.19.174.175]:36130)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <guile-qf1qmg@HIDDEN>) id 1k2BL6-0001J5-Aj
 for 42669 <at> debbugs.gnu.org; Sun, 02 Aug 2020 06:35:37 -0400
Received: from [127.0.0.1] (helo=localhost) by rdmp.org with esmtp (Exim 4.92)
 (envelope-from <guile-qf1qmg@HIDDEN>) id 1k2BL0-0005uK-HV
 for 42669 <at> debbugs.gnu.org; Sun, 02 Aug 2020 10:35:30 +0000
Message-ID: <1b9343bcc12baad44fd35efa691b507ad0293d12.camel@HIDDEN>
From: Dale Mellor <guile-qf1qmg@HIDDEN>
Date: Sun, 02 Aug 2020 11:35:30 +0100
Organization: DM Bespoke Computer Solutions Ltd
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.30.5-1.1 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: 1.9 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  All of the original tests also still pass. Also the entire
 guile build actually depends on the correct functioning of this module, so
 we can be quite confident that nothing has been broken. *
 module/ice-9/getopt-long.scm:
 Substantially re-written. --- module/ice-9/getopt-long.scm | 459
 +++++++++++++++++++++++
 1 file changed, 309 insertions(+), 150 deletions(-) 
 Content analysis details:   (1.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 1.0 SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
 0.9 SPF_HELO_SOFTFAIL      SPF: HELO does not match SPF record (softfail)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.9 (/)

All of the original tests also still pass.  Also the entire guile build
actually depends on the correct functioning of this module, so we can be
quite confident that nothing has been broken.

* module/ice-9/getopt-long.scm: Substantially re-written.
---
 module/ice-9/getopt-long.scm | 459 +++++++++++++++++++++++------------
 1 file changed, 309 insertions(+), 150 deletions(-)

diff --git a/module/ice-9/getopt-long.scm b/module/ice-9/getopt-long.scm
index 14eaf8e23..06aa1a879 100644
--- a/module/ice-9/getopt-long.scm
+++ b/module/ice-9/getopt-long.scm
@@ -157,16 +157,15 @@
 ;;; Code:
 
 (define-module (ice-9 getopt-long)
-  #:use-module ((ice-9 common-list) #:select (remove-if-not))
+  #:use-module (ice-9 control)
+  #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9)
   #:use-module (ice-9 match)
   #:use-module (ice-9 regex)
-  #:use-module (ice-9 optargs)
+  #:use-module (ice-9 receive)
   #:export (getopt-long option-ref))
 
-(define %program-name (make-fluid "guile"))
-(define (program-name)
-  (fluid-ref %program-name))
+(define program-name (make-parameter "guile"))
 
 (define (fatal-error fmt . args)
   (format (current-error-port) "~a: " (program-name))
@@ -174,19 +173,16 @@
   (newline (current-error-port))
   (exit 1))
 
+;; name: string, required?: bool, single-char: char or #f, predicate:
+;; procedure or #f, value-policy: bool or 'optional.
 (define-record-type option-spec
-  (%make-option-spec name required? option-spec->single-char predicate value-policy)
+  (%make-option-spec name required? single-char predicate value-policy)
   option-spec?
-  (name
-   option-spec->name set-option-spec-name!)
-  (required?
-   option-spec->required? set-option-spec-required?!)
-  (option-spec->single-char
-   option-spec->single-char set-option-spec-single-char!)
-  (predicate
-   option-spec->predicate set-option-spec-predicate!)
-  (value-policy
-   option-spec->value-policy set-option-spec-value-policy!))
+  (name         option-spec->name)
+  (required?    option-spec->required?    set-option-spec-required?!)
+  (single-char  option-spec->single-char  set-option-spec-single-char!)
+  (predicate    option-spec->predicate    set-option-spec-predicate!)
+  (value-policy option-spec->value-policy set-option-spec-value-policy!))
 
 (define (make-option-spec name)
   (%make-option-spec name #f #f #f #f))
@@ -194,117 +190,292 @@
 (define (parse-option-spec desc)
   (let ((spec (make-option-spec (symbol->string (car desc)))))
     (for-each (match-lambda
-               (('required? val)
-                (set-option-spec-required?! spec val))
-               (('value val)
-                (set-option-spec-value-policy! spec val))
-               (('single-char val)
-                (or (char? val)
-                    (error "`single-char' value must be a char!"))
-                (set-option-spec-single-char! spec val))
-               (('predicate pred)
-                (set-option-spec-predicate!
-                 spec (lambda (name val)
-                        (or (not val)
-                            (pred val)
-                            (fatal-error "option predicate failed: --~a"
-                                         name)))))
-               ((prop val)
-                (error "invalid getopt-long option property:" prop)))
-              (cdr desc))
+                (('required? val)
+                 (set-option-spec-required?! spec val))
+                (('value val)
+                 (set-option-spec-value-policy! spec val))
+                (('single-char val)
+                 (unless (char? val)
+                   (fatal-error "‘single-char’ value must be a char!"))
+                 (set-option-spec-single-char! spec val))
+                (('predicate pred)
+                 (set-option-spec-predicate! spec pred))
+                ((prop val)
+                 (fatal-error "invalid getopt-long option property: " prop)))
+               (cdr desc))
     spec))
 
-(define (split-arg-list argument-list)
-  ;; Scan ARGUMENT-LIST for "--" and return (BEFORE-LS . AFTER-LS).
-  ;; Discard the "--".  If no "--" is found, AFTER-LS is empty.
-  (let loop ((yes '()) (no argument-list))
-    (cond ((null? no)               (cons (reverse yes) no))
-	  ((string=? "--" (car no)) (cons (reverse yes) (cdr no)))
-	  (else (loop (cons (car no) yes) (cdr no))))))
-
-(define short-opt-rx           (make-regexp "^-([a-zA-Z]+)(.*)"))
-(define long-opt-no-value-rx   (make-regexp "^--([^=]+)$"))
-(define long-opt-with-value-rx (make-regexp "^--([^=]+)=(.*)"))
-
-(define (looks-like-an-option string)
-  (or (regexp-exec short-opt-rx string)
-      (regexp-exec long-opt-with-value-rx string)
-      (regexp-exec long-opt-no-value-rx string)))
-
-(define (process-options specs argument-ls stop-at-first-non-option)
-  ;; Use SPECS to scan ARGUMENT-LS; return (FOUND . ETC).
-  ;; FOUND is an unordered list of option specs for found options, while ETC
-  ;; is an order-maintained list of elements in ARGUMENT-LS that are neither
-  ;; options nor their values.
-  (let ((idx (map (lambda (spec)
-                    (cons (option-spec->name spec) spec))
-                  specs))
-        (sc-idx (map (lambda (spec)
-                       (cons (make-string 1 (option-spec->single-char spec))
-                             spec))
-                     (remove-if-not option-spec->single-char specs))))
-    (let loop ((unclumped 0) (argument-ls argument-ls) (found '()) (etc '()))
-      (define (eat! spec ls)
-        (cond
-         ((eq? 'optional (option-spec->value-policy spec))
-          (if (or (null? ls)
-                  (looks-like-an-option (car ls)))
-              (loop (- unclumped 1) ls (acons spec #t found) etc)
-              (loop (- unclumped 2) (cdr ls) (acons spec (car ls) found) etc)))
-         ((eq? #t (option-spec->value-policy spec))
-          (if (or (null? ls)
-                  (looks-like-an-option (car ls)))
-              (fatal-error "option must be specified with argument: --~a"
-                           (option-spec->name spec))
-              (loop (- unclumped 2) (cdr ls) (acons spec (car ls) found) etc)))
+
+;; Extract the name of a long option given that it may or may not be
+;; surrounded by '--' and '=...'.
+(define isolate-long-name-re (make-regexp "^(--)?([^=]+)"))
+
+(define (isolate-long-name name)
+  (and=> (regexp-exec isolate-long-name-re name)
+    (λ (match) (match:substring match 2))))
+
+
+;;  Whatever the presentation of the long option, make sure it is in a
+;;  clean, normalized form (but this does NOT account for any value the
+;;  option might have).
+(define (re-present option)
+  (string-append "--" (isolate-long-name option) "="))
+
+
+;;  The /name/ can take the form of a long option entry on the command
+;;  line, with whatever decoration that entails.  Will return #f if a
+;;  spec does not exist for this named option.
+(define (find-spec-long specs name)
+  (and=> (isolate-long-name name)
+    (λ (name) (find (λ (a) (string=? (option-spec->name a) name)) specs))))
+
+
+;;  Return #f if a spec with the short /letter/ name does not exist.
+(define (find-spec-short specs letter)
+  (find (λ (a) (eqv? (option-spec->single-char a) letter)) specs))
+
+
+;;  Take, for example, /short/='-h' to '--help='.
+(define (double-up short specs)
+  (and=> (find-spec-short specs (string-ref short 1))
+    (λ (spec) (string-append "--" (option-spec->name spec) "="))))
+
+
+;;  This procedure does whatever is necessary to put the (ostensibly)
+;;  first item on the command-line into the canonical (normal) form
+;;  '--item=value'; this may mean consuming the next item of the
+;;  command-line (the first item of /rest/) to get the value.  Note that
+;;  the value may be missing, but the '=' sign will always be there in
+;;  the return.  The first item (/A/) will always be more than two
+;;  characters, and the first two characters will be "--", i.e. we are
+;;  processing a long option.
+;;
+;;  A          IN   string               The first argument on the command-line
+;;  rest       IN   list of strings      The remaining items of the command-line
+;;  specs      IN   list of option-spec  Options specification
+;;  remnant    OUT  list of strings      The unprocessed command line
+;;  processed  OUT  string               New command-line argument
+(define (normalize-long-option A rest specs)
+  (define (return-empty-arg) (values rest (re-present A)))
+  (define (return-arg-with-value)
+    (values (cdr rest) (string-append (re-present A) (car rest))))
+  (cond
+   ((string-index A #\=)
+    ;; The argument is already in the canonical form.
+    (values rest A))
+   ((find-spec-long specs A)
+    => (λ (spec)
+          (if (cond ((null? rest) #f)
+                    ((option-spec->predicate spec)
+                     => (λ (pred) (pred (car rest))))
+                    (else
+                      (case (option-spec->value-policy spec)
+                        ((#f) #f)
+                        ((optional) 
+                         (not (eqv? (string-ref (car rest) 0) #\-)))
+                        ((#t)
+                         (or (string->number (car rest))
+                             (not (eqv? (string-ref (car rest) 0) #\-)))))))
+              (return-arg-with-value)
+              (return-empty-arg))))
+   (else
+    ;; We know nothing about this option, abort operations.
+    (fatal-error "no such option: --~a" (isolate-long-name A)))))
+
+
+
+;;  This procedure does whatever is necessary to put the (ostensibly)
+;;  first item on the command-line into the canonical form
+;;  '--item=value'; this may mean consuming the next item of the
+;;  command-line (the first item of /rest/) to get the value.  Note that
+;;  the value may be missing, but the '=' sign will always be there in
+;;  the return.  The first item (/A/) will always be exactly two
+;;  characters, and the first character will be "-", i.e. we are
+;;  processing an isolated short option.
+;;
+;;  A          IN   string               The first argument on the command-line
+;;  rest       IN   list of strings      The remaining items of the command-line
+;;  specs      IN   list of option-spec  Options specification
+;;  remnant    OUT  list of strings      The unprocessed command line
+;;  processed  OUT  string               New command-line argument
+(define (normalize-free-short-option A rest specs)
+  (define (return-empty-arg) (values rest (double-up A specs)))
+  (define (return-arg-with-next-value)
+    (values (cdr rest) (string-append (double-up A specs) (car rest))))
+  (let* ((name (string-ref A 1))
+         (spec (find-spec-short specs name)))
+    (if (cond ((not spec) (fatal-error "no such option: -~a" name))
+              ((null? rest) #f)
+              ((option-spec->predicate spec)
+               => (λ (pred) (pred (car rest))))
+              (else (case (option-spec->value-policy spec)
+                      ((optional) (not (eq? (string-ref (car rest) 0) #\-)))
+                      (else => identity))))
+        (return-arg-with-next-value)
+        (return-empty-arg))))
+
+
+
+;; The /sequence/ is a string of characters from the command line, and
+;; the task is to decide if those characters are a viable clumped option
+;; sequence, possibly using some of the trailing characters as option
+;; values, or not.
+(define (viable-short sequence specs)
+  (cond ((eq? 0 (string-length sequence)) #t)
+        ((find-spec-short specs (string-ref sequence 0))
+         => (λ (spec)
+               (cond ((option-spec->predicate spec)
+                      => (λ (pred) (pred (substring sequence 1))))
+                     (else
+                      ;; If this optionʼs /value-policy/ allows the
+                      ;; option to take a value then this string is
+                      ;; viable as the remainder can be taken as that
+                      ;; value.  Otherwise we must assert the viability
+                      ;; of the rest of the line by recursion.
+                      (or (not (eq? #f (option-spec->value-policy spec)))
+                          (viable-short (substring sequence 1) specs))))))
+        (else #f)))
+
+
+
+;;  This procedure does whatever is necessary to put the (ostensibly)
+;;  first item on the command-line into the canonical form
+;;  '--item=value'.  Note that the value may be missing, but the '='
+;;  sign will always be there in the return.  The first item (/A/) will
+;;  always be *more* than two characters, and the first character will
+;;  be "-", i.e. we are processing a short option which is either
+;;  clumped with other short options, or is clumped with its value.
+;;
+;;  NOTE that, contrary to the other normalize procedures, the return
+;;  value of /processed/ can be #f, with the expectation that the
+;;  modified /remnant/ will be re-processed.
+;;
+;;  A          IN   string               The first argument on the command-line
+;;  rest       IN   list of strings      The remaining items of the command-line
+;;  specs      IN   list of option-spec  Options specification
+;;  remnant    OUT  list of strings      The unprocessed command line
+;;  processed  OUT  string               New command-line argument
+(define (normalize-clumped-short-option A rest specs)
+  (define (declump-arg) (values (cons* (string-append "-" (substring A 1 2))
+                                       (string-append "-" (substring A 2))
+                                       rest)
+                                #f))
+  (define (construct-arg-from-clumped-value)
+    (values rest (string-append (double-up A specs) (substring A 2))))
+  (unless (char-alphabetic? (string-ref A 1)) (values rest A))
+  (let ((spec (find-spec-short specs (string-ref A 1))))
+    (if (cond ((not spec) (fatal-error "no such option: -~a" (string-ref A 1)))
+              ((option-spec->predicate spec)
+               => (λ (pred) (pred (substring A 2))))
+              (else (case (option-spec->value-policy spec)
+                      ((optional) (not (viable-short (substring A 2) specs)))
+                      (else => identity))))
+        (construct-arg-from-clumped-value)
+        (declump-arg))))
+
+
+
+;;  Return a version of the command-line /args/ in which all options are
+;;  represented in long form with an equals sign (whether they have a
+;;  value or not).
+(define (normalize args specs stop-at-first-non-option?)
+  (call/ec
+    (λ (return)
+      (let loop ((args args) (processed '()))
+        (when (null? args) (return (reverse processed)))
+        (define A (car args))
+        (define L (string-length A))
+        (define (when-loop cond normalizer)
+          (when cond
+            (receive (remainder-args processed-arg)
+                     (normalizer A (cdr args) specs)
+                     (loop
+                       remainder-args
+                       (if processed-arg
+                           (cons processed-arg processed)
+                           processed)))))
+        (when (string=? "--" A)
+          (return (append (reverse processed) args)))
+        (when-loop (and (> L 2) (string=? (substring A 0 2) "--"))
+          normalize-long-option)
+        (when-loop (and (eqv? L 2) (eqv? (string-ref A 0) #\-))
+          normalize-free-short-option)
+        (when-loop (and (> L 1) (eqv? (string-ref A 0) #\-))
+          normalize-clumped-short-option)
+        (if stop-at-first-non-option?
+            (return (append (reverse processed) args))
+            (loop (cdr args) (cons A processed)))))))
+
+
+
+;;  Check that all the rules inherent in the /specs/ are fulfilled by
+;;  the /options/.
+(define (verify-specs-fullfilled specs options)
+  (for-each
+    (λ (spec)
+      (let* ((name (option-spec->name spec))
+             (value (assq-ref options (string->symbol name))))
+        (when (and (option-spec->required? spec) (not value))
+          (fatal-error "option must be specified: --~a" name))
+        (let ((policy (option-spec->value-policy spec)))
+          (when (and (eq? policy #t) (eq? value  #t))
+            (fatal-error "option must be specified with argument: --~a" name))
+          (when (and (eq? policy #f) (string? value))
+            (fatal-error "option does not support argument: --~a" name)))
+        (let ((pred (option-spec->predicate spec)))
+          (when (and pred (string? value) (not (pred value)))
+            (fatal-error "option predicate failed: --~a" name)))))
+   specs))
+
+
+
+;;  Check that all the options are matched by a specification.
+(define (verify-options options specs)
+  (for-each
+    (λ (value)
+      (unless (or (null? (car value))
+                  (find-spec-long specs (symbol->string (car value))))
+        (fatal-error "no such option: --~a" (car value))))
+   options))
+
+
+
+;;  This procedure will simply return if the options and the specs
+;;  conform with each other, or else will bail out with an error
+;;  message.
+(define (check-compliance options specs)
+  (verify-specs-fullfilled specs options)
+  (verify-options options specs))
+
+
+
+(define full-option-re (make-regexp "^--([^=]+)=(.+)?$"))
+
+;; The /normal-args/ are a normalized command line in which all
+;; options are expressed long-form, and the task here is to construct an
+;; /options/ object: an associative array of option names onto values
+;; (or #t if there is no value).
+(define (extract-options normal-args stop-at-first-non-option?)
+  (let loop ((args normal-args)
+             (options '())
+             (non-options '()))
+       (cond
+         ((null? args) (acons '() (reverse non-options) options))
+         ((string=? (car args) "--")
+          (acons '() (append (reverse non-options) (cdr args)) options))
+         ((regexp-exec full-option-re (car args))
+          => (λ (match)
+                (loop (cdr args)
+                      (acons (string->symbol (match:substring match 1))
+                             (or (match:substring match 2) #t)
+                             options)
+                      non-options)))
+         (stop-at-first-non-option?
+          (acons '() (append (reverse non-options) args) options))
          (else
-          (loop (- unclumped 1) ls (acons spec #t found) etc))))
-      
-      (match argument-ls
-        (()
-         (cons found (reverse etc)))
-        ((opt . rest)
-         (cond
-          ((regexp-exec short-opt-rx opt)
-           => (lambda (match)
-                (if (> unclumped 0)
-                    ;; Next option is known not to be clumped.
-                    (let* ((c (match:substring match 1))
-                           (spec (or (assoc-ref sc-idx c)
-                                     (fatal-error "no such option: -~a" c))))
-                      (eat! spec rest))
-                    ;; Expand a clumped group of short options.
-                    (let* ((extra (match:substring match 2))
-                           (unclumped-opts
-                            (append (map (lambda (c)
-                                           (string-append "-" (make-string 1 c)))
-                                         (string->list
-                                          (match:substring match 1)))
-                                    (if (string=? "" extra) '() (list extra)))))
-                      (loop (length unclumped-opts)
-                            (append unclumped-opts rest)
-                            found
-                            etc)))))
-          ((regexp-exec long-opt-no-value-rx opt)
-           => (lambda (match)
-                (let* ((opt (match:substring match 1))
-                       (spec (or (assoc-ref idx opt)
-                                 (fatal-error "no such option: --~a" opt))))
-                  (eat! spec rest))))
-          ((regexp-exec long-opt-with-value-rx opt)
-           => (lambda (match)
-                (let* ((opt (match:substring match 1))
-                       (spec (or (assoc-ref idx opt)
-                                 (fatal-error "no such option: --~a" opt))))
-                  (if (option-spec->value-policy spec)
-                      (eat! spec (cons (match:substring match 2) rest))
-                      (fatal-error "option does not support argument: --~a"
-                                   opt)))))
-          ((and stop-at-first-non-option
-                (<= unclumped 0))
-           (cons found (append (reverse etc) argument-ls)))
-          (else
-           (loop (- unclumped 1) rest found (cons opt etc)))))))))
+           (loop (cdr args) options (cons (car args) non-options))))))
+
+
 
 (define* (getopt-long program-arguments option-desc-list
                       #:key stop-at-first-non-option)
@@ -338,34 +509,22 @@ or option values.
 required.  By default, single character equivalents are not supported;
 if you want to allow the user to use single character options, you need
 to add a `single-char' clause to the option description."
-  (with-fluids ((%program-name (car program-arguments)))
-    (let* ((specifications (map parse-option-spec option-desc-list))
-           (pair (split-arg-list (cdr program-arguments)))
-           (split-ls (car pair))
-           (non-split-ls (cdr pair))
-           (found/etc (process-options specifications split-ls
-                                       stop-at-first-non-option))
-           (found (car found/etc))
-           (rest-ls (append (cdr found/etc) non-split-ls)))
-      (for-each (lambda (spec)
-                  (let ((name (option-spec->name spec))
-                        (val (assq-ref found spec)))
-                    (and (option-spec->required? spec)
-                         (or val
-                             (fatal-error "option must be specified: --~a"
-                                          name)))
-                    (let ((pred (option-spec->predicate spec)))
-                      (and pred (pred name val)))))
-                specifications)
-      (for-each (lambda (spec+val)
-                  (set-car! spec+val
-                            (string->symbol (option-spec->name (car spec+val)))))
-                found)
-      (cons (cons '() rest-ls) found))))
+  (parameterize ((program-name (car program-arguments)))
+    (let* ((specs (map parse-option-spec option-desc-list))
+           (options (extract-options
+                      (normalize (cdr program-arguments)
+                                 specs
+                                 stop-at-first-non-option)
+                      stop-at-first-non-option)))
+      (check-compliance options specs)
+      options)))
+
+
 
 (define (option-ref options key default)
-  "Return value in alist OPTIONS using KEY, a symbol; or DEFAULT if not found.
-The value is either a string or `#t'."
+  "Return value in OPTIONS (as returned from getopt-long), using KEY--
+a symbol--, or DEFAULT if not found.  The value is either a string or
+‘#t’, or whatever DEFAULT is."
   (or (assq-ref options key) default))
 
 ;;; getopt-long.scm ends here
-- 
2.27.0

Message sent to bug-guile@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#42669: [PATCH 4/4 (v2)] (ice-9 getopt-long): update commentary and doc-strings
References: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
In-Reply-To: <dc9f1e0ff74355711278a6475370adface46ee0c.camel@HIDDEN>
Resent-From: Dale Mellor <guile-qf1qmg@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guile@HIDDEN
Resent-Date: Sun, 02 Aug 2020 10:37:01 +0000
Resent-Message-ID: <handler.42669.B42669.15963645945140 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 42669
X-GNU-PR-Package: guile
X-GNU-PR-Keywords: patch
To: 42669 <at> debbugs.gnu.org
Received: via spool by 42669-submit <at> debbugs.gnu.org id=B42669.15963645945140
          (code B ref 42669); Sun, 02 Aug 2020 10:37:01 +0000
Received: (at 42669) by debbugs.gnu.org; 2 Aug 2020 10:36:34 +0000
Received: from localhost ([127.0.0.1]:41771 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1k2BM1-0001Kq-PS
	for submit <at> debbugs.gnu.org; Sun, 02 Aug 2020 06:36:34 -0400
Received: from rdmp.org ([52.19.174.175]:36134)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <guile-qf1qmg@HIDDEN>) id 1k2BLz-0001Kc-Ii
 for 42669 <at> debbugs.gnu.org; Sun, 02 Aug 2020 06:36:32 -0400
Received: from [127.0.0.1] (helo=localhost) by rdmp.org with esmtp (Exim 4.92)
 (envelope-from <guile-qf1qmg@HIDDEN>) id 1k2BLt-0005uQ-St
 for 42669 <at> debbugs.gnu.org; Sun, 02 Aug 2020 10:36:26 +0000
Message-ID: <e78e7188ada3b7bf31da96252f932c42be03c872.camel@HIDDEN>
From: Dale Mellor <guile-qf1qmg@HIDDEN>
Date: Sun, 02 Aug 2020 11:36:25 +0100
Organization: DM Bespoke Computer Solutions Ltd
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.30.5-1.1 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Score: 1.9 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Emphasise importance of predicate part of specification of
 options with optional values. Minor clarifications elsewhere. Update copyright
 years and authorship. * module/ice-9/getopt-long.scm: Small changes only
 in non-code parts of source file. --- module/ice-9/getopt-long.scm | 108
 +++++++++++++++++++++
 1 file changed, 65 insertions(+), 43 del [...] 
 Content analysis details:   (1.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 1.0 SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
 0.9 SPF_HELO_SOFTFAIL      SPF: HELO does not match SPF record (softfail)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.9 (/)

Emphasise importance of predicate part of specification of options with
optional values.  Minor clarifications elsewhere.  Update copyright
years and authorship.

* module/ice-9/getopt-long.scm: Small changes only in non-code parts of
  source file.
---
 module/ice-9/getopt-long.scm | 108 +++++++++++++++++++++--------------
 1 file changed, 65 insertions(+), 43 deletions(-)

diff --git a/module/ice-9/getopt-long.scm b/module/ice-9/getopt-long.scm
index 06aa1a879..5726fb5eb 100644
--- a/module/ice-9/getopt-long.scm
+++ b/module/ice-9/getopt-long.scm
@@ -1,5 +1,8 @@
-;;; Copyright (C) 1998, 2001, 2006, 2009, 2011 Free Software Foundation, Inc.
-;;;
+;;;; getopt-long.scm --- long options processing   -*- scheme -*-
+;;;;
+;;;; Copyright (C) 1998, 2001, 2006, 2009, 2011, 2020
+;;;;                                            Free Software Foundation, Inc.
+;;;;
 ;;;; This library is free software; you can redistribute it and/or
 ;;;; modify it under the terms of the GNU Lesser General Public
 ;;;; License as published by the Free Software Foundation; either
@@ -12,54 +15,59 @@
 ;;;; 
 ;;;; You should have received a copy of the GNU Lesser General Public
 ;;;; License along with this library; if not, write to the Free Software
-;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+;;;; 02110-1301 USA
 
-;;; Author: Russ McManus (rewritten by Thien-Thi Nguyen)
+;;; Author: Russ McManus
+;;;         Rewritten by Thien-Thi Nguyen
+;;;         Rewritten by Dale Mellor 2020-04-14
 
 ;;; Commentary:
 
 ;;; This module implements some complex command line option parsing, in
-;;; the spirit of the GNU C library function `getopt_long'.  Both long
+;;; the spirit of the GNU C library function ‘getopt_long’.  Both long
 ;;; and short options are supported.
 ;;;
 ;;; The theory is that people should be able to constrain the set of
-;;; options they want to process using a grammar, rather than some arbitrary
-;;; structure.  The grammar makes the option descriptions easy to read.
+;;; options they want to process using a grammar, rather than some ad
+;;; hoc procedure.  The grammar makes the option descriptions easy to
+;;; read.
 ;;;
-;;; `getopt-long' is a procedure for parsing command-line arguments in a
-;;; manner consistent with other GNU programs.  `option-ref' is a procedure
-;;; that facilitates processing of the `getopt-long' return value.
+;;; ‘getopt-long’ is a procedure for parsing command-line arguments in a
+;;; manner consistent with other GNU programs.  ‘option-ref’ is a procedure
+;;; that facilitates processing of the ‘getopt-long’ return value.
 
 ;;; (getopt-long ARGS GRAMMAR)
 ;;; Parse the arguments ARGS according to the argument list grammar GRAMMAR.
 ;;;
 ;;; ARGS should be a list of strings.  Its first element should be the
-;;; name of the program; subsequent elements should be the arguments
+;;; name of the program, and subsequent elements should be the arguments
 ;;; that were passed to the program on the command line.  The
-;;; `program-arguments' procedure returns a list of this form.
+;;; ‘program-arguments’ procedure returns a list of this form.
 ;;;
 ;;; GRAMMAR is a list of the form:
 ;;; ((OPTION (PROPERTY VALUE) ...) ...)
 ;;;
-;;; Each OPTION should be a symbol.  `getopt-long' will accept a
-;;; command-line option named `--OPTION'.
+;;; Each OPTION should be a symbol.  ‘getopt-long’ will accept a
+;;; command-line option named ‘--OPTION’.
 ;;; Each option can have the following (PROPERTY VALUE) pairs:
 ;;;
-;;;   (single-char CHAR) --- Accept `-CHAR' as a single-character
-;;;		equivalent to `--OPTION'.  This is how to specify traditional
+;;;   (single-char CHAR) --- Accept ‘-CHAR’ as a single-character
+;;;		equivalent to ‘--OPTION’.  This is how to specify traditional
 ;;;		Unix-style flags.
 ;;;   (required? BOOL) --- If BOOL is true, the option is required.
 ;;;		getopt-long will raise an error if it is not found in ARGS.
 ;;;   (value BOOL) --- If BOOL is #t, the option accepts a value; if
 ;;;		it is #f, it does not; and if it is the symbol
-;;;		`optional', the option may appear in ARGS with or
+;;;		‘optional’, the option may appear in ARGS with or
 ;;;		without a value.
 ;;;   (predicate FUNC) --- If the option accepts a value (i.e. you
-;;;		specified `(value #t)' for this option), then getopt
-;;;		will apply FUNC to the value, and throw an exception
-;;;		if it returns #f.  FUNC should be a procedure which
-;;;		accepts a string and returns a boolean value; you may
-;;;		need to use quasiquotes to get it into GRAMMAR.
+;;;		specified ‘(value #t)’ or ‘(value 'optional)’ for this
+;;;		option), then getopt will apply FUNC to the value, and
+;;;		will not take the value if it returns #f.  FUNC should
+;;;		be a procedure which accepts a string and returns a
+;;;		boolean value; you may need to use quasiquotes to get it
+;;;		into GRAMMAR.
 ;;;
 ;;; The (PROPERTY VALUE) pairs may occur in any order, but each
 ;;; property may occur only once.  By default, options do not have
@@ -79,16 +87,22 @@
 ;;;                                         for "blimps" and "catalexis")
 ;;;    ("-ab" "bang" "-c" "couth")         (same)
 ;;;    ("-ac" "couth" "-b" "bang")         (same)
-;;;    ("-abc" "couth" "bang")             (an error, since `-b' is not the
-;;;                                         last option in its combination)
 ;;;
-;;; If an option's value is optional, then `getopt-long' decides
-;;; whether it has a value by looking at what follows it in ARGS.  If
-;;; the next element is does not appear to be an option itself, then
-;;; that element is the option's value.
+;;; If an option's value is optional, then ‘getopt-long’ decides whether
+;;; it has a value by looking at what follows it in ARGS.  If the next
+;;; element does not appear to be an option itself, and passes a
+;;; predicate if given, then that element is taken to be the option's
+;;; value.  Note that predicate functions are invaluable in this respect
+;;; for differentiating options and option values, and in the case of
+;;; options with optional values, PREDICATES REALLY SHOULD BE GIVEN.  If
+;;; an option is supposed to take a numerical value, then
+;;; ‘string->number’ can be used as the predicate; this will also allow
+;;; negative values to be used, which would ordinarily be regarded as
+;;; bad options causing the module, and the application consuming it, to
+;;; bail out with an immediate exit to the operating system.
 ;;;
 ;;; The value of a long option can appear as the next element in ARGS,
-;;; or it can follow the option name, separated by an `=' character.
+;;; or it can follow the option name, separated by an ‘=’ character.
 ;;; Thus, using the same grammar as above, the following argument lists
 ;;; are equivalent:
 ;;;   ("--apples" "Braeburn" "--blimps" "Goodyear")
@@ -99,27 +113,30 @@
 ;;; subsequent arguments are returned as ordinary arguments, even if
 ;;; they resemble options.  So, in the argument list:
 ;;;         ("--apples" "Granny Smith" "--" "--blimp" "Goodyear")
-;;; `getopt-long' will recognize the `apples' option as having the
-;;; value "Granny Smith", but it will not recognize the `blimp'
-;;; option; it will return the strings "--blimp" and "Goodyear" as
-;;; ordinary argument strings.
+;;; ‘getopt-long’ will recognize the ‘apples’ option as having the value
+;;; "Granny Smith", but it will not recognize the ‘blimp’ option; it
+;;; will return the strings "--blimp" and "Goodyear" as ordinary
+;;; argument strings.  The first "--" argument itself will *not* appear
+;;; in the ordinary arguments list, although the occurrence of
+;;; subsequent ones will.
 ;;;
-;;; The `getopt-long' function returns the parsed argument list as an
+;;; The ‘getopt-long’ function returns the parsed argument list as an
 ;;; assocation list, mapping option names --- the symbols from GRAMMAR
 ;;; --- onto their values, or #t if the option does not accept a value.
 ;;; Unused options do not appear in the alist.
 ;;;
-;;; All arguments that are not the value of any option are returned
-;;; as a list, associated with the empty list.
+;;; All arguments that are not the value of any option are returned as a
+;;; list, associated with the empty list in the above returned
+;;; association.
 ;;;
-;;; `getopt-long' throws an exception if:
+;;; ‘getopt-long’ throws an exception if:
 ;;; - it finds an unrecognized property in GRAMMAR
-;;; - the value of the `single-char' property is not a character
+;;; - the value of the ‘single-char’ property is not a character
 ;;; - it finds an unrecognized option in ARGS
 ;;; - a required option is omitted
 ;;; - an option that requires an argument doesn't get one
 ;;; - an option that doesn't accept an argument does get one (this can
-;;;   only happen using the long option `--opt=value' syntax)
+;;;   only happen using the long option ‘--opt=value’ syntax)
 ;;; - an option predicate fails
 ;;;
 ;;; So, for example:
@@ -147,9 +164,10 @@
 
 ;;; (option-ref OPTIONS KEY DEFAULT)
 ;;; Return value in alist OPTIONS using KEY, a symbol; or DEFAULT if not
-;;; found.  The value is either a string or `#t'.
+;;; found.  The return is either a string or ‘#t’, or whatever DEFAULT
+;;; is.
 ;;;
-;;; For example, using the `getopt-long' return value from above:
+;;; For example, using the ‘getopt-long’ return value from above:
 ;;;
 ;;; (option-ref (getopt-long ...) 'x-includes 42) => "/usr/include"
 ;;; (option-ref (getopt-long ...) 'not-a-key! 31) => 31
@@ -479,7 +497,10 @@
 
 (define* (getopt-long program-arguments option-desc-list
                       #:key stop-at-first-non-option)
-  "Process options, handling both long and short options, similar to
+  "- Scheme Procedure: getopt-long PROGRAM-ARGUMENTS OPTION-DESC-LIST
+                                 [#:stop-at-first-non-option]
+
+Process options, handling both long and short options, similar to
 the glibc function 'getopt_long'.  PROGRAM-ARGUMENTS should be a value
 similar to what (program-arguments) returns.  OPTION-DESC-LIST is a
 list of option descriptions.  Each option description must satisfy the
@@ -508,7 +529,7 @@ or option values.
     By default, options are not required, and option values are not
 required.  By default, single character equivalents are not supported;
 if you want to allow the user to use single character options, you need
-to add a `single-char' clause to the option description."
+to add a ‘single-char’ clause to the option description."
   (parameterize ((program-name (car program-arguments)))
     (let* ((specs (map parse-option-spec option-desc-list))
            (options (extract-options
@@ -527,4 +548,5 @@ a symbol--, or DEFAULT if not found.  The value is either a string or
 ‘#t’, or whatever DEFAULT is."
   (or (assq-ref options key) default))
 
+
 ;;; getopt-long.scm ends here
-- 
2.27.0

Last modified: Sun, 2 Aug 2020 10:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.