GNU bug report logs - #53941
27.2; socks + tor dont work with https

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Jacobo <gnuhacker@HIDDEN>; Keywords: patch; dated Fri, 11 Feb 2022 14:32:01 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 53941 <at> debbugs.gnu.org:


Received: (at 53941) by debbugs.gnu.org; 6 Mar 2022 02:40:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 05 21:40:49 2022
Received: from localhost ([127.0.0.1]:49570 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nQgpF-0003Lo-B0
	for submit <at> debbugs.gnu.org; Sat, 05 Mar 2022 21:40:49 -0500
Received: from mta763.solicitae.com ([89.35.150.78]:56120
 helo=smtp.gnuhacker.org) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <gnuhacker@HIDDEN>) id 1nQgpD-0003Lf-MI
 for 53941 <at> debbugs.gnu.org; Sat, 05 Mar 2022 21:40:48 -0500
Received: from hackerlab.gnu.org (localhost [127.0.0.1])
 by smtp.gnuhacker.org (Postfix) with ESMTP id C088C6008CF;
 Sun,  6 Mar 2022 03:40:45 +0100 (CET)
From: Jacobo <gnuhacker@HIDDEN>
To: "J.P." <jp@HIDDEN>
Subject: Re: bug#53941: 27.2; socks + tor dont work with https
References: <87pmntfym7.fsf@HIDDEN> <8735kl1v58.fsf@HIDDEN>
 <87a6emftzx.fsf@HIDDEN> <87k0do5km1.fsf@HIDDEN>
 <87pmn5n3tu.fsf@HIDDEN> <87mti99j1f.fsf@HIDDEN>
Date: Sun, 06 Mar 2022 03:40:45 +0100
In-Reply-To: <87mti99j1f.fsf@HIDDEN> (J. P.'s message of "Tue, 01 Mar
 2022 18:37:16 -0800")
Message-ID: <87wnh7hkgi.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 53941
Cc: 53941 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

"J.P." <jp@HIDDEN> writes:

> v3. Passing around an opener function was clunky, so I've opted for
> passing around contact params instead. I've also gone back to explicitly
> setting the coding to binary because folks may not be using
> `url-open-stream' (which does this indirectly by let-binding
> `coding-system-for-{read,write}').

Emacs 28.0.91 compiled with this patches, dont work, connections dont
use the proxy

-- 
Emacs Lover.
FSF Member.
Free/Libre Software supporter.
stallmansupport.org - Disinformation succeeds because so many people
care deeply about injustice but do not take the time to check the facts.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#53941; Package emacs. Full text available.

Message received at 53941 <at> debbugs.gnu.org:


Received: (at 53941) by debbugs.gnu.org; 2 Mar 2022 02:37:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 01 21:37:37 2022
Received: from localhost ([127.0.0.1]:38720 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nPErt-0001SC-Pe
	for submit <at> debbugs.gnu.org; Tue, 01 Mar 2022 21:37:37 -0500
Received: from mail-108-mta130.mxroute.com ([136.175.108.130]:35873)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jp@HIDDEN>) id 1nPErr-0001Rx-HI
 for 53941 <at> debbugs.gnu.org; Tue, 01 Mar 2022 21:37:32 -0500
Received: from filter006.mxroute.com ([140.82.40.27] 140.82.40.27.vultr.com)
 (Authenticated sender: mN4UYu2MZsgR)
 by mail-108-mta130.mxroute.com (ZoneMTA) with ESMTPSA id
 17f487d3c7d0005a20.001 for <53941 <at> debbugs.gnu.org>
 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256);
 Wed, 02 Mar 2022 02:37:21 +0000
X-Zone-Loop: f59c6fe4143e200293f5a4d0b1fa812f35b3a50cf18c
X-Originating-IP: [140.82.40.27]
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=neverwas.me
 ; s=x;
 h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:References:
 Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID:
 Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
 :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:
 List-Post:List-Owner:List-Archive;
 bh=Z5Z6pLBgpsbZfN4oq1RpXEKLQC+TYkRMfKTvniQYR4s=; b=nJqFmZbBm5htBMzLCsoqPiTNX5
 7vjCP1peDxKOskhXLWmdq32gDGNGXVozJC+riaXJ7tjtl4ndVNjxMDLddzBd2aFA5OD9E82dSTvOY
 YWRypC5OwOsC8n9PHMEpQ6BFKGKQ490SwrDs1kVRDK3JU9vghSPle5T80Rg51n6qBmehUaSFOKYLy
 cLhQwgPpJnqrcs7kUSLp6nXej1TCG12y7kzGl/piYwAYa1O8M0DheP947O9xOF9Yx/sp2aKIv5DZg
 ZBb7iQ0DcqEV8SWVQTnjlxkT+4xBgAXdeW6aCfu7vQhmgrgMxpvW/YgItdtZU2K3XSKtsZ/VxGNP4
 5GxAixRA==;
From: "J.P." <jp@HIDDEN>
To: 53941 <at> debbugs.gnu.org
Subject: Re: bug#53941: 27.2; socks + tor dont work with https
References: <87pmntfym7.fsf@HIDDEN> <8735kl1v58.fsf@HIDDEN>
 <87a6emftzx.fsf@HIDDEN> <87k0do5km1.fsf@HIDDEN>
 <87pmn5n3tu.fsf@HIDDEN>
Date: Tue, 01 Mar 2022 18:37:16 -0800
In-Reply-To: <87pmn5n3tu.fsf@HIDDEN> (J. P.'s message of "Tue, 01 Mar
 2022 06:29:49 -0800")
Message-ID: <87mti99j1f.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-AuthUser: masked@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 53941
Cc: Jacobo <gnuhacker@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--=-=-=
Content-Type: text/plain

v3. Passing around an opener function was clunky, so I've opted for
passing around contact params instead. I've also gone back to explicitly
setting the coding to binary because folks may not be using
`url-open-stream' (which does this indirectly by let-binding
`coding-system-for-{read,write}').

--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment; filename=0000-NOT-A-PATCH-v2-v3.diff

From 45be9bbb941e91efe9dacf1b3c34d4d362593d53 Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Tue, 1 Mar 2022 14:45:26 -0800
Subject: [PATCH 0/5] NOT A PATCH

*** BLURB HERE ***

F. Jason Park (5):
  Simplify network-stream opener in socks.el
  Fix string encoding bug in socks tests
  Add support for SOCKS 4a
  Support SOCKS resolve extension
  [POC] Demo SOCKS resolve with HTTPS

 lisp/net/socks.el            | 133 ++++++++++++++++++++++++++++-------
 test/lisp/net/socks-tests.el | 113 +++++++++++++++++++++++++++--
 2 files changed, 213 insertions(+), 33 deletions(-)

Interdiff:
diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index cd026fd163..02edd95328 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -334,18 +334,22 @@ socks-filter
 (defvar socks-override-functions nil
   "If non-nil, overwrite `open-network-stream' function with SOCKSified version.")
 (make-obsolete-variable 'socks-override-functions
-                        "`socks--open-network-stream' now takes a process arg."
+                        "use custom opener with `socks-open-stream-function'."
                         "29.1")
 
-(defun socks-open-connection (server-info &optional opener)
+(defvar socks-open-stream-function #'open-network-stream
+  "Function called to open a network stream connection.")
+
+(defun socks-open-connection (server-info &rest params)
   "Create and initialize a SOCKS process.
 Perform authentication if needed.  SERVER-INFO should resemble
-`socks-server'.  OPENER, when present, should be a substitute for
-`open-network-stream' and take the same arguments."
+`socks-server'.  PARAMS are those accepted by `make-network-process'."
   (interactive)
+  (unless (plist-member params :coding)
+    (setf (plist-get params :coding) '(binary . binary)))
   (save-excursion
-    (let ((proc (funcall (or opener #'open-network-stream)
-                         "socks" nil (nth 1 server-info) (nth 2 server-info)))
+    (let ((proc (apply socks-open-stream-function "socks" nil
+                       (nth 1 server-info) (nth 2 server-info) params))
 	  (authtype nil)
 	  version)
 
@@ -531,11 +535,11 @@ socks-find-services-entry
 
 (defun socks-open-network-stream (name buffer host service &rest params)
   (if-let* ((route (socks-find-route host service))
-            (proc (socks-open-connection route #'open-network-stream)))
+            (proc (apply #'socks-open-connection route params)))
       (socks--open-network-stream proc buffer host service)
     (message "Warning: no SOCKS route found for %s:%s" host service)
     ;; Support legacy behavior (likely undesirable in most cases)
-    (apply #'open-network-stream name buffer host service params)))
+    (apply socks-open-stream-function name buffer host service params)))
 
 (defun socks--open-network-stream (proc buffer host service)
   (progn ; temporarily preserve git blame for easier reviewing
@@ -684,17 +688,20 @@ socks-tor-resolve
   (let ((socks-password (or socks-password ""))
         host
         (port 80)  ; unused for now
+        route
         proc
         ip)
     (unless (string-suffix-p ".onion" name)
       (setq host (if (string-match "\\`[[:ascii:]]+\\'" name)
                      name
                    (require 'puny)
-                   (puny-encode-domain name)))
+                   (puny-encode-domain name))
+            route (socks-find-route host port))
+      (cl-assert route)
       ;; "Host unreachable" may be raised when the lookup fails
       (unwind-protect
           (progn
-            (setq proc (socks-open-connection (socks-find-route host port)))
+            (setq proc (socks-open-connection route))
             (socks-send-command proc
                                 socks-resolve-command
                                 socks-address-type-name
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0001-Simplify-network-stream-opener-in-socks.el.patch

From 90247189d5fe90619f00ef3319012df0f6f6688e Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Tue, 1 Mar 2022 02:12:02 -0800
Subject: [PATCH 1/5] Simplify network-stream opener in socks.el

* lisp/net/socks.el (socks-override-functions,
socks-open-stream-function): Make first variable obsolete and remove
uses.  Replace somewhat with the second, which holds a network stream
opener that defaults to `open-network-stream'.
(socks-open-connection): Accept additional `make-network-process'
params passed on to opener.
(socks-open-network-stream): Likewise with the additional params.
Call `open-network-stream' as a fallback when a route cannot be found.
(socks--open-network-stream): Reduce role to merely issuing the first
command using an existing process.  This may warrant a renaming.
Change signature accordingly.
---
 lisp/net/socks.el | 50 +++++++++++++++++++++++------------------------
 1 file changed, 25 insertions(+), 25 deletions(-)

diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index 8df0773e1d..5b78eb6e84 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -323,19 +323,23 @@ socks-filter
 
 (defvar socks-override-functions nil
   "If non-nil, overwrite `open-network-stream' function with SOCKSified version.")
+(make-obsolete-variable 'socks-override-functions
+                        "use custom opener with `socks-open-stream-function'."
+                        "29.1")
 
-(when socks-override-functions
-  (advice-add 'open-network-stream :around #'socks--open-network-stream))
+(defvar socks-open-stream-function #'open-network-stream
+  "Function called to open a network stream connection.")
 
-(defun socks-open-connection (server-info)
+(defun socks-open-connection (server-info &rest params)
+  "Create and initialize a SOCKS process.
+Perform authentication if needed.  SERVER-INFO should resemble
+`socks-server'.  PARAMS are those accepted by `make-network-process'."
   (interactive)
+  (unless (plist-member params :coding)
+    (setf (plist-get params :coding) '(binary . binary)))
   (save-excursion
-    (let ((proc
-           (let ((socks-override-functions nil))
-             (open-network-stream "socks"
-				  nil
-				  (nth 1 server-info)
-				  (nth 2 server-info))))
+    (let ((proc (apply socks-open-stream-function "socks" nil
+                       (nth 1 server-info) (nth 2 server-info) params))
 	  (authtype nil)
 	  version)
 
@@ -508,22 +512,18 @@ socks-find-services-entry
   (gethash (downcase service)
 	      (if udp socks-udp-services socks-tcp-services)))
 
-(defun socks-open-network-stream (name buffer host service)
-  (let ((socks-override-functions t))
-    (socks--open-network-stream
-     (lambda (&rest args)
-       (let ((socks-override-functions nil))
-         (apply #'open-network-stream args)))
-     name buffer host service)))
-
-(defun socks--open-network-stream (orig-fun name buffer host service &rest params)
-  (let ((route (and socks-override-functions
-                    (socks-find-route host service))))
-    (if (not route)
-	(apply orig-fun name buffer host service params)
-      ;; FIXME: Obey `params'!
-      (let* ((proc (socks-open-connection route))
-	     (version (process-get proc 'socks-server-protocol))
+(defun socks-open-network-stream (name buffer host service &rest params)
+  (if-let* ((route (socks-find-route host service))
+            (proc (apply #'socks-open-connection route params)))
+      (socks--open-network-stream proc buffer host service)
+    (message "Warning: no SOCKS route found for %s:%s" host service)
+    ;; Support legacy behavior (likely undesirable in most cases)
+    (apply socks-open-stream-function name buffer host service params)))
+
+(defun socks--open-network-stream (proc buffer host service)
+  (progn ; temporarily preserve git blame for easier reviewing
+    (progn ; could rename to something like `socks--initiate-command-sequence'
+      (let* ((version (process-get proc 'socks-server-protocol))
              (atype
               (cond
                ((equal version 4)
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0002-Fix-string-encoding-bug-in-socks-tests.patch

From 181548ce7f931fedd66e243632c42c5c51af640e Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 2/5] Fix string encoding bug in socks tests

* test/lisp/net/socks-tests.el (socks-tests-canned-server-create,
socks-tests-filter-response-parsing-v4): Fix bug in process filter to
prevent prepared outgoing responses from being implicitly encoded as
utf-8.  Fix similar mistake in v4 filter test.
---
 test/lisp/net/socks-tests.el | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 461796bdf9..d9ef53ae35 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -63,21 +63,21 @@ socks-tests-filter-response-parsing-v4
     (process-put proc 'socks-state socks-state-waiting)
     (process-put proc 'socks-server-protocol 4)
     (ert-info ("Receive initial incomplete segment")
-      (socks-filter proc (concat [0 90 0 0 93 184 216]))
-      ;; From example.com: OK status ^      ^ msg start
+      (socks-filter proc (unibyte-string 0 90 0 0 93 184 216))
+      ;; From example.com: OK status       ^      ^ msg start
       (ert-info ("State still set to waiting")
         (should (eq (process-get proc 'socks-state) socks-state-waiting)))
       (ert-info ("Response field is nil because processing incomplete")
         (should-not (process-get proc 'socks-response)))
       (ert-info ("Scratch field holds stashed partial payload")
-        (should (string= (concat [0 90 0 0 93 184 216])
+        (should (string= (unibyte-string 0 90 0 0 93 184 216)
                          (process-get proc 'socks-scratch)))))
     (ert-info ("Last part arrives")
       (socks-filter proc "\42") ; ?\" 34
       (ert-info ("State transitions to complete (length check passes)")
         (should (eq (process-get proc 'socks-state) socks-state-connected)))
       (ert-info ("Scratch and response fields hold stash w. last chunk")
-        (should (string= (concat [0 90 0 0 93 184 216 34])
+        (should (string= (unibyte-string 0 90 0 0 93 184 216 34)
                          (process-get proc 'socks-response)))
         (should (string= (process-get proc 'socks-response)
                          (process-get proc 'socks-scratch)))))
@@ -140,10 +140,11 @@ socks-tests-canned-server-create
                    (unless (or (and (vectorp pat) (equal pat (vconcat line)))
                                (string-match-p pat line))
                      (error "Unknown request: %s" line))
+                   (setq resp (apply #'unibyte-string (append resp nil)))
                    (let ((print-escape-control-characters t))
                      (message "[%s] <- %s" name (prin1-to-string line))
                      (message "[%s] -> %s" name (prin1-to-string resp)))
-                   (process-send-string proc (concat resp)))))
+                   (process-send-string proc resp))))
          (serv (make-network-process :server 1
                                      :buffer (get-buffer-create name)
                                      :filter filt
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0003-Add-support-for-SOCKS-4a.patch

From db601f1fcbaf5cf088b280966cbac2808a773ee0 Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 3/5] Add support for SOCKS 4a

* lisp/net/socks.el (socks-server): Add new choice `4a' to version
field of option.  This may appear to change the type of the field from
a number to a union of symbols and numbers.  However,
`socks-send-command' and `socks-filter' already expect a possible
`http' value for this field (also a symbol).
(socks--errors-4): Add new constant containing error messages for
socks version 4.  The semantics are faithful, but the wording is
ad-libbed.
(socks-send-command): Massage existing handling for version 4 to
accommodate 4a.

* test/lisp/net/socks-tests.el (socks-tests-v4a-basic): add test for
4a.
Bug#53941
---
 lisp/net/socks.el            | 22 ++++++++++++++++++++--
 test/lisp/net/socks-tests.el | 13 +++++++++++++
 2 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index 5b78eb6e84..a2198d898a 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -162,6 +162,7 @@ socks-server
 	  (radio-button-choice :tag "SOCKS Version"
 			       :format "%t: %v"
 			       (const :tag "SOCKS v4  " :format "%t" :value 4)
+                               (const :tag "SOCKS v4a"  :format "%t" :value 4a)
 			       (const :tag "SOCKS v5"   :format "%t" :value 5))))
 
 
@@ -202,6 +203,12 @@ socks-errors
     "Command not supported"
     "Address type not supported"))
 
+(defconst socks--errors-4
+  '("Granted"
+    "Rejected or failed"
+    "Cannot connect to identd on the client"
+    "Client and identd report differing user IDs"))
+
 ;; The socks v5 address types
 (defconst socks-address-type-v4   1)
 (defconst socks-address-type-name 3)
@@ -404,6 +411,7 @@ socks-send-command
 		(format "%c%s" (length address) address))
 	       (t
 		(error "Unknown address type: %d" atype))))
+        trailing
 	request version)
     (or (process-get proc 'socks)
         (error "socks-send-command called on non-SOCKS connection %S" proc))
@@ -421,6 +429,12 @@ socks-send-command
 			     (t
 			      (error "Unsupported address type for HTTP: %d" atype)))
 			    port)))
+     ((when (eq version '4a)
+        (setf addr "\0\0\0\1"
+              trailing (concat address "\0")
+              version 4 ; done with the "a" part
+              (process-get proc 'socks-server-protocol) 4)
+        nil)) ; fall through
      ((equal version 4)
       (setq request (concat
 		     (unibyte-string
@@ -430,7 +444,8 @@ socks-send-command
 		      (logand port #xff)) ; port, low byte
 		     addr                 ; address
 		     (user-full-name)     ; username
-		     "\0")))              ; terminate username
+                     "\0"                 ; terminate username
+                     trailing)))          ; optional host to look up
      ((equal version 5)
       (setq request (concat
 		     (unibyte-string
@@ -451,7 +466,10 @@ socks-send-command
 	nil				; Sweet sweet success!
       (delete-process proc)
       (error "SOCKS: %s"
-             (nth (or (process-get proc 'socks-reply) 1) socks-errors)))
+             (let ((no (or (process-get proc 'socks-reply) 1)))
+               (if (eq version 5)
+                   (nth no socks-errors)
+                 (nth (+ 90 no) socks--errors-4)))))
     proc))
 
 
diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index d9ef53ae35..4e990ffdba 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -207,6 +207,19 @@ socks-tests-v4-basic
                  (lambda (&optional _) "foo")))
         (socks-tests-perform-hello-world-http-request)))))
 
+(ert-deftest socks-tests-v4a-basic ()
+  "Show correct preparation of SOCKS4a connect command."
+  (let ((socks-server '("server" "127.0.0.1" 10083 4a))
+        (url-user-agent "Test/4a-basic")
+        (socks-tests-canned-server-patterns
+         `(([4 1 0 80 0 0 0 1 ?f ?o ?o 0 ?e ?x ?a ?m ?p ?l ?e ?. ?c ?o ?m 0]
+            . [0 90 0 0 0 0 0 0])
+           ,socks-tests--hello-world-http-request-pattern)))
+    (ert-info ("Make HTTP request over SOCKS4A")
+      (cl-letf (((symbol-function 'user-full-name)
+                 (lambda (&optional _) "foo")))
+        (socks-tests-perform-hello-world-http-request)))))
+
 ;; Replace first pattern below with ([5 3 0 1 2] . [5 2]) to validate
 ;; against curl 7.71 with the following options:
 ;; $ curl --verbose -U foo:bar --proxy socks5h://127.0.0.1:10080 example.com
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0004-Support-SOCKS-resolve-extension.patch

From 67ba3f6e6fcb12b99757fcc49f86f951ad59c02b Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 4/5] Support SOCKS resolve extension

* lisp/net/socks.el (socks-resolve-command): Add new constant for the
SOCKS command RESOLVE, which comes by way of a nonstandard extension
from the TOR project.  It mirrors CONNECT in most respects but asks
the server to RESOLVE a host name and return its IP.  For details, see
https://github.com/torproject/torspec/blob/master/socks-extensions.txt
This shouldn't be confused with 5h/5-hostname, which is used to by
clients like cURL to allow users to bypass attempts to resolve a name
locally.
(socks--extract-resolve-response, socks-tor-resolve): Add utility
functions to query a SOCKS service supporting the RESOLVE extension.
Bug#53941
---
 lisp/net/socks.el            | 61 ++++++++++++++++++++++++++++++++++++
 test/lisp/net/socks-tests.el | 34 ++++++++++++++++++++
 2 files changed, 95 insertions(+)

diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index a2198d898a..02edd95328 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -181,6 +181,9 @@ socks-udp-associate-command
 (defconst socks-authentication-null 0)
 (defconst socks-authentication-failure 255)
 
+;; Extensions
+(defconst socks-resolve-command #xf0)
+
 ;; Response codes
 (defconst socks-response-success               0)
 (defconst socks-response-general-failure       1)
@@ -653,6 +656,64 @@ socks-nslookup-host
 	res)
     host))
 
+(defun socks--extract-resolve-response (proc)
+  "Parse response for PROC and maybe return destination IP address."
+  (let ((response (process-get proc 'socks-response)))
+    (cl-assert response) ; otherwise, msg not received in its entirety
+    (pcase (process-get proc 'socks-server-protocol)
+      (4 ; https://www.openssh.com/txt/socks4a.protocol
+       (when-let (((zerop (process-get proc 'socks-reply)))
+                  ((eq (aref response 1) 90)) ; #x5a request granted
+                  (a (substring response 4)) ; ignore port for now
+                  ((not (string-empty-p a)))
+                  ((not (string= a "\0\0\0\0"))))
+         a))
+      (5 ; https://tools.ietf.org/html/rfc1928
+       (cl-assert (eq 5 (aref response 0)) t)
+       (pcase (aref response 3) ; ATYP
+         (1 (and-let* ((a (substring response 4 8))
+                       ((not (string= a "\0\0\0\0")))
+                       a)))
+         ;; No reason to support RESOLVE_PTR [F1] extension, right?
+         (3 (let ((len (1- (aref response 4))))
+              (substring response 5 (+ 5 len))))
+         (4 (substring response 4 20)))))))
+
+(declare-function puny-encode-domain "puny" (domain))
+
+(defun socks-tor-resolve (name &optional _family)
+  "Return list of one vector IPv4 address for domain NAME.
+Or return nil on failure.  See `network-lookup-address-info' for format
+of return value.  Server must support the Tor RESOLVE command."
+  (let ((socks-password (or socks-password ""))
+        host
+        (port 80)  ; unused for now
+        route
+        proc
+        ip)
+    (unless (string-suffix-p ".onion" name)
+      (setq host (if (string-match "\\`[[:ascii:]]+\\'" name)
+                     name
+                   (require 'puny)
+                   (puny-encode-domain name))
+            route (socks-find-route host port))
+      (cl-assert route)
+      ;; "Host unreachable" may be raised when the lookup fails
+      (unwind-protect
+          (progn
+            (setq proc (socks-open-connection route))
+            (socks-send-command proc
+                                socks-resolve-command
+                                socks-address-type-name
+                                host
+                                port)
+            (cl-assert (eq (process-get proc 'socks-state)
+                           socks-state-connected))
+            (setq ip (socks--extract-resolve-response proc)))
+        (when proc
+          (delete-process proc)))
+      (list (vconcat ip [0])))))
+
 (provide 'socks)
 
 ;;; socks.el ends here
diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 4e990ffdba..3d1aca9af4 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -295,4 +295,38 @@ socks-tests-v5-auth-none
       (socks-tests-perform-hello-world-http-request)))
   (should (assq 2 socks-authentication-methods)))
 
+(ert-deftest tor-resolve-4a ()
+  "Make request to TOR resolve service over SOCKS4a"
+  (let* ((socks-server '("server" "127.0.0.1" 19050 4a))
+         (socks-tests-canned-server-patterns
+          '(([4 #xf0 0 80 0 0 0 1 ?f ?o ?o 0 ?e ?x ?a ?m ?p ?l ?e ?. ?c ?o ?m 0]
+             . [0 90 0 0 93 184 216 34])))
+         (inhibit-message noninteractive)
+         (server (socks-tests-canned-server-create)))
+    (ert-info ("Query TOR RESOLVE service over SOCKS4")
+      (cl-letf (((symbol-function 'user-full-name)
+                 (lambda (&optional _) "foo")))
+        (should (equal '([93 184 216 34 0])
+                       (socks-tor-resolve "example.com")))))
+    (kill-buffer (process-buffer server))
+    (delete-process server)))
+
+(ert-deftest tor-resolve-5 ()
+  "Make request to TOR resolve service over SOCKS5"
+  (let* ((socks-server '("server" "127.0.0.1" 19051 5))
+         (socks-username "foo")
+         (socks-authentication-methods (append socks-authentication-methods
+                                               nil))
+         (inhibit-message noninteractive)
+         (socks-tests-canned-server-patterns
+          '(([5 2 0 2] . [5 2])
+            ([1 3 ?f ?o ?o 0] . [1 0])
+            ([5 #xf0 0 3 11 ?e ?x ?a ?m ?p ?l ?e ?. ?c ?o ?m 0 80]
+             . [5 0 0 1 93 184 216 34 0 0])))
+         (server (socks-tests-canned-server-create)))
+    (ert-info ("Query TOR RESOLVE service over SOCKS5")
+      (should (equal '([93 184 216 34 0]) (socks-tor-resolve "example.com"))))
+    (kill-buffer (process-buffer server))
+    (delete-process server)))
+
 ;;; socks-tests.el ends here
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0005-POC-Demo-SOCKS-resolve-with-HTTPS.patch

From 45be9bbb941e91efe9dacf1b3c34d4d362593d53 Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 5/5] [POC] Demo SOCKS resolve with HTTPS

* test/lisp/net/socks-test.el (test-socks-https-poc): Provide
throwaway test demoing an HTTPS connection over a TOR proxy service.
---
 test/lisp/net/socks-tests.el | 55 +++++++++++++++++++++++++++++++++++-
 1 file changed, 54 insertions(+), 1 deletion(-)

diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 3d1aca9af4..f2600210b0 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -21,7 +21,7 @@
 
 ;;; Code:
 
-(require 'ert)
+(require 'ert-x)
 (require 'socks)
 (require 'url-http)
 
@@ -329,4 +329,57 @@ tor-resolve-5
     (kill-buffer (process-buffer server))
     (delete-process server)))
 
+(defvar test-socks-service ; "127.0.0.1:1080" -> ("127.0.0.1", 1080)
+  (when-let ((present (getenv "TEST_SOCKS_SERVICE"))
+             (parts (split-string present ":")))
+    (list (car parts) (string-to-number (cadr parts)))))
+
+(declare-function gnutls-negotiate "gnutls"
+                  (&rest spec
+                         &key process type hostname priority-string
+                         trustfiles crlfiles keylist min-prime-bits
+                         verify-flags verify-error verify-hostname-error
+                         &allow-other-keys))
+
+(ert-deftest test-socks-https-poc ()
+  :tags '(:unstable)
+  (unless test-socks-service (ert-skip "SOCKS service missing"))
+  (unless (gnutls-available-p) (ert-skip "SOCKS resolve test needs GNUTLS"))
+  (ert-with-temp-file tempfile
+    :prefix "emacs-test-socks-network-security-"
+    (let* ((socks-server `("tor" ,@test-socks-service 5))
+           (socks-password "")
+           (nsm-settings-file tempfile)
+           (url-gateway-method 'socks)
+           (id "sha1:df77269389e537fcc9a5fe61667133b5bb97d42e")
+           (host "check.torproject.org")
+           (url (url-generic-parse-url "https://check.torproject.org"))
+           ;;
+           done
+           ;;
+           (cb (lambda (&rest _r)
+                 (goto-char (point-min))
+                 (should (search-forward "Congratulations" nil t))
+                 (setq done t)))
+           (orig (symbol-function #'socks--open-network-stream)))
+      (cl-letf (((symbol-function 'socks--open-network-stream)
+                 (lambda (&rest rest)
+                   (let ((proc (apply orig rest)))
+                     (gnutls-negotiate :process proc :hostname host)
+                     (should (nsm-verify-connection proc host 443 t))))))
+        (ert-info ("Connect to HTTPS endpoint over Tor SOCKS proxy")
+          (unwind-protect
+              (progn
+                (advice-add 'network-lookup-address-info :override
+                            #'socks-tor-resolve)
+                (should-not (nsm-host-settings id))
+                (url-http url cb '(nil))
+                (ert-info ("Wait for response")
+                  (with-timeout (3 (error "Request timed out"))
+                    (unless done
+                      (sleep-for 0.1))))
+                (should (nsm-host-settings id)))
+            (advice-remove 'network-lookup-address-info
+                           #'socks-tor-resolve)))))))
+
 ;;; socks-tests.el ends here
-- 
2.35.1


--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#53941; Package emacs. Full text available.

Message received at 53941 <at> debbugs.gnu.org:


Received: (at 53941) by debbugs.gnu.org; 1 Mar 2022 14:30:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 01 09:30:09 2022
Received: from localhost ([127.0.0.1]:35918 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nP3Vu-0003mq-3N
	for submit <at> debbugs.gnu.org; Tue, 01 Mar 2022 09:30:09 -0500
Received: from mail-108-mta15.mxroute.com ([136.175.108.15]:39339)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jp@HIDDEN>) id 1nP3Vq-0003ly-03
 for 53941 <at> debbugs.gnu.org; Tue, 01 Mar 2022 09:30:05 -0500
Received: from filter006.mxroute.com ([140.82.40.27] 140.82.40.27.vultr.com)
 (Authenticated sender: mN4UYu2MZsgR)
 by mail-108-mta15.mxroute.com (ZoneMTA) with ESMTPSA id 17f45e338e00005a20.001
 for <53941 <at> debbugs.gnu.org>
 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256);
 Tue, 01 Mar 2022 14:29:53 +0000
X-Zone-Loop: 79703327aa60909ab55ae994c6fa01a522f9c1064320
X-Originating-IP: [140.82.40.27]
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=neverwas.me
 ; s=x;
 h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:References:
 Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID:
 Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
 :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:
 List-Post:List-Owner:List-Archive;
 bh=woorlDzlTwh0zTEf0re8lScUuF36Ls4pYn9sbPnEJZk=; b=ZtsPCdeDutK87cD+dSAXKFDy3G
 VyQazXoR65gED0ag1WUaEI15UOuHfb1WowRRcRIGDwYJRk6ojrrlG6fHks4ZacDf5IZeCvhT9+Y4g
 X6XG3xbYvypm7/EiUjfjRk2USMTTps+MlbdNwskFpom4dBCHWR+rJn3MBP+UYgNaD62QkfITrI8PK
 csKjTdFEd3OhmkB1SlYhtH1+XLJZlkQksWvoaWM9WZb4TqqElXaYV4vonzJXxGGglOlG7k/XLfuCN
 mtjte7ZWEERFey2x+MC76VPQPXrbHj6AJ5ZuD9P+sKM8aQ7FaZqkDDJq8lLshj4RWNmwCff/AgEnF
 uAmEEc2Q==;
From: "J.P." <jp@HIDDEN>
To: Jacobo <gnuhacker@HIDDEN>
Subject: Re: bug#53941: 27.2; socks + tor dont work with https
References: <87pmntfym7.fsf@HIDDEN> <8735kl1v58.fsf@HIDDEN>
 <87a6emftzx.fsf@HIDDEN> <87k0do5km1.fsf@HIDDEN>
Date: Tue, 01 Mar 2022 06:29:49 -0800
In-Reply-To: <87k0do5km1.fsf@HIDDEN> (J. P.'s message of "Mon, 21 Feb
 2022 07:01:58 -0800")
Message-ID: <87pmn5n3tu.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-AuthUser: masked@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 53941
Cc: 53941 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--=-=-=
Content-Type: text/plain

v2. Minor corrections (another bug in existing test, etc.).

--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment; filename=0000-NOT-A-PATCH-v1-v2.diff

From 598e8471789bd6e7eb5a7f3ebc1bbed3cf61f4c6 Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Tue, 1 Mar 2022 06:09:00 -0800
Subject: [PATCH 0/5] NOT A PATCH

*** BLURB HERE ***

F. Jason Park (5):
  Simplify network-stream opener in socks.el
  Fix string encoding bug in socks tests
  Add support for SOCKS 4a
  Support SOCKS RESOLVE extension
  [POC] Demo SOCKS RESOLVE over HTTPS

 lisp/net/socks.el            | 130 +++++++++++++++++++++++++++--------
 test/lisp/net/socks-tests.el | 113 ++++++++++++++++++++++++++++--
 2 files changed, 208 insertions(+), 35 deletions(-)

Interdiff:
diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index 7201ed8e06..cd026fd163 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -333,24 +333,23 @@ socks-filter
 
 (defvar socks-override-functions nil
   "If non-nil, overwrite `open-network-stream' function with SOCKSified version.")
-
-(when socks-override-functions
-  (advice-add 'open-network-stream :around #'socks--open-network-stream))
-
-(defun socks-open-connection (server-info)
+(make-obsolete-variable 'socks-override-functions
+                        "`socks--open-network-stream' now takes a process arg."
+                        "29.1")
+
+(defun socks-open-connection (server-info &optional opener)
+  "Create and initialize a SOCKS process.
+Perform authentication if needed.  SERVER-INFO should resemble
+`socks-server'.  OPENER, when present, should be a substitute for
+`open-network-stream' and take the same arguments."
   (interactive)
   (save-excursion
-    (let ((proc
-           (let ((socks-override-functions nil))
-             (open-network-stream "socks"
-				  nil
-				  (nth 1 server-info)
-				  (nth 2 server-info))))
+    (let ((proc (funcall (or opener #'open-network-stream)
+                         "socks" nil (nth 1 server-info) (nth 2 server-info)))
 	  (authtype nil)
 	  version)
 
       ;; Initialize process and info about the process
-      (set-process-coding-system proc 'binary 'binary)
       (set-process-filter proc #'socks-filter)
       (set-process-query-on-exit-flag proc nil)
       (process-put proc 'socks t)
@@ -530,22 +529,18 @@ socks-find-services-entry
   (gethash (downcase service)
 	      (if udp socks-udp-services socks-tcp-services)))
 
-(defun socks-open-network-stream (name buffer host service)
-  (let ((socks-override-functions t))
-    (socks--open-network-stream
-     (lambda (&rest args)
-       (let ((socks-override-functions nil))
-         (apply #'open-network-stream args)))
-     name buffer host service)))
-
-(defun socks--open-network-stream (orig-fun name buffer host service &rest params)
-  (let ((route (and socks-override-functions
-                    (socks-find-route host service))))
-    (if (not route)
-	(apply orig-fun name buffer host service params)
-      ;; FIXME: Obey `params'!
-      (let* ((proc (socks-open-connection route))
-	     (version (process-get proc 'socks-server-protocol))
+(defun socks-open-network-stream (name buffer host service &rest params)
+  (if-let* ((route (socks-find-route host service))
+            (proc (socks-open-connection route #'open-network-stream)))
+      (socks--open-network-stream proc buffer host service)
+    (message "Warning: no SOCKS route found for %s:%s" host service)
+    ;; Support legacy behavior (likely undesirable in most cases)
+    (apply #'open-network-stream name buffer host service params)))
+
+(defun socks--open-network-stream (proc buffer host service)
+  (progn ; temporarily preserve git blame for easier reviewing
+    (progn ; could rename to something like `socks--initiate-command-sequence'
+      (let* ((version (process-get proc 'socks-server-protocol))
              (atype
               (cond
                ((equal version 4)
diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 4963dd7b40..f2600210b0 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -63,21 +63,21 @@ socks-tests-filter-response-parsing-v4
     (process-put proc 'socks-state socks-state-waiting)
     (process-put proc 'socks-server-protocol 4)
     (ert-info ("Receive initial incomplete segment")
-      (socks-filter proc (concat [0 90 0 0 93 184 216]))
-      ;; From example.com: OK status ^      ^ msg start
+      (socks-filter proc (unibyte-string 0 90 0 0 93 184 216))
+      ;; From example.com: OK status       ^      ^ msg start
       (ert-info ("State still set to waiting")
         (should (eq (process-get proc 'socks-state) socks-state-waiting)))
       (ert-info ("Response field is nil because processing incomplete")
         (should-not (process-get proc 'socks-response)))
       (ert-info ("Scratch field holds stashed partial payload")
-        (should (string= (concat [0 90 0 0 93 184 216])
+        (should (string= (unibyte-string 0 90 0 0 93 184 216)
                          (process-get proc 'socks-scratch)))))
     (ert-info ("Last part arrives")
       (socks-filter proc "\42") ; ?\" 34
       (ert-info ("State transitions to complete (length check passes)")
         (should (eq (process-get proc 'socks-state) socks-state-connected)))
       (ert-info ("Scratch and response fields hold stash w. last chunk")
-        (should (string= (concat [0 90 0 0 93 184 216 34])
+        (should (string= (unibyte-string 0 90 0 0 93 184 216 34)
                          (process-get proc 'socks-response)))
         (should (string= (process-get proc 'socks-response)
                          (process-get proc 'socks-scratch)))))
@@ -137,10 +137,10 @@ socks-tests-canned-server-create
          (pats socks-tests-canned-server-patterns)
          (filt (lambda (proc line)
                  (pcase-let ((`(,pat . ,resp) (pop pats)))
-                   (setq resp (apply #'unibyte-string (append resp nil)))
                    (unless (or (and (vectorp pat) (equal pat (vconcat line)))
                                (string-match-p pat line))
                      (error "Unknown request: %s" line))
+                   (setq resp (apply #'unibyte-string (append resp nil)))
                    (let ((print-escape-control-characters t))
                      (message "[%s] <- %s" name (prin1-to-string line))
                      (message "[%s] -> %s" name (prin1-to-string resp)))
@@ -374,11 +374,11 @@ test-socks-https-poc
                             #'socks-tor-resolve)
                 (should-not (nsm-host-settings id))
                 (url-http url cb '(nil))
-                (should (nsm-host-settings id))
                 (ert-info ("Wait for response")
                   (with-timeout (3 (error "Request timed out"))
                     (unless done
-                      (sleep-for 0.1)))))
+                      (sleep-for 0.1))))
+                (should (nsm-host-settings id)))
             (advice-remove 'network-lookup-address-info
                            #'socks-tor-resolve)))))))
 
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0001-Simplify-network-stream-opener-in-socks.el.patch

From e1b377ee054f95a4f2064eef6972d350f69767f3 Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Tue, 1 Mar 2022 02:12:02 -0800
Subject: [PATCH 1/5] Simplify network-stream opener in socks.el

* lisp/net/socks.el (socks-override-functions): Make variable
obsolete and remove uses.
(socks-open-connection): Add optional opener arg.
(socks-open-network-stream): Accept additional params for calling
`open-network-stream' as a fallback when a route cannot be found.
(socks--open-network-stream): Reduce role to merely issuing the first
command using an existing process.  Change signature accordingly.
---
 lisp/net/socks.el | 50 ++++++++++++++++++++++-------------------------
 1 file changed, 23 insertions(+), 27 deletions(-)

diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index 8df0773e1d..9bc301618c 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -323,19 +323,19 @@ socks-filter
 
 (defvar socks-override-functions nil
   "If non-nil, overwrite `open-network-stream' function with SOCKSified version.")
-
-(when socks-override-functions
-  (advice-add 'open-network-stream :around #'socks--open-network-stream))
-
-(defun socks-open-connection (server-info)
+(make-obsolete-variable 'socks-override-functions
+                        "`socks--open-network-stream' now takes a process arg."
+                        "29.1")
+
+(defun socks-open-connection (server-info &optional opener)
+  "Create and initialize a SOCKS process.
+Perform authentication if needed.  SERVER-INFO should resemble
+`socks-server'.  OPENER, when present, should be a substitute for
+`open-network-stream' and take the same arguments."
   (interactive)
   (save-excursion
-    (let ((proc
-           (let ((socks-override-functions nil))
-             (open-network-stream "socks"
-				  nil
-				  (nth 1 server-info)
-				  (nth 2 server-info))))
+    (let ((proc (funcall (or opener #'open-network-stream)
+                         "socks" nil (nth 1 server-info) (nth 2 server-info)))
 	  (authtype nil)
 	  version)
 
@@ -508,22 +508,18 @@ socks-find-services-entry
   (gethash (downcase service)
 	      (if udp socks-udp-services socks-tcp-services)))
 
-(defun socks-open-network-stream (name buffer host service)
-  (let ((socks-override-functions t))
-    (socks--open-network-stream
-     (lambda (&rest args)
-       (let ((socks-override-functions nil))
-         (apply #'open-network-stream args)))
-     name buffer host service)))
-
-(defun socks--open-network-stream (orig-fun name buffer host service &rest params)
-  (let ((route (and socks-override-functions
-                    (socks-find-route host service))))
-    (if (not route)
-	(apply orig-fun name buffer host service params)
-      ;; FIXME: Obey `params'!
-      (let* ((proc (socks-open-connection route))
-	     (version (process-get proc 'socks-server-protocol))
+(defun socks-open-network-stream (name buffer host service &rest params)
+  (if-let* ((route (socks-find-route host service))
+            (proc (socks-open-connection route #'open-network-stream)))
+      (socks--open-network-stream proc buffer host service)
+    (message "Warning: no SOCKS route found for %s:%s" host service)
+    ;; Support legacy behavior (likely undesirable in most cases)
+    (apply #'open-network-stream name buffer host service params)))
+
+(defun socks--open-network-stream (proc buffer host service)
+  (progn ; temporarily preserve git blame for easier reviewing
+    (progn ; could rename to something like `socks--initiate-command-sequence'
+      (let* ((version (process-get proc 'socks-server-protocol))
              (atype
               (cond
                ((equal version 4)
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0002-Fix-string-encoding-bug-in-socks-tests.patch

From 8f33588517c7333d3bd08375c406cd46726b51d6 Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 2/5] Fix string encoding bug in socks tests

* test/lisp/net/socks-tests.el (socks-tests-canned-server-create,
socks-tests-filter-response-parsing-v4): Fix bug in process filter to
prevent prepared outgoing responses from being implicitly encoded as
utf-8.  Fix similar mistake in v4 filter test.
---
 test/lisp/net/socks-tests.el | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 461796bdf9..d9ef53ae35 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -63,21 +63,21 @@ socks-tests-filter-response-parsing-v4
     (process-put proc 'socks-state socks-state-waiting)
     (process-put proc 'socks-server-protocol 4)
     (ert-info ("Receive initial incomplete segment")
-      (socks-filter proc (concat [0 90 0 0 93 184 216]))
-      ;; From example.com: OK status ^      ^ msg start
+      (socks-filter proc (unibyte-string 0 90 0 0 93 184 216))
+      ;; From example.com: OK status       ^      ^ msg start
       (ert-info ("State still set to waiting")
         (should (eq (process-get proc 'socks-state) socks-state-waiting)))
       (ert-info ("Response field is nil because processing incomplete")
         (should-not (process-get proc 'socks-response)))
       (ert-info ("Scratch field holds stashed partial payload")
-        (should (string= (concat [0 90 0 0 93 184 216])
+        (should (string= (unibyte-string 0 90 0 0 93 184 216)
                          (process-get proc 'socks-scratch)))))
     (ert-info ("Last part arrives")
       (socks-filter proc "\42") ; ?\" 34
       (ert-info ("State transitions to complete (length check passes)")
         (should (eq (process-get proc 'socks-state) socks-state-connected)))
       (ert-info ("Scratch and response fields hold stash w. last chunk")
-        (should (string= (concat [0 90 0 0 93 184 216 34])
+        (should (string= (unibyte-string 0 90 0 0 93 184 216 34)
                          (process-get proc 'socks-response)))
         (should (string= (process-get proc 'socks-response)
                          (process-get proc 'socks-scratch)))))
@@ -140,10 +140,11 @@ socks-tests-canned-server-create
                    (unless (or (and (vectorp pat) (equal pat (vconcat line)))
                                (string-match-p pat line))
                      (error "Unknown request: %s" line))
+                   (setq resp (apply #'unibyte-string (append resp nil)))
                    (let ((print-escape-control-characters t))
                      (message "[%s] <- %s" name (prin1-to-string line))
                      (message "[%s] -> %s" name (prin1-to-string resp)))
-                   (process-send-string proc (concat resp)))))
+                   (process-send-string proc resp))))
          (serv (make-network-process :server 1
                                      :buffer (get-buffer-create name)
                                      :filter filt
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0003-Add-support-for-SOCKS-4a.patch

From b90a6474b6edb4dd33cffa0e05f1a7f1a3e1c9be Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 3/5] Add support for SOCKS 4a

* lisp/net/socks.el (socks-server): Add new choice `4a' to version
field of option.  This may appear to change the type of the field from
a number to a union of symbols and numbers.  However,
`socks-send-command' and `socks-filter' already expect a possible
`http' value for this field (also a symbol).
(socks--errors-4): Add new constant containing error messages for
socks version 4.  The semantics are faithful, but the wording is
ad-libbed.
(socks-send-command): Massage existing handling for version 4 to
accommodate 4a.

* test/lisp/net/socks-tests.el (socks-tests-v4a-basic): add test for
4a.
Bug#53941
---
 lisp/net/socks.el            | 22 ++++++++++++++++++++--
 test/lisp/net/socks-tests.el | 13 +++++++++++++
 2 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index 9bc301618c..0615db8681 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -162,6 +162,7 @@ socks-server
 	  (radio-button-choice :tag "SOCKS Version"
 			       :format "%t: %v"
 			       (const :tag "SOCKS v4  " :format "%t" :value 4)
+                               (const :tag "SOCKS v4a"  :format "%t" :value 4a)
 			       (const :tag "SOCKS v5"   :format "%t" :value 5))))
 
 
@@ -202,6 +203,12 @@ socks-errors
     "Command not supported"
     "Address type not supported"))
 
+(defconst socks--errors-4
+  '("Granted"
+    "Rejected or failed"
+    "Cannot connect to identd on the client"
+    "Client and identd report differing user IDs"))
+
 ;; The socks v5 address types
 (defconst socks-address-type-v4   1)
 (defconst socks-address-type-name 3)
@@ -400,6 +407,7 @@ socks-send-command
 		(format "%c%s" (length address) address))
 	       (t
 		(error "Unknown address type: %d" atype))))
+        trailing
 	request version)
     (or (process-get proc 'socks)
         (error "socks-send-command called on non-SOCKS connection %S" proc))
@@ -417,6 +425,12 @@ socks-send-command
 			     (t
 			      (error "Unsupported address type for HTTP: %d" atype)))
 			    port)))
+     ((when (eq version '4a)
+        (setf addr "\0\0\0\1"
+              trailing (concat address "\0")
+              version 4 ; done with the "a" part
+              (process-get proc 'socks-server-protocol) 4)
+        nil)) ; fall through
      ((equal version 4)
       (setq request (concat
 		     (unibyte-string
@@ -426,7 +440,8 @@ socks-send-command
 		      (logand port #xff)) ; port, low byte
 		     addr                 ; address
 		     (user-full-name)     ; username
-		     "\0")))              ; terminate username
+                     "\0"                 ; terminate username
+                     trailing)))          ; optional host to look up
      ((equal version 5)
       (setq request (concat
 		     (unibyte-string
@@ -447,7 +462,10 @@ socks-send-command
 	nil				; Sweet sweet success!
       (delete-process proc)
       (error "SOCKS: %s"
-             (nth (or (process-get proc 'socks-reply) 1) socks-errors)))
+             (let ((no (or (process-get proc 'socks-reply) 1)))
+               (if (eq version 5)
+                   (nth no socks-errors)
+                 (nth (+ 90 no) socks--errors-4)))))
     proc))
 
 
diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index d9ef53ae35..4e990ffdba 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -207,6 +207,19 @@ socks-tests-v4-basic
                  (lambda (&optional _) "foo")))
         (socks-tests-perform-hello-world-http-request)))))
 
+(ert-deftest socks-tests-v4a-basic ()
+  "Show correct preparation of SOCKS4a connect command."
+  (let ((socks-server '("server" "127.0.0.1" 10083 4a))
+        (url-user-agent "Test/4a-basic")
+        (socks-tests-canned-server-patterns
+         `(([4 1 0 80 0 0 0 1 ?f ?o ?o 0 ?e ?x ?a ?m ?p ?l ?e ?. ?c ?o ?m 0]
+            . [0 90 0 0 0 0 0 0])
+           ,socks-tests--hello-world-http-request-pattern)))
+    (ert-info ("Make HTTP request over SOCKS4A")
+      (cl-letf (((symbol-function 'user-full-name)
+                 (lambda (&optional _) "foo")))
+        (socks-tests-perform-hello-world-http-request)))))
+
 ;; Replace first pattern below with ([5 3 0 1 2] . [5 2]) to validate
 ;; against curl 7.71 with the following options:
 ;; $ curl --verbose -U foo:bar --proxy socks5h://127.0.0.1:10080 example.com
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0004-Support-SOCKS-RESOLVE-extension.patch

From 23a430c6d7fb2707dba7e217f279ba293ae2fce6 Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 4/5] Support SOCKS RESOLVE extension

* lisp/net/socks.el (socks-resolve-command): Add new constant for the
SOCKS command RESOLVE, which comes by way of a nonstandard extension
from the TOR project.  It mirrors CONNECT in most respects but asks
the server to RESOLVE a host name and return its IP.  For details, see
https://github.com/torproject/torspec/blob/master/socks-extensions.txt
This shouldn't be confused with 5h/5-hostname, which is used to by
clients like cURL to allow users to bypass attempts to resolve a name
locally.
(socks--extract-resolve-response, socks-tor-resolve): Add utility
functions to query a SOCKS service supporting the RESOLVE extension.
Bug#53941
---
 lisp/net/socks.el            | 58 ++++++++++++++++++++++++++++++++++++
 test/lisp/net/socks-tests.el | 34 +++++++++++++++++++++
 2 files changed, 92 insertions(+)

diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index 0615db8681..cd026fd163 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -181,6 +181,9 @@ socks-udp-associate-command
 (defconst socks-authentication-null 0)
 (defconst socks-authentication-failure 255)
 
+;; Extensions
+(defconst socks-resolve-command #xf0)
+
 ;; Response codes
 (defconst socks-response-success               0)
 (defconst socks-response-general-failure       1)
@@ -649,6 +652,61 @@ socks-nslookup-host
 	res)
     host))
 
+(defun socks--extract-resolve-response (proc)
+  "Parse response for PROC and maybe return destination IP address."
+  (let ((response (process-get proc 'socks-response)))
+    (cl-assert response) ; otherwise, msg not received in its entirety
+    (pcase (process-get proc 'socks-server-protocol)
+      (4 ; https://www.openssh.com/txt/socks4a.protocol
+       (when-let (((zerop (process-get proc 'socks-reply)))
+                  ((eq (aref response 1) 90)) ; #x5a request granted
+                  (a (substring response 4)) ; ignore port for now
+                  ((not (string-empty-p a)))
+                  ((not (string= a "\0\0\0\0"))))
+         a))
+      (5 ; https://tools.ietf.org/html/rfc1928
+       (cl-assert (eq 5 (aref response 0)) t)
+       (pcase (aref response 3) ; ATYP
+         (1 (and-let* ((a (substring response 4 8))
+                       ((not (string= a "\0\0\0\0")))
+                       a)))
+         ;; No reason to support RESOLVE_PTR [F1] extension, right?
+         (3 (let ((len (1- (aref response 4))))
+              (substring response 5 (+ 5 len))))
+         (4 (substring response 4 20)))))))
+
+(declare-function puny-encode-domain "puny" (domain))
+
+(defun socks-tor-resolve (name &optional _family)
+  "Return list of one vector IPv4 address for domain NAME.
+Or return nil on failure.  See `network-lookup-address-info' for format
+of return value.  Server must support the Tor RESOLVE command."
+  (let ((socks-password (or socks-password ""))
+        host
+        (port 80)  ; unused for now
+        proc
+        ip)
+    (unless (string-suffix-p ".onion" name)
+      (setq host (if (string-match "\\`[[:ascii:]]+\\'" name)
+                     name
+                   (require 'puny)
+                   (puny-encode-domain name)))
+      ;; "Host unreachable" may be raised when the lookup fails
+      (unwind-protect
+          (progn
+            (setq proc (socks-open-connection (socks-find-route host port)))
+            (socks-send-command proc
+                                socks-resolve-command
+                                socks-address-type-name
+                                host
+                                port)
+            (cl-assert (eq (process-get proc 'socks-state)
+                           socks-state-connected))
+            (setq ip (socks--extract-resolve-response proc)))
+        (when proc
+          (delete-process proc)))
+      (list (vconcat ip [0])))))
+
 (provide 'socks)
 
 ;;; socks.el ends here
diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 4e990ffdba..3d1aca9af4 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -295,4 +295,38 @@ socks-tests-v5-auth-none
       (socks-tests-perform-hello-world-http-request)))
   (should (assq 2 socks-authentication-methods)))
 
+(ert-deftest tor-resolve-4a ()
+  "Make request to TOR resolve service over SOCKS4a"
+  (let* ((socks-server '("server" "127.0.0.1" 19050 4a))
+         (socks-tests-canned-server-patterns
+          '(([4 #xf0 0 80 0 0 0 1 ?f ?o ?o 0 ?e ?x ?a ?m ?p ?l ?e ?. ?c ?o ?m 0]
+             . [0 90 0 0 93 184 216 34])))
+         (inhibit-message noninteractive)
+         (server (socks-tests-canned-server-create)))
+    (ert-info ("Query TOR RESOLVE service over SOCKS4")
+      (cl-letf (((symbol-function 'user-full-name)
+                 (lambda (&optional _) "foo")))
+        (should (equal '([93 184 216 34 0])
+                       (socks-tor-resolve "example.com")))))
+    (kill-buffer (process-buffer server))
+    (delete-process server)))
+
+(ert-deftest tor-resolve-5 ()
+  "Make request to TOR resolve service over SOCKS5"
+  (let* ((socks-server '("server" "127.0.0.1" 19051 5))
+         (socks-username "foo")
+         (socks-authentication-methods (append socks-authentication-methods
+                                               nil))
+         (inhibit-message noninteractive)
+         (socks-tests-canned-server-patterns
+          '(([5 2 0 2] . [5 2])
+            ([1 3 ?f ?o ?o 0] . [1 0])
+            ([5 #xf0 0 3 11 ?e ?x ?a ?m ?p ?l ?e ?. ?c ?o ?m 0 80]
+             . [5 0 0 1 93 184 216 34 0 0])))
+         (server (socks-tests-canned-server-create)))
+    (ert-info ("Query TOR RESOLVE service over SOCKS5")
+      (should (equal '([93 184 216 34 0]) (socks-tor-resolve "example.com"))))
+    (kill-buffer (process-buffer server))
+    (delete-process server)))
+
 ;;; socks-tests.el ends here
-- 
2.35.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0005-POC-Demo-SOCKS-RESOLVE-over-HTTPS.patch

From 598e8471789bd6e7eb5a7f3ebc1bbed3cf61f4c6 Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 5/5] [POC] Demo SOCKS RESOLVE over HTTPS

* test/lisp/net/socks-test.el (test-socks-https-poc): Provide
throwaway test demoing an HTTPS connection over a TOR proxy service.
---
 test/lisp/net/socks-tests.el | 55 +++++++++++++++++++++++++++++++++++-
 1 file changed, 54 insertions(+), 1 deletion(-)

diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 3d1aca9af4..f2600210b0 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -21,7 +21,7 @@
 
 ;;; Code:
 
-(require 'ert)
+(require 'ert-x)
 (require 'socks)
 (require 'url-http)
 
@@ -329,4 +329,57 @@ tor-resolve-5
     (kill-buffer (process-buffer server))
     (delete-process server)))
 
+(defvar test-socks-service ; "127.0.0.1:1080" -> ("127.0.0.1", 1080)
+  (when-let ((present (getenv "TEST_SOCKS_SERVICE"))
+             (parts (split-string present ":")))
+    (list (car parts) (string-to-number (cadr parts)))))
+
+(declare-function gnutls-negotiate "gnutls"
+                  (&rest spec
+                         &key process type hostname priority-string
+                         trustfiles crlfiles keylist min-prime-bits
+                         verify-flags verify-error verify-hostname-error
+                         &allow-other-keys))
+
+(ert-deftest test-socks-https-poc ()
+  :tags '(:unstable)
+  (unless test-socks-service (ert-skip "SOCKS service missing"))
+  (unless (gnutls-available-p) (ert-skip "SOCKS resolve test needs GNUTLS"))
+  (ert-with-temp-file tempfile
+    :prefix "emacs-test-socks-network-security-"
+    (let* ((socks-server `("tor" ,@test-socks-service 5))
+           (socks-password "")
+           (nsm-settings-file tempfile)
+           (url-gateway-method 'socks)
+           (id "sha1:df77269389e537fcc9a5fe61667133b5bb97d42e")
+           (host "check.torproject.org")
+           (url (url-generic-parse-url "https://check.torproject.org"))
+           ;;
+           done
+           ;;
+           (cb (lambda (&rest _r)
+                 (goto-char (point-min))
+                 (should (search-forward "Congratulations" nil t))
+                 (setq done t)))
+           (orig (symbol-function #'socks--open-network-stream)))
+      (cl-letf (((symbol-function 'socks--open-network-stream)
+                 (lambda (&rest rest)
+                   (let ((proc (apply orig rest)))
+                     (gnutls-negotiate :process proc :hostname host)
+                     (should (nsm-verify-connection proc host 443 t))))))
+        (ert-info ("Connect to HTTPS endpoint over Tor SOCKS proxy")
+          (unwind-protect
+              (progn
+                (advice-add 'network-lookup-address-info :override
+                            #'socks-tor-resolve)
+                (should-not (nsm-host-settings id))
+                (url-http url cb '(nil))
+                (ert-info ("Wait for response")
+                  (with-timeout (3 (error "Request timed out"))
+                    (unless done
+                      (sleep-for 0.1))))
+                (should (nsm-host-settings id)))
+            (advice-remove 'network-lookup-address-info
+                           #'socks-tor-resolve)))))))
+
 ;;; socks-tests.el ends here
-- 
2.35.1


--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#53941; Package emacs. Full text available.

Message received at 53941 <at> debbugs.gnu.org:


Received: (at 53941) by debbugs.gnu.org; 21 Feb 2022 15:02:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 21 10:02:15 2022
Received: from localhost ([127.0.0.1]:38608 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nMACc-0005XN-KB
	for submit <at> debbugs.gnu.org; Mon, 21 Feb 2022 10:02:15 -0500
Received: from mail-108-mta1.mxroute.com ([136.175.108.1]:40489)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jp@HIDDEN>) id 1nMACa-0005S1-La
 for 53941 <at> debbugs.gnu.org; Mon, 21 Feb 2022 10:02:13 -0500
Received: from filter006.mxroute.com ([140.82.40.27] 140.82.40.27.vultr.com)
 (Authenticated sender: mN4UYu2MZsgR)
 by mail-108-mta1.mxroute.com (ZoneMTA) with ESMTPSA id 17f1ccdc5b20005a20.001
 for <53941 <at> debbugs.gnu.org>
 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256);
 Mon, 21 Feb 2022 15:02:01 +0000
X-Zone-Loop: 2557600d9f6b113017a2a58ac6ba48e0ece56b75b38c
X-Originating-IP: [140.82.40.27]
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=neverwas.me
 ; s=x;
 h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:References:
 Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID:
 Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
 :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:
 List-Post:List-Owner:List-Archive;
 bh=figKuNx6501QgoeDkYL1r+901Iay72/4c65H1ue+nsY=; b=A0zy5VbOV/QnxGnibBrCwcYrc6
 LYEzkkTfKwm0OKxoxgyXX4gFwcUXZtbgRhbNlGX8pp0UweNbCY4xpK7NCn7iR43ws7GYF/88EOU28
 VKEiEP6PFTfelj2g7d2idQE9pgILIsVu0HAoxQ5GqlKPAexG7idjn4lU1OuIWBynDVaWodVhjQ7ZD
 e+eKrn0ZPNqyjzZ2rs6r7ovtfVy/nITG8I76hPR3qsEVw5P1xy6S68889v+CTikCEfLPaBDh2QFtd
 vWnDxO/A/nI5zZufsma7NAU22cBVEQEEpkiGAZW33k3F5VqAiByxyuxfz8azi/m7k7ax6wusyC1Qf
 VDpQ64Uw==;
From: "J.P." <jp@HIDDEN>
To: Jacobo <gnuhacker@HIDDEN>
Subject: Re: bug#53941: 27.2; socks + tor dont work with https
References: <87pmntfym7.fsf@HIDDEN> <8735kl1v58.fsf@HIDDEN>
 <87a6emftzx.fsf@HIDDEN>
Date: Mon, 21 Feb 2022 07:01:58 -0800
In-Reply-To: <87a6emftzx.fsf@HIDDEN> (Jacobo's message of "Sat, 19 Feb
 2022 22:04:34 +0100")
Message-ID: <87k0do5km1.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-AuthUser: masked@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 53941
Cc: 53941 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Jacobo <gnuhacker@HIDDEN> writes:

>> In this day and age, when processes and services resolve host names in
>> all manner of ways, how can we be confident there won't be any leaks?
>
> oh, nevermind

I certainly don't want to discourage anyone from trying to solve this.
But DNS leaks aside, predicting what ought to be proxied still seems
like a serious undertaking (at least from my peabrained perspective).
For example, if you connect to an IRC network over Tor and click a
hyperlink in a channel, should the resulting connection also happen over
Tor? What about when the SOCKS service isn't Tor but something else,
like SSH? Should similar follow-on connections also originate from the
proxy host (your VPS or shell provider, for example)?

>> I also have some examples with shims for 27 running periodically in CI.
>> These include a demo of using ERC to connect to Libera.Chat via SOCKS
>> over TLS. (But that requires an IRCv3 library, which is still a work in
>> progress.)
>
> Now Ive tryed same in other computer with Trisquel 10, the problem still
> happend

Sorry, are you saying you repeated the steps in your original post and
got the same result (failure) on another computer? If so, that's to be
expected because Tor over SOCKS with TLS isn't supported OOTB with any
Emacs, not even 29. Apologies if I implied otherwise.

>> If you're interested in experimenting with any of this stuff, please
>> let me know. That goes for anyone else out there as well. Thanks.
>
> yes, how can I help?

When 28 comes out, you can try applying those patches. Or, if you're not
cool with that, I can give you a replacement socks.el to shadow the
original. To check whether it's working, do

  M-: (boundp 'socks--errors-4) RET

or similar and then try mimicking the recipe in that last patch (the one
named POC demo something). Once that works, try adapting it to your
needs for whatever protocol (except for ERC, for which you'll need to
install an unofficial WIP version).




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#53941; Package emacs. Full text available.

Message received at 53941 <at> debbugs.gnu.org:


Received: (at 53941) by debbugs.gnu.org; 19 Feb 2022 21:04:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 19 16:04:44 2022
Received: from localhost ([127.0.0.1]:58813 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nLWuJ-0000Kd-UO
	for submit <at> debbugs.gnu.org; Sat, 19 Feb 2022 16:04:44 -0500
Received: from mx1.riseup.net ([198.252.153.129]:44756)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <gnuhacker@HIDDEN>) id 1nLWuI-0000KO-E7
 for 53941 <at> debbugs.gnu.org; Sat, 19 Feb 2022 16:04:43 -0500
Received: from fews1.riseup.net (fews1-pn.riseup.net [10.0.1.83])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "mail.riseup.net", Issuer "R3" (not verified))
 by mx1.riseup.net (Postfix) with ESMTPS id 4K1LfX4FSwzF44M;
 Sat, 19 Feb 2022 13:04:36 -0800 (PST)
X-Riseup-User-ID: C84BA236F5101290351971B5CC63B5D706A0B5CABE282B6EA335C44739BD944D
Received: from [127.0.0.1] (localhost [127.0.0.1])
 by fews1.riseup.net (Postfix) with ESMTPSA id 4K1LfW5r9pz5vMd;
 Sat, 19 Feb 2022 13:04:35 -0800 (PST)
From: Jacobo <gnuhacker@HIDDEN>
To: "J.P." <jp@HIDDEN>
Subject: Re: bug#53941: 27.2; socks + tor dont work with https
References: <87pmntfym7.fsf@HIDDEN> <8735kl1v58.fsf@HIDDEN>
Date: Sat, 19 Feb 2022 22:04:34 +0100
In-Reply-To: <8735kl1v58.fsf@HIDDEN> (J. P.'s message of "Mon, 14 Feb
 2022 04:37:39 -0800")
Message-ID: <87a6emftzx.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 53941
Cc: 53941 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

"J.P." <jp@HIDDEN> writes:
> Jacobo <gnuhacker@HIDDEN> writes:
>> Emacs can not resolve domains when it is https if you are using a
>> socks proxy (socks.el) [...] It works, load http://gnu.org (HTTP in
>> plain) Also work with .onion domains in HTTP plain No problems with
>> HTTP but When I try: M-x eww RET https://gnu.org RET
>> Return an error: Bad Request

> It's certainly possible (see attached). But can it be done responsibly?
> In this day and age, when processes and services resolve host names in
> all manner of ways, how can we be confident there won't be any leaks?

oh, nevermind

> At present, the main interfaces to various protocol stacks (for
> example, url-gw.el and friends) don't seem geared toward making those
> kinds of assurances. (Not that they ought to be.)  That said,
> providing the building blocks on the SOCKS side doesn't seem like the
> crime of the century. I've been sitting on what became the basis for
> these patches for a while now, but these here were hastily adapted and
> might come with some warts. Still, I believe them straightforward
> enough to illustrate a basic means of achieving what you're after.

>> In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30,

> I also have some examples with shims for 27 running periodically in CI.
> These include a demo of using ERC to connect to Libera.Chat via SOCKS
> over TLS. (But that requires an IRCv3 library, which is still a work in
> progress.)

Now Ive tryed same in other computer with Trisquel 10, the problem still
happend

> If you're interested in experimenting with any of this stuff, please
> let me know. That goes for anyone else out there as well. Thanks.

yes, how can I help?

-- 
Emacs Lover.
FSF Member.
Free/Libre Software supporter.
stallmansupport.org - Disinformation succeeds because so many people
care deeply about injustice but do not take the time to check the facts.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#53941; Package emacs. Full text available.
Added tag(s) patch. Request was from "J.P." <jp@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 53941 <at> debbugs.gnu.org:


Received: (at 53941) by debbugs.gnu.org; 14 Feb 2022 12:37:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Feb 14 07:37:58 2022
Received: from localhost ([127.0.0.1]:40106 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nJac4-00063l-TO
	for submit <at> debbugs.gnu.org; Mon, 14 Feb 2022 07:37:58 -0500
Received: from mail-108-mta150.mxroute.com ([136.175.108.150]:37557)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jp@HIDDEN>) id 1nJac1-00063P-JU
 for 53941 <at> debbugs.gnu.org; Mon, 14 Feb 2022 07:37:51 -0500
Received: from filter006.mxroute.com ([140.82.40.27] 140.82.40.27.vultr.com)
 (Authenticated sender: mN4UYu2MZsgR)
 by mail-108-mta150.mxroute.com (ZoneMTA) with ESMTPSA id
 17ef83d221b0005a20.001 for <53941 <at> debbugs.gnu.org>
 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256);
 Mon, 14 Feb 2022 12:37:43 +0000
X-Zone-Loop: 7d9641cbbe00570d41177f6add9f3c394b9783e5d52e
X-Originating-IP: [140.82.40.27]
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=neverwas.me
 ; s=x;
 h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:References:
 Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID:
 Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
 :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:
 List-Post:List-Owner:List-Archive;
 bh=4I70xsFPdMkL1jol51sIg2ChZOP+jHKU8IgodPtvWCg=; b=YcHrlI5U2TnzgJ4jJGKmkqDGSQ
 TNWpK3YTLGWvmxBsjvhMSp5uOjDHP4fPq5RAtVMH4PzqcJw0Tp0wwzHXqLOY4twQgVREB1i9mDgGz
 4sTAWuLE8cq209waU0UO7kw91PEvO91h4VIv5CzUaZE3dA1oWAJlYYOhTsNXGX3tYbetfuYBXgQOq
 l9Vf9WdIkn9kKCChDPgRXUZKMn4lW0JsGNaC3CSEX4IDp1LWXXIWwFhi9lql49AOeK5mOYP1dA+7n
 7bgjdHlMi1QWuT6ib/9MqrsRRY1SwZi44ojk4Geh6scxztRPIeW6PhBBN6/MzkwCZkeDXeIb1yxfa
 v/snx4rA==;
From: "J.P." <jp@HIDDEN>
To: Jacobo <gnuhacker@HIDDEN>
Subject: Re: bug#53941: 27.2; socks + tor dont work with https
References: <87pmntfym7.fsf@HIDDEN>
Date: Mon, 14 Feb 2022 04:37:39 -0800
In-Reply-To: <87pmntfym7.fsf@HIDDEN> (Jacobo's message of "Fri, 11 Feb
 2022 12:09:52 +0100")
Message-ID: <8735kl1v58.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-AuthUser: masked@HIDDEN
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 53941
Cc: 53941 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--=-=-=
Content-Type: text/plain

Hi Jacobo,

Jacobo <gnuhacker@HIDDEN> writes:

> Emacs can not resolve domains when it is https if you are using a
> socks proxy (socks.el) [...] It works, load http://gnu.org (HTTP in
> plain) Also work with .onion domains in HTTP plain No problems with
> HTTP but When I try: M-x eww RET https://gnu.org RET
>
> Return an error: Bad Request

It's certainly possible (see attached). But can it be done responsibly?

In this day and age, when processes and services resolve host names in
all manner of ways, how can we be confident there won't be any leaks? At
present, the main interfaces to various protocol stacks (for example,
url-gw.el and friends) don't seem geared toward making those kinds of
assurances. (Not that they ought to be.)

That said, providing the building blocks on the SOCKS side doesn't seem
like the crime of the century. I've been sitting on what became the
basis for these patches for a while now, but these here were hastily
adapted and might come with some warts. Still, I believe them
straightforward enough to illustrate a basic means of achieving what
you're after.

> In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30,

I also have some examples with shims for 27 running periodically in CI.
These include a demo of using ERC to connect to Libera.Chat via SOCKS
over TLS. (But that requires an IRCv3 library, which is still a work in
progress.) If you're interested in experimenting with any of this stuff,
please let me know. That goes for anyone else out there as well. Thanks.


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0001-Set-coding-system-for-SOCKS-connections-to-binary.patch

From 1cf058fe106e01d55e9269503994e2e9b274b07a Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 1/4] Set coding system for SOCKS connections to binary

* lisp/net/socks.el (socks-opens-connection): Don't perform
conversions when receiving and sending text.

* test/lisp/net/socks-tests.el (socks-tests-canned-server-create): Fix
bug in process filter to prevent prepared outgoing responses from
being implicitly encoded as utf-8.
---
 lisp/net/socks.el            | 1 +
 test/lisp/net/socks-tests.el | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index 8df0773e1d..c15b323c9c 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -340,6 +340,7 @@ socks-open-connection
 	  version)
 
       ;; Initialize process and info about the process
+      (set-process-coding-system proc 'binary 'binary)
       (set-process-filter proc #'socks-filter)
       (set-process-query-on-exit-flag proc nil)
       (process-put proc 'socks t)
diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 461796bdf9..708b964020 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -137,13 +137,14 @@ socks-tests-canned-server-create
          (pats socks-tests-canned-server-patterns)
          (filt (lambda (proc line)
                  (pcase-let ((`(,pat . ,resp) (pop pats)))
+                   (setq resp (apply #'unibyte-string (append resp nil)))
                    (unless (or (and (vectorp pat) (equal pat (vconcat line)))
                                (string-match-p pat line))
                      (error "Unknown request: %s" line))
                    (let ((print-escape-control-characters t))
                      (message "[%s] <- %s" name (prin1-to-string line))
                      (message "[%s] -> %s" name (prin1-to-string resp)))
-                   (process-send-string proc (concat resp)))))
+                   (process-send-string proc resp))))
          (serv (make-network-process :server 1
                                      :buffer (get-buffer-create name)
                                      :filter filt
-- 
2.34.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0002-Add-support-for-SOCKS-4a.patch

From 84299e3e9dac1e3620a83cf807b564ee276f0cdf Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 2/4] Add support for SOCKS 4a

* lisp/net/socks.el (socks-server): Add new choice `4a' to version
field of option.  This may appear to change the type of the field from
a number to a union of symbols and numbers.  However,
`socks-send-command' and `socks-filter' already expect a possible
`http' value for this field (also a symbol).
(socks--errors-4): Add new constant containing error messages for
socks version 4.  The semantics are faithful, but the wording is
ad-libbed.
(socks-send-command): Massage existing handling for version 4 to
accommodate 4a.

* test/lisp/net/socks-tests.el (socks-tests-v4a-basic): add test for
4a.
---
 lisp/net/socks.el            | 22 ++++++++++++++++++++--
 test/lisp/net/socks-tests.el | 13 +++++++++++++
 2 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index c15b323c9c..0d5ef307e7 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -162,6 +162,7 @@ socks-server
 	  (radio-button-choice :tag "SOCKS Version"
 			       :format "%t: %v"
 			       (const :tag "SOCKS v4  " :format "%t" :value 4)
+                               (const :tag "SOCKS v4a"  :format "%t" :value 4a)
 			       (const :tag "SOCKS v5"   :format "%t" :value 5))))
 
 
@@ -202,6 +203,12 @@ socks-errors
     "Command not supported"
     "Address type not supported"))
 
+(defconst socks--errors-4
+  '("Granted"
+    "Rejected or failed"
+    "Cannot connect to identd on the client"
+    "Client and identd report differing user IDs"))
+
 ;; The socks v5 address types
 (defconst socks-address-type-v4   1)
 (defconst socks-address-type-name 3)
@@ -401,6 +408,7 @@ socks-send-command
 		(format "%c%s" (length address) address))
 	       (t
 		(error "Unknown address type: %d" atype))))
+        trailing
 	request version)
     (or (process-get proc 'socks)
         (error "socks-send-command called on non-SOCKS connection %S" proc))
@@ -418,6 +426,12 @@ socks-send-command
 			     (t
 			      (error "Unsupported address type for HTTP: %d" atype)))
 			    port)))
+     ((when (eq version '4a)
+        (setf addr "\0\0\0\1"
+              trailing (concat address "\0")
+              version 4 ; done with the "a" part
+              (process-get proc 'socks-server-protocol) 4)
+        nil)) ; fall through
      ((equal version 4)
       (setq request (concat
 		     (unibyte-string
@@ -427,7 +441,8 @@ socks-send-command
 		      (logand port #xff)) ; port, low byte
 		     addr                 ; address
 		     (user-full-name)     ; username
-		     "\0")))              ; terminate username
+                     "\0"                 ; terminate username
+                     trailing)))          ; optional host to look up
      ((equal version 5)
       (setq request (concat
 		     (unibyte-string
@@ -448,7 +463,10 @@ socks-send-command
 	nil				; Sweet sweet success!
       (delete-process proc)
       (error "SOCKS: %s"
-             (nth (or (process-get proc 'socks-reply) 1) socks-errors)))
+             (let ((no (or (process-get proc 'socks-reply) 1)))
+               (if (eq version 5)
+                   (nth no socks-errors)
+                 (nth (+ 90 no) socks--errors-4)))))
     proc))
 
 
diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 708b964020..b81923fc56 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -207,6 +207,19 @@ socks-tests-v4-basic
                  (lambda (&optional _) "foo")))
         (socks-tests-perform-hello-world-http-request)))))
 
+(ert-deftest socks-tests-v4a-basic ()
+  "Show correct preparation of SOCKS4a connect command."
+  (let ((socks-server '("server" "127.0.0.1" 10083 4a))
+        (url-user-agent "Test/4a-basic")
+        (socks-tests-canned-server-patterns
+         `(([4 1 0 80 0 0 0 1 ?f ?o ?o 0 ?e ?x ?a ?m ?p ?l ?e ?. ?c ?o ?m 0]
+            . [0 90 0 0 0 0 0 0])
+           ,socks-tests--hello-world-http-request-pattern)))
+    (ert-info ("Make HTTP request over SOCKS4A")
+      (cl-letf (((symbol-function 'user-full-name)
+                 (lambda (&optional _) "foo")))
+        (socks-tests-perform-hello-world-http-request)))))
+
 ;; Replace first pattern below with ([5 3 0 1 2] . [5 2]) to validate
 ;; against curl 7.71 with the following options:
 ;; $ curl --verbose -U foo:bar --proxy socks5h://127.0.0.1:10080 example.com
-- 
2.34.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0003-Support-SOCKS-RESOLVE-extension.patch

From ffda45081444e14ca687a505f1fc697b8ef59e0f Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 3/4] Support SOCKS RESOLVE extension

* lisp/net/socks.el (socks-resolve-command): Add new constant for the
SOCKS command RESOLVE, which comes by way of a nonstandard extension
from the TOR project.  It mirrors CONNECT in most respects but asks
the server to RESOLVE a host name and return its IP.  For details, see
https://github.com/torproject/torspec/blob/master/socks-extensions.txt
This shouldn't be confused with 5h/5-hostname, which is used to by
clients like cURL to allow users to bypass attempts to resolve a name
locally.
(socks--extract-resolve-response, socks-tor-resolve): Add utility
functions to query a SOCKS service supporting the RESOLVE extension.
---
 lisp/net/socks.el            | 58 ++++++++++++++++++++++++++++++++++++
 test/lisp/net/socks-tests.el | 34 +++++++++++++++++++++
 2 files changed, 92 insertions(+)

diff --git a/lisp/net/socks.el b/lisp/net/socks.el
index 0d5ef307e7..7201ed8e06 100644
--- a/lisp/net/socks.el
+++ b/lisp/net/socks.el
@@ -181,6 +181,9 @@ socks-udp-associate-command
 (defconst socks-authentication-null 0)
 (defconst socks-authentication-failure 255)
 
+;; Extensions
+(defconst socks-resolve-command #xf0)
+
 ;; Response codes
 (defconst socks-response-success               0)
 (defconst socks-response-general-failure       1)
@@ -654,6 +657,61 @@ socks-nslookup-host
 	res)
     host))
 
+(defun socks--extract-resolve-response (proc)
+  "Parse response for PROC and maybe return destination IP address."
+  (let ((response (process-get proc 'socks-response)))
+    (cl-assert response) ; otherwise, msg not received in its entirety
+    (pcase (process-get proc 'socks-server-protocol)
+      (4 ; https://www.openssh.com/txt/socks4a.protocol
+       (when-let (((zerop (process-get proc 'socks-reply)))
+                  ((eq (aref response 1) 90)) ; #x5a request granted
+                  (a (substring response 4)) ; ignore port for now
+                  ((not (string-empty-p a)))
+                  ((not (string= a "\0\0\0\0"))))
+         a))
+      (5 ; https://tools.ietf.org/html/rfc1928
+       (cl-assert (eq 5 (aref response 0)) t)
+       (pcase (aref response 3) ; ATYP
+         (1 (and-let* ((a (substring response 4 8))
+                       ((not (string= a "\0\0\0\0")))
+                       a)))
+         ;; No reason to support RESOLVE_PTR [F1] extension, right?
+         (3 (let ((len (1- (aref response 4))))
+              (substring response 5 (+ 5 len))))
+         (4 (substring response 4 20)))))))
+
+(declare-function puny-encode-domain "puny" (domain))
+
+(defun socks-tor-resolve (name &optional _family)
+  "Return list of one vector IPv4 address for domain NAME.
+Or return nil on failure.  See `network-lookup-address-info' for format
+of return value.  Server must support the Tor RESOLVE command."
+  (let ((socks-password (or socks-password ""))
+        host
+        (port 80)  ; unused for now
+        proc
+        ip)
+    (unless (string-suffix-p ".onion" name)
+      (setq host (if (string-match "\\`[[:ascii:]]+\\'" name)
+                     name
+                   (require 'puny)
+                   (puny-encode-domain name)))
+      ;; "Host unreachable" may be raised when the lookup fails
+      (unwind-protect
+          (progn
+            (setq proc (socks-open-connection (socks-find-route host port)))
+            (socks-send-command proc
+                                socks-resolve-command
+                                socks-address-type-name
+                                host
+                                port)
+            (cl-assert (eq (process-get proc 'socks-state)
+                           socks-state-connected))
+            (setq ip (socks--extract-resolve-response proc)))
+        (when proc
+          (delete-process proc)))
+      (list (vconcat ip [0])))))
+
 (provide 'socks)
 
 ;;; socks.el ends here
diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index b81923fc56..51e2e40631 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -295,4 +295,38 @@ socks-tests-v5-auth-none
       (socks-tests-perform-hello-world-http-request)))
   (should (assq 2 socks-authentication-methods)))
 
+(ert-deftest tor-resolve-4a ()
+  "Make request to TOR resolve service over SOCKS4a"
+  (let* ((socks-server '("server" "127.0.0.1" 19050 4a))
+         (socks-tests-canned-server-patterns
+          '(([4 #xf0 0 80 0 0 0 1 ?f ?o ?o 0 ?e ?x ?a ?m ?p ?l ?e ?. ?c ?o ?m 0]
+             . [0 90 0 0 93 184 216 34])))
+         (inhibit-message noninteractive)
+         (server (socks-tests-canned-server-create)))
+    (ert-info ("Query TOR RESOLVE service over SOCKS4")
+      (cl-letf (((symbol-function 'user-full-name)
+                 (lambda (&optional _) "foo")))
+        (should (equal '([93 184 216 34 0])
+                       (socks-tor-resolve "example.com")))))
+    (kill-buffer (process-buffer server))
+    (delete-process server)))
+
+(ert-deftest tor-resolve-5 ()
+  "Make request to TOR resolve service over SOCKS5"
+  (let* ((socks-server '("server" "127.0.0.1" 19051 5))
+         (socks-username "foo")
+         (socks-authentication-methods (append socks-authentication-methods
+                                               nil))
+         (inhibit-message noninteractive)
+         (socks-tests-canned-server-patterns
+          '(([5 2 0 2] . [5 2])
+            ([1 3 ?f ?o ?o 0] . [1 0])
+            ([5 #xf0 0 3 11 ?e ?x ?a ?m ?p ?l ?e ?. ?c ?o ?m 0 80]
+             . [5 0 0 1 93 184 216 34 0 0])))
+         (server (socks-tests-canned-server-create)))
+    (ert-info ("Query TOR RESOLVE service over SOCKS5")
+      (should (equal '([93 184 216 34 0]) (socks-tor-resolve "example.com"))))
+    (kill-buffer (process-buffer server))
+    (delete-process server)))
+
 ;;; socks-tests.el ends here
-- 
2.34.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0004-POC-Demo-SOCKS-RESOLVE-over-HTTPS.patch

From efe0b1bff206efb6f6559154a560a71239aaa78e Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@HIDDEN>
Date: Mon, 14 Feb 2022 02:36:57 -0800
Subject: [PATCH 4/4] [POC] Demo SOCKS RESOLVE over HTTPS

* test/lisp/net/socks-test.el (test-socks-https-poc): Provide
throwaway test demoing an HTTPS connection over a TOR proxy service.
---
 test/lisp/net/socks-tests.el | 55 +++++++++++++++++++++++++++++++++++-
 1 file changed, 54 insertions(+), 1 deletion(-)

diff --git a/test/lisp/net/socks-tests.el b/test/lisp/net/socks-tests.el
index 51e2e40631..4963dd7b40 100644
--- a/test/lisp/net/socks-tests.el
+++ b/test/lisp/net/socks-tests.el
@@ -21,7 +21,7 @@
 
 ;;; Code:
 
-(require 'ert)
+(require 'ert-x)
 (require 'socks)
 (require 'url-http)
 
@@ -329,4 +329,57 @@ tor-resolve-5
     (kill-buffer (process-buffer server))
     (delete-process server)))
 
+(defvar test-socks-service ; "127.0.0.1:1080" -> ("127.0.0.1", 1080)
+  (when-let ((present (getenv "TEST_SOCKS_SERVICE"))
+             (parts (split-string present ":")))
+    (list (car parts) (string-to-number (cadr parts)))))
+
+(declare-function gnutls-negotiate "gnutls"
+                  (&rest spec
+                         &key process type hostname priority-string
+                         trustfiles crlfiles keylist min-prime-bits
+                         verify-flags verify-error verify-hostname-error
+                         &allow-other-keys))
+
+(ert-deftest test-socks-https-poc ()
+  :tags '(:unstable)
+  (unless test-socks-service (ert-skip "SOCKS service missing"))
+  (unless (gnutls-available-p) (ert-skip "SOCKS resolve test needs GNUTLS"))
+  (ert-with-temp-file tempfile
+    :prefix "emacs-test-socks-network-security-"
+    (let* ((socks-server `("tor" ,@test-socks-service 5))
+           (socks-password "")
+           (nsm-settings-file tempfile)
+           (url-gateway-method 'socks)
+           (id "sha1:df77269389e537fcc9a5fe61667133b5bb97d42e")
+           (host "check.torproject.org")
+           (url (url-generic-parse-url "https://check.torproject.org"))
+           ;;
+           done
+           ;;
+           (cb (lambda (&rest _r)
+                 (goto-char (point-min))
+                 (should (search-forward "Congratulations" nil t))
+                 (setq done t)))
+           (orig (symbol-function #'socks--open-network-stream)))
+      (cl-letf (((symbol-function 'socks--open-network-stream)
+                 (lambda (&rest rest)
+                   (let ((proc (apply orig rest)))
+                     (gnutls-negotiate :process proc :hostname host)
+                     (should (nsm-verify-connection proc host 443 t))))))
+        (ert-info ("Connect to HTTPS endpoint over Tor SOCKS proxy")
+          (unwind-protect
+              (progn
+                (advice-add 'network-lookup-address-info :override
+                            #'socks-tor-resolve)
+                (should-not (nsm-host-settings id))
+                (url-http url cb '(nil))
+                (should (nsm-host-settings id))
+                (ert-info ("Wait for response")
+                  (with-timeout (3 (error "Request timed out"))
+                    (unless done
+                      (sleep-for 0.1)))))
+            (advice-remove 'network-lookup-address-info
+                           #'socks-tor-resolve)))))))
+
 ;;; socks-tests.el ends here
-- 
2.34.1


--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#53941; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 11 Feb 2022 14:31:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Feb 11 09:31:29 2022
Received: from localhost ([127.0.0.1]:58646 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nIWxL-00077A-SP
	for submit <at> debbugs.gnu.org; Fri, 11 Feb 2022 09:31:28 -0500
Received: from lists.gnu.org ([209.51.188.17]:50892)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <gnuhacker@HIDDEN>) id 1nIUI8-0006ku-SE
 for submit <at> debbugs.gnu.org; Fri, 11 Feb 2022 06:40:46 -0500
Received: from eggs.gnu.org ([209.51.188.92]:60914)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <gnuhacker@HIDDEN>)
 id 1nIUI8-0003Em-Ld
 for bug-gnu-emacs@HIDDEN; Fri, 11 Feb 2022 06:40:44 -0500
Received: from mx1.riseup.net ([198.252.153.129]:40512)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <gnuhacker@HIDDEN>)
 id 1nIUI6-0005LF-HC
 for bug-gnu-emacs@HIDDEN; Fri, 11 Feb 2022 06:40:44 -0500
Received: from fews1.riseup.net (fews1-pn.riseup.net [10.0.1.83])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "mail.riseup.net", Issuer "R3" (not verified))
 by mx1.riseup.net (Postfix) with ESMTPS id 4JwBWY1fcwzF6wX
 for <bug-gnu-emacs@HIDDEN>; Fri, 11 Feb 2022 03:40:41 -0800 (PST)
X-Riseup-User-ID: 72825AEC4D3D62747A3BD9DBC2923A4AB12628366A5A452E59997002A98726D4
Received: from [127.0.0.1] (localhost [127.0.0.1])
 by fews1.riseup.net (Postfix) with ESMTPSA id 4JwBWX4jshz5vMh
 for <bug-gnu-emacs@HIDDEN>; Fri, 11 Feb 2022 03:40:40 -0800 (PST)
From: Jacobo <gnuhacker@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 27.2; socks + tor dont work with https
Date: Fri, 11 Feb 2022 12:09:52 +0100
Message-ID: <87pmntfym7.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=198.252.153.129;
 envelope-from=gnuhacker@HIDDEN; helo=mx1.riseup.net
X-Spam_score_int: -25
X-Spam_score: -2.6
X-Spam_bar: --
X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7,
 RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Fri, 11 Feb 2022 09:31:26 -0500
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)


Emacs can not resolve domains when it is https if you are using a socks
proxy (socks.el)

Emacs Config:

#+begin_src elisp

(setq socks-override-functions t)
(setq url-gateway-method 'socks) ; same problem without this line in conf
(require 'socks)

(setq socks-noproxy '("localhost"))

(setq socks-server
  '("TOR"
    "localhost"
    9050
    5))
    
#+end_src


Tor is the socks proxy

Im running tor.

When I try:

M-x eww RET gnu.org RET

It works, load http://gnu.org (HTTP in plain)

Also work with .onion domains in HTTP plain

No problems with HTTP but



When I try:

M-x eww RET https://gnu.org RET


Return an error:

   Bad Request

   Your browser sent a request that this server could not understand.
   Reason: You're speaking plain HTTP to an SSL-enabled server port.
   Instead use the HTTPS scheme to access this URL, please.


In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30, cairo version 1.16.0)
Windowing system distributor 'The X.Org Foundation', version 11.0.12101002
System Description: GNU Guix System

Configured using:
 'configure
 CONFIG_SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash
 SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash
 --prefix=/gnu/store/c4bb68f53mw3sjf6xbyr7ba83csgjdkk-emacs-27.2
 --enable-fast-install --with-modules --with-cairo
 --disable-build-details'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS JSON PDUMPER GMP




Acknowledgement sent to Jacobo <gnuhacker@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#53941; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sun, 6 Mar 2022 02:45:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.