GNU bug report logs - #62598
29.0.60; url-https-proxy-connect doesn't support multi-stage auth to proxies

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Severity: wishlist; Reported by: Spencer Baugh <sbaugh@HIDDEN>; dated Sat, 1 Apr 2023 20:29:01 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.
Severity set to 'wishlist' from 'normal' Request was from Stefan Kangas <stefankangas@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 62598 <at> debbugs.gnu.org:


Received: (at 62598) by debbugs.gnu.org; 9 Sep 2023 14:21:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Sep 09 10:21:27 2023
Received: from localhost ([127.0.0.1]:48199 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qeypy-0003dH-Vd
	for submit <at> debbugs.gnu.org; Sat, 09 Sep 2023 10:21:27 -0400
Received: from mail-108-mta50.mxroute.com ([136.175.108.50]:44187)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jp@HIDDEN>) id 1qeypw-0003d8-GC
 for 62598 <at> debbugs.gnu.org; Sat, 09 Sep 2023 10:21:25 -0400
Received: from mail-111-mta2.mxroute.com ([136.175.111.2]
 filter006.mxroute.com) (Authenticated sender: mN4UYu2MZsgR)
 by mail-108-mta50.mxroute.com (ZoneMTA) with ESMTPSA id 18a7a5183b3000d7b6.001
 for <62598 <at> debbugs.gnu.org>
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384);
 Sat, 09 Sep 2023 14:21:17 +0000
X-Zone-Loop: d96eb850f60afc475a1a7003f00f556cdb6b98608014
X-Originating-IP: [136.175.111.2]
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=neverwas.me
 ; s=x;
 h=Content-Type:MIME-Version:Message-ID:Date:References:In-Reply-To:
 Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID:
 Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
 :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:
 List-Post:List-Owner:List-Archive;
 bh=mZC2goDA107DtecU41kpuHVjsKczhKm+Sbpbrrq4G7s=; b=CoeQkZWBdEj+x2vqHQIMfhnbrq
 siNvIvmatvoL3eRtZsF2hA7EVf+pcTrCYC9NjfZtOC4m49u1SN9NG0YdH5x78aWdLmpbMz3mUNzXH
 VM6k+YM1CxmSIedu7gBXYGFH1an9agolxYTU967EeQgx0rv/OX6R4ZdCiEp2bLnZFm0ebbd0/T93d
 m2KpvzAOy3Ez/YV5ANozZoRiC6m1ahVAQf4i+1wUhbSwPHmttTDKRZppcarIGFT+VaPGWMjArC4Tg
 gNIJjNauQlLSBeXQOcdE8grEdI7mupq7d5at8froM26b7HFOYW4R9ND4ETStlmas5XBAqrVYrPExV
 DVIFZgTw==;
From: "J.P." <jp@HIDDEN>
To: Spencer Baugh <sbaugh@HIDDEN>
Subject: Re: bug#62598: 29.0.60; url-https-proxy-connect doesn't support
 multi-stage auth to proxies
In-Reply-To: <ierlejb2vpk.fsf@HIDDEN> (Spencer Baugh's message of
 "Sat, 01 Apr 2023 16:28:39 -0400")
References: <ierlejb2vpk.fsf@HIDDEN>
Date: Sat, 09 Sep 2023 07:21:13 -0700
Message-ID: <87r0n7a0me.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-Authenticated-Id: masked@HIDDEN
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 62598
Cc: 62598 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi, just wondering if you might be interested in broadening the scope of
this bug into something more ambitious, namely, making proxy handling
more flexible and predictable for libraries doing business with `url'.
I've been tinkering with

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=53941

off and on for a bit, but I'm not familiar enough with the `url'
landscape to go all in. From your bug description, you seem to have a
good handle on the `url-http' parts, so perhaps you're open to exploring
ideas for improving the overall proxy situation `url'-wide. If so, I'd
be willing to investigate how best to adapt `socks' to whatever you
might propose. Just a thought, though (no pressure).




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#62598; Package emacs. Full text available.

Message received at 62598 <at> debbugs.gnu.org:


Received: (at 62598) by debbugs.gnu.org; 5 Apr 2023 23:34:36 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Apr 05 19:34:36 2023
Received: from localhost ([127.0.0.1]:53020 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1pkCeB-0005OR-RD
	for submit <at> debbugs.gnu.org; Wed, 05 Apr 2023 19:34:36 -0400
Received: from mail.fitzsim.org ([69.165.165.189]:44420)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <fitzsim@HIDDEN>) id 1pkCeA-0005OE-2x
 for 62598 <at> debbugs.gnu.org; Wed, 05 Apr 2023 19:34:35 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=fitzsim.org
 ; s=20220430;
 h=Content-Type:MIME-Version:Message-ID:Date:References:
 In-Reply-To:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:
 Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender:
 Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:
 List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=gJeeQXvcjJEZ0GfCiGpffsJRymZ7E06k5ajWLjWCheM=; b=EEHEasbFy0Tp6Vq01+8M7ZfDN4
 InLGlwOSGUM4M8JeFNnbunSXhAUGBlchSMqnea8MQXiu6N/UAOQRE5Qltl15ixAlVSHSKhQynzrdg
 uD7SiJovUgnApdAWsM70AJwueHR3KVv9aPDOkN1AaKctXSvZTRaKhjwp9pl17ContPVuDDrfkw+jd
 Www4wqlPrD2mR8jBkFucHk0v6qtdtjPfucr5ZOCNhXU2JPkzOIyWQLvcKhROxUr72kjNx7MjUA5lU
 sCh0fxt4z4Vw3CMLPIchNGalmHrqk6sk57/t0t2EJHSHRVaA+Xnx5Hu9Te7sWHHrfWk+aTdDlfgwC
 SwfOtF4Q==;
Received: from [192.168.1.1] (helo=localhost.localdomain)
 by mail.fitzsim.org with esmtpsa (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2)
 (envelope-from <fitzsim@HIDDEN>)
 id 1pkCe3-000GKg-Nh; Wed, 05 Apr 2023 19:34:27 -0400
From: Thomas Fitzsimmons <fitzsim@HIDDEN>
To: Spencer Baugh <sbaugh@HIDDEN>
Subject: Re: bug#62598: 29.0.60; url-https-proxy-connect doesn't support
 multi-stage auth to proxies
In-Reply-To: <ierlejb2vpk.fsf@HIDDEN> (Spencer Baugh's message of
 "Sat, 01 Apr 2023 16:28:39 -0400")
References: <ierlejb2vpk.fsf@HIDDEN>
Date: Wed, 05 Apr 2023 19:34:21 -0400
Message-ID: <m3bkk1dhtu.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 62598
Cc: 62598 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Spencer,

Spencer Baugh <sbaugh@HIDDEN> writes:

> url-http knows how to use HTTPS proxies, primarily in
> url-https-proxy-connect.  It even knows to authenticate to those
> proxies, as fixed in bug#42422.
>
> But some HTTP authentication methods (e.g. NTLM as supported by
> url-http-ntlm) require multiple stages of back-and-forth in
> authentication.  This works fine with regular HTTP requests and requests
> to HTTP (non-S) proxies; it's handled by url-http-handle-authentication
> which is called by url-http-parse-headers when it sees a 401 or 407
> (auth required and proxy auth required) status.
>
> But this does not work with the HTTPS proxy support, because if it sees
> 401 or 407 as a response to CONNECT, it just immediately fails.

Why can't that code path call url-http-handle-authentication instead of
just failing?  What makes HTTPS different from HTTP in this respect?

> I'm very interested in adding this but I'm unsure how to approach it.  I
> guess that url-https-proxy-after-change-function should be calling
> something similar to url-http-handle-authentication.  Or maybe the whole
> design of how HTTPS proxy support works today is wrong, and it should be
> calling url-http-parse-headers like everything else?

I'd say try to make both approaches work, and see which one results in
the minimum set of changes.

Thomas




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#62598; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 1 Apr 2023 20:28:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Apr 01 16:28:45 2023
Received: from localhost ([127.0.0.1]:38650 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1pihq8-0008Pf-MO
	for submit <at> debbugs.gnu.org; Sat, 01 Apr 2023 16:28:44 -0400
Received: from lists.gnu.org ([209.51.188.17]:39976)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <sbaugh@HIDDEN>) id 1pihq6-0008PX-Qg
 for submit <at> debbugs.gnu.org; Sat, 01 Apr 2023 16:28:43 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <sbaugh@HIDDEN>)
 id 1pihq6-0004Rd-D8
 for bug-gnu-emacs@HIDDEN; Sat, 01 Apr 2023 16:28:42 -0400
Received: from mxout5.mail.janestreet.com ([64.215.233.18])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <sbaugh@HIDDEN>)
 id 1pihq4-0000bF-QL
 for bug-gnu-emacs@HIDDEN; Sat, 01 Apr 2023 16:28:41 -0400
From: Spencer Baugh <sbaugh@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 29.0.60; url-https-proxy-connect doesn't support multi-stage auth
 to proxies
Date: Sat, 01 Apr 2023 16:28:39 -0400
Message-ID: <ierlejb2vpk.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=64.215.233.18; envelope-from=sbaugh@HIDDEN;
 helo=mxout5.mail.janestreet.com
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)

url-http knows how to use HTTPS proxies, primarily in
url-https-proxy-connect.  It even knows to authenticate to those
proxies, as fixed in bug#42422.

But some HTTP authentication methods (e.g. NTLM as supported by
url-http-ntlm) require multiple stages of back-and-forth in
authentication.  This works fine with regular HTTP requests and requests
to HTTP (non-S) proxies; it's handled by url-http-handle-authentication
which is called by url-http-parse-headers when it sees a 401 or 407
(auth required and proxy auth required) status.

But this does not work with the HTTPS proxy support, because if it sees
401 or 407 as a response to CONNECT, it just immediately fails.

I'm very interested in adding this but I'm unsure how to approach it.  I
guess that url-https-proxy-after-change-function should be calling
something similar to url-http-handle-authentication.  Or maybe the whole
design of how HTTPS proxy support works today is wrong, and it should be
calling url-http-parse-headers like everything else?




Acknowledgement sent to Spencer Baugh <sbaugh@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#62598; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 11 Sep 2023 23:45:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.