GNU bug report logs - #8931
24.0.50; url-http.el terminating HTTP packets with extra carriage return and newline

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 8931 in the body.
You can then email your comments to 8931 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Avrutin <nicolasavru <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.0.50; url-http.el terminating HTTP packets with extra carriage
	return and newline
Date: Fri, 24 Jun 2011 15:45:36 -0400

[Message part 1 (text/plain, inline)]

`browse-url-emacs' and `url-retrieve-synchronously' (and probably other
functions) appear to be generating malformed HTTP packets by appending an
extra \r\n to the packet, resulting in them being terminated with "\r\n\r\n"
instead of "\r\n". Some pages (such as http://www.google.com) to still
return properly and just send back a packet saying that the sent packet was
malformed. Other pages, in my case last.fm's client API (access via
emms-lastfm-client) return 400 Bad Request. In addition, this causes
Wireshark to report the packet as "HEAD / HTTP/1.1 Continuation or non-HTTP
traffic".

Removing the "\r\n" on line 341 of url-http.el (see attached patch) removes
the duplicate "'\r\n" and appears to fix the issue. Calls to last.fm's
client API return 200 OK as expected and Wireshark properly reports the
packet as "HEAD / HTTP/1.1". Google, however, still reports a malformed
packet.

This bug is preset in Emacs 23.1, but is not in 23.2. My patch restores the
line to what it was in 23.2, but I do not know if it was changed for a
reason or in error.

--
Nicolas Avrutin

[Message part 2 (text/html, inline)]

[url-http.patch (text/x-patch, attachment)]

Message #12 received at 8931 <at> debbugs.gnu.org (full text, mbox):

From: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
To: Nicolas Avrutin <nicolasavru <at> gmail.com>
Cc: 8931 <at> debbugs.gnu.org
Subject: Re: 24.0.50;
	url-http.el terminating HTTP packets with extra carriage return and
	newline
Date: Sun, 03 Jul 2011 18:04:44 +0200

Nicolas Avrutin <nicolasavru <at> gmail.com> writes:

> This bug is preset in Emacs 23.1, but is not in 23.2. My patch
> restores the line to what it was in 23.2, but I do not know if it was
> changed for a reason or in error.

Thanks for the patch; I've now applied it to Emacs 24.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Message #19 received at 8931 <at> debbugs.gnu.org (full text, mbox):

From: "Christopher J. White" <chris <at> grierwhite.com>
To: 8931 <at> debbugs.gnu.org
Subject: url-http-create-request needs extra CRLF for https
Date: Sat, 24 Sep 2011 00:04:43 -0400

There is a tug of war going on here.  

Prior to 100681, the code had no CRLF after url-http-data.  Mark
Hershberger submitted a patch to add the CRLF because without it, POSTs
with content to https urls just hang:

  http://bzr.savannah.gnu.org/lh/emacs/trunk/revision/100681

This change was undone by Nicola Avrutin in 104908:

  http://bzr.savannah.gnu.org/lh/emacs/trunk/revision/104908

The problem was that with no url-http-data, the request ends with a
double CRLF, which apparently confuses some servers.

I recently came across the exact same problem (with url-http.el
without the extra CRL), and Mark's patch above works just fine for me
(also POSTs to https).

I'm thinking the correct solution is to only add the CRLF if there is
http-data:

-    url-http-data))
+    url-http-data (if url-http-data "\r\n")))

...cj

 

Message #22 received at 8931 <at> debbugs.gnu.org (full text, mbox):

From: "Christopher J. White" <emacs <at> grierwhite.com>
To: 8931 <at> debbugs.gnu.org
Subject: May be related to 9592
Date: Sat, 24 Sep 2011 14:30:12 -0400

Further testing with this indicates this may not be a complete fix.
The fix works reliably for a single connection, but when making
multiple POST requests, an older process is grabbed for reuse, but then
leads to an error as the process seems to terminate. 

See:  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9592

Message #25 received at 8931 <at> debbugs.gnu.org (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: "Christopher J. White" <chris <at> grierwhite.com>
Cc: 8931 <at> debbugs.gnu.org
Subject: Re: bug#8931: url-http-create-request needs extra CRLF for https
Date: Sat, 24 Sep 2011 18:58:53 -0400

"Christopher J. White" <chris <at> grierwhite.com> writes:

> I'm thinking the correct solution is to only add the CRLF if there is
> http-data:
>
> -    url-http-data))
> +    url-http-data (if url-http-data "\r\n")))

Yes, this looks right.  Committed to trunk.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.