Repository navigation
Conversation
Pin the behaviour of the current commons-httpclient 3 based implementation (folding, trimming, duplicates, termination, errors) before replacing it. Signed-off-by: Nico Piel <nico.piel@hotmail.de>
Replace HttpParser.parseHeaders with an equivalent inline parser so the userutil API no longer needs the EOL httpclient 3 jar. Line splitting, folding, trimming, UTF-8 decoding and the error message are unchanged; the parse error is now a java.net.ProtocolException (still an IOException) instead of the httpclient 3 ProtocolException. Signed-off-by: Nico Piel <nico.piel@hotmail.de>
Test Results130 files ± 0 130 suites ±0 3m 58s ⏱️ +47s Results for commit 90c8e09. ± Comparison against base commit 4589238. This pull request removes 2 and adds 23 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
mgaffigan
left a comment
There was a problem hiding this comment.
We should not be implementing an HTTP parser. This is how you get more request smuggling vulnerabilities.
Change to some existing parser we're already using. That may break some corner case where a client is sending nonstandard requests - but such things are not reasonably avoidable.
Signed-off-by: Nico Piel <nico.piel@hotmail.de>
|
Agreed. |
mgaffigan
left a comment
There was a problem hiding this comment.
Looks better - can we deprecate this method? Seems like it should not be used - doesn't handle repeated headers - and how is someone getting a set of http headers as a string?
tonygermano
left a comment
There was a problem hiding this comment.
Not approved yet. Switching to httpcore is the right call, and the characterization tests make the behaviour changes easy to follow. Three asks before merge:
- Encode the string as UTF-8. The new decoder rejects malformed input instead of replacing it, so on a JVM whose default charset isn't UTF-8 (e.g. Java 17 on Windows), any non-ASCII header value now throws
MalformedInputException. httpclient 3 returned a mangled value instead. With UTF-8 on both ends, the round trip is exact on every platform:buffer.bind(new ByteArrayInputStream(str.getBytes(StandardCharsets.UTF_8)));
testNonAsciiValuecan then drop itsassumeTrue, and the description's "platform charset behaviour unchanged" note should be updated. - Squash the inline-parser commit into the httpcore commit. We rebase-merge, so otherwise the hand-written parser lands on
mainwith a message that no longer matches what ships. SinceHTTPUtilis user-script API, please give the remaining commit a body listing the behaviour changes, for example:Keeping the characterization tests as their own first commit is good.Parse HTTPUtil headers with httpcore instead of httpclient 3 Delegate HTTPUtil.parseHeaders to httpcore 4.4's AbstractMessageParser.parseHeaders, which is already on the server classpath, so the userutil API no longer needs the EOL commons-httpclient 3 jar. Well-formed headers parse the same. Corner cases that change: - A malformed header throws org.apache.http.ProtocolException ("Invalid header: ..."), which is not an IOException, instead of httpclient 3's ProtocolException ("Unable to parse header"). - Empty header names, a folded line before the first header and bare-CR lines are rejected. - A whitespace-only line is treated as a continuation instead of ending the header block. - Folding onto an empty value no longer adds a leading space. - Only spaces and tabs are trimmed from values, not other control characters. - The string is encoded as UTF-8 instead of the platform charset. - In
HTTPUtilTest, use JUnit 4.13'sassertThrowsinassertParseFailureinstead of try/fail/catch. If the wrong exception is thrown, it reports the expected and actual types and keeps the original as the cause:This replaces theprivate static void assertParseFailure(String input, String badLine) { ProtocolException e = assertThrows(ProtocolException.class, () -> HTTPUtil.parseHeaders(input)); assertEquals("Invalid header: " + badLine, e.getMessage()); }
assertTrueandfailimports withimport static org.junit.Assert.assertThrows;.
Maintainer follow-up
Mitch's question about deprecating parseHeaders deserves its own issue.
Claude checked:
- The PR merges cleanly onto current
main, andHTTPUtilTestpasses (21 tests), both as-is and with theassertThrowshelper from ask 3. - With
-Dfile.encoding=ISO-8859-1and the headerX-Name: Müller, httpclient 3.0.1 returnsM?llerand httpcore 4.4.13 throwsMalformedInputException. With UTF-8 as the default, both returnMüller.
🤖 Drafted with Claude Code
What
HTTPUtil.parseHeadersno longer uses commons-httpclient 3'sHttpParser. It now delegates to httpcore 4.4'sAbstractMessageParser.parseHeaders(httpcore is already on the server classpath), instead of using a custom parser.Why
This is one step toward dropping commons-httpclient 3.0.1. Following review, it uses an existing, maintained parser rather than hand-written parsing code.
Behaviour changes against httpclient 3
HTTPUtilis user-script API, so these are worth a line in the release notes. Well-formed headers (Name: valuelines separated by CRLF or LF, ending at a blank line or EOF) parse the same as before. These corner cases change:org.apache.commons.httpclient.ProtocolException(anIOException),Unable to parse header: <line>org.apache.http.ProtocolException(not anIOException),Invalid header: <line>:v)""orphan)\t)\r\r\n)Invalid header: [0x10])X:thenb)" b""b"a,b,\tc)"a b c""a b c"(only leading whitespace of each fold is dropped)\u000ba\u0001)String.trim)Reviewer notes
str.getBytes()still uses the platform charset and the bytes are decoded as UTF-8, as before.-1), matching the old unlimited behaviour.🤖 Generated with Claude Code