Fix stale Content-Encoding header in ProxyServlet after HttpClient bump

[labkey-tchad]

Rationale

The bug, in ProxyServlet:

  proxyResponse = proxyClient.execute(getTargetHost(servletRequest), proxyRequest);
  copyResponseHeaders(proxyResponse, servletRequest, servletResponse);   // copies Content-Encoding verbatim
  copyResponseEntity(proxyResponse, servletResponse, proxyRequest, servletRequest);  // streams whatever entity HttpClient gave it
  copyResponseHeader only skips hop-by-hop headers (Connection, Keep-Alive, Transfer-Encoding, etc.) — Content-Encoding and Content-Length are copied through unconditionally.

Why 5.6.4 breaks this: Apache HttpClient5's internal ContentCompressionExec auto-decompresses gzip/deflate response bodies transparently. In 5.5.2 it also scrubbed the now-stale headers off the raw response:

  response.setEntity(new DecompressingEntity(response.getEntity(), decoderFactory));
  response.removeHeaders(HttpHeaders.CONTENT_LENGTH);
  response.removeHeaders(HttpHeaders.CONTENT_ENCODING);   // <- removed
  response.removeHeaders(HttpHeaders.CONTENT_MD5);

In 5.6.4 (part of a rewrite adding pluggable codecs — Brotli/Zstd/etc. — via a new ContentCodecRegistry), that cleanup was dropped:

  response.setEntity(decoder.apply(response.getEntity()));
  // no more removeHeaders() calls

The entity is still silently decompressed, but Content-Encoding: gzip (and the old compressed Content-Length) now survive on the response object and get forwarded verbatim by ProxyServlet.

Net effect: browser receives Content-Encoding: gzip header + already-decompressed plaintext body → Firefox refuses to render it → exactly the contentEncodingError you're seeing.

Related Pull Requests

• Update Apache HttpClient server#1476 (httpclient5 5.5.2 -> 5.6.4 bump that exposed this)

Changes

• Disable HttpClient's automatic content-decompression on the shared proxyClient in ProxyServlet.createHttpClient(), so ProxyServlet always streams the backend's original bytes and headers through unmodified.

[labkey-tchad]

Verifying with RStudio on TeamCity: https://teamcity.labkey.org/buildConfiguration/LabKey_267Release_Premium_ModulesSuites_RStudioPostgres/4134256
Not sure how this will interact with the doHandleCompression flag added for AWS.

[XingY]

It seems our ProxyServlet diverged from the original implementation:
https://github.com/mitre/HTTP-Proxy-Servlet/blame/9a522c5c021e6941cd951941ed49d96ff996f1eb/src/main/java/org/mitre/dsmiley/httpproxy/ProxyServlet.java#L286

Based on the original code, if should check doHandleCompression to set the config.

[XingY]

Can this be added in LabKeyProxyServlet as override instead? We try to keep ProxyServlet consistent with the original library to make the next sync easier.

[labkey-tchad]

On, sure, that makes sense.
I suspect something similar will be added to the upstream if/when they update HttpClient. They haven't really made any significant changes since 2023.

[labkey-tchad]

Verifying on TeamCity with latest changes: https://teamcity.labkey.org/buildConfiguration/LabKey_267Release_Premium_ModulesSuites_RStudioPostgres/4137530