Unable to CONNECT through Broadcom proxy

[jms135]

Describe the issue

Our Distributed Media Engine (DME) is a CentOS Linux system that use the AWS C++ SDK to push files to S3. This feature is running successfully at many customers but several have experienced a problem where the DME is unable to make a CONNECT to S3 through this particular Blue Coat/Broadcom proxy: https://www.edgeblue.com/DataSheets/ds_sg_proxy_v1-2.pdf

Proxy logs, Broadcom support, and customer proxy experts indicate that the CONNECT is being rejected by the proxy because it has invalid headers, and they are specifically identifying “content-length” as the problem.

The following paragraph in RFC 7231 https://datatracker.ietf.org/doc/html/rfc7231#page-31 advises against sending a payload body on a CONNECT:

“A payload within a CONNECT request message has no defined semantics;
sending a payload body on a CONNECT request might cause some existing
implementations to reject the request.”

Attached is a packet trace screenshot showing a sample CONNECT from DME using AWS SDK version 1.9.132. The DME use case for the SDK is pushing files to S3 and apparently the SDK is including the first file payload with the CONNECT, which triggers the use of the content-length header, which is causing the block from the proxy.

Not sure whether to call it a feature request or a bug or maybe there is a different way for us to use the SDK -- can you tell us how to avoid having the SDK include a content-length header on a proxy CONNECT?

Note that other services in the same DME system are using curl directly (not via the AWS SDK) and those work fine with the same proxy, it's only the CONNECT from the SDK that has content-length and is blocked.

Steps to Reproduce

Full source file attached but to summarize: it calls SetAwsClient to configure and then repeatedly calls ProcessMsg to post files to S3.

AwsS3Push.txt

Current behavior

Attached jpg is screens shot from a packet trace showing the headers on the proxy CONNECT as it works today.
We need to have the CONNECT not include content-length.

AWS CPP SDK version used

1.9.132

compiler and version used

clang 5.0.1

Operating System and version

CentOS 7

[vudh1]

Hi @jms135 thanks for reaching out. This is a feature that our Go team decides to work on, will update more when there is a timeline for this.

[jms135]

Thanks @vudh1 we are looking forward to your update and please let me know if you need additional information.

[jmklix]

Looking into using this curl feature: https://curl.se/libcurl/c/CURLOPT_HTTPHEADER.html

[jms135]

@jmklix thanks for the update.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[jms135]

Sorry, that was an accidental close, we are still very interested in this enhancement.