HTTP Digest authentication

[wom-bat]

Hi folks,
How can I create a websocket client where the server requires digest authentication?

[aaugustin]

If I remember correctly Digest Authentication requires two HTTP requests: one to get the challenge from the server, one to send the response and open the websocket connection.

If I had to do this, here's what I'd try:

1. make the first HTTP request and get the challenge from the response; this will likely easier to do with another library than websockets, as websockets.connect raises an exception if it fails to establish a WebSocket connection;
2. calculate the proper authentication header and send it with the extra_headers argument of websockets.connect.

It would be interesting to build this into websockets. Since we already handle basic auth and http redirects, I think we have all the pieces we need.

[wom-bat]

It's a little more complex than that. The Authorization: header to be sent changes on each request, so it can't be calculated just once. What's more the server can issue a new challenge at any time.

I don't know how that fits into websockets. It may be that we can do it just once and then after the upgrade there's no need. I'll do some experiments on Monday.

[aaugustin]

In a WebSocket connection you send HTTP headers only once. After you've upgraded from a HTTP to a HTTP connection you're fine.

[apteronal]

Two options have been discussed for adding digest authentication support:

1. Using a separate library for the digest authentication.
2. Implementing digest authentication within this library.

Option 1 is disfavored by the maintainer because it adds a dependency to websockets which currently has none. @Nicolas-Feude has done some great work in #1111 toward option 2. However, looking at the code in #1111, it would seem to add a lot of complexity to this package, and duplicates code from requests which may diverge over time. It also seems fraught to re-implement security-related code.

I'm wondering if another option would be to use a library for the digest authentication but as an optional dependency. When digest authentication is required, websockets could output a message saying that the dependency needs to be installed. Digest authentication is an uncommon need, so it may be reasonable for the user to have to install an extra package to support it. (Not an experienced Python developer so unsure whether this is a good idea, but wanted to suggest it just in case.)

[aaugustin]

Unfortunately #1111 isn't on the right track for the following reasons so I'm closing it.

First, it targets the legacy implementation. However, I'm not adding new features to that implementation. The feature should target the new asyncio and threading implementation.

Second, the fundamental design problem lies in how we handle HTTP sequences prior to the WebSocket handshake. Currently, websockets supports only one request and one response while Digest Authentication requires two.

That sequence is handled in each I/O layer because validating credentials may require I/O e.g. if you want to check them in a database. This makes it more expensive to implement and to test than if it was in the Sans-I/O layer.

I'm wondering if there's a way to redesign, at least on the client side, to move authentication logic to the Sans-I/O layer? On one hand, I'm very unsure that it can work; on the other hand, it could bring the cost of building and maintaining that feature down to a level where I'm willing to take it.

Overall, I'm not planning to make websockets a full-featured HTTP client, yet HTTP features are creeping on the client side e.g. supporting proxies, redirects, etc. I resist taking a dependency on a full-featured HTTP client because it could create more complexity that I'm willing to deal with.

---

The above is heavily influenced by the fact that I'm a solo maintainer and this is a hobby project. While I believe that it's a high quality hobby project, this quality depends on keeping the maintenance workload load very low.

[aaugustin]

While #788 also targets the legacy implementation — which was the only implementation until the the Sans-I/O implementation landed one year later — there's a couple interesting ideas in there:

• It takes a dependency on a small library to handle the HTTP Digest Authentication protocol. While the protocol isn't very complicated and maintaining our own implementation with 100% branch coverage would be doable, leveraging a dependency reduces the effort. That particular dependency appears to be unmaintained, though, so I wouldn't take that one.
• It's implemented next to the redirect logic. I was considering piggybacking on the redirect implementation because it's the component that knows how to make multiple HTTP requests. That being said, right now, websockets closes the TCP connection between each HTTP request; that would require modifying.

[aaugustin]

Since the authentication is embedded in the WebSocket handshake, we cannot replicate the design used for proxies — connecting a socket then handing it over to websockets.