WebSockets support

[ibaryshnikov]

Is there a way to use websockets in tide?

[aturon]

Not yet! This would require exposing a bit more from hyper, in particular http upgrades. Definitely worth taking on!

[aturon]

I'm going to mark this as a "Design" question right now, because it'd be good to talk through the API for exposing http upgrades within Tide.

[secretfader]

I recently found tokio-tungstenite, a project which provides tokio bindings for Tungstenite, a WebSocket library. I haven't personally reviewed the code, but it might be worth a look (at least for inspiration).

[najamelan]

Tungstenite works on a socket which implements io::Read + io::Write (tungstenite), or AsyncRead + AsyncWrite (tokio-tungstenite).

Here you can have a stream (like a TCP) and tungstenite will handle the websocket handshake for you.

When looking at the examples of hyper, it looks like hyper handles the handshake for you. This makes sense, as it's also listening for other http requests that might not be upgrade requests. On the other hand, it might be cleaner to leave every library their own speciality. I don't know how feasible it would be for hyper to see that it's an upgrade request, leave the data in the stream and expose that, but that's not how it works right now anyway.

In any case this is supported by tungstenite and tokio-tungstenite by the from_raw_socket method, which assumes the handshake has already taken place. It's not possible I think with these libraries to use custom subprotocols and websocket extensions I think (it doesn't look like hyper supports them either), and since these are negotiated in the handshake, a method like from_raw_socket would have to take information about subprotocols, which it doesn't.

Currently tokio-tungstenite still operates on futures 0.1 types.

So in brief, to enable (tokio-)tungstenite, it suffices to expose a socket which implements io::Read + io::Write (tungstenite), or AsyncRead + AsyncWrite (tokio-tungstenite). When using hyper under the hood, it's a matter of exposing the Upgraded object from hyper.

I have not looked into other websocket crates to see what would be needed in order to support them.

[najamelan]

ps: warp has all the code for exposing a Stream+Sink of tungstenite::Message on upgrade. This can inspire implementors.

[najamelan]

Ok, I've given the whole thing some more thought. Here is what I think:

Currently webservers like hyper handle http(s). Tungstenite also does this, as you can let it listen on a type that provides io::Read + io::Write. Unfortunately hyper only allows listening for http connections on a TCP (AFAICT). So tungstenite allows you to listen for websocket connections on any type of connection, could be UDS, named pipes, mem mapped file etc, where hyper only supports TCP.

The downside is that tungstenite also has to deal with HTTP and TLS (I suppose that when you upgrade an HTTPS connection, the websocket logic/crate no longer needs to worry about the encryption). This is redundant and creates extra work and security risks.

If hyper would be more generic, and support any AsyncRead/AsyncWrite rather than only TCP, crates like tungstenite could rely on it for the HTTP upgrade part, eliminating a bunch of code.

Both sides could focus on implementing support for subprotocols and websocket extensions to be feature complete and on converting to futures 0.3.

Now frameworks like warp and tide can expose an object that implements Sink + Stream of tungstenite::Message and holds information about subprotocols and extensions. Ideally, the type of the object returned by both warp and tide (and possibly other frameworks) would be the same and only implemented once in a separate crate. It would be a type that takes in an upgraded http connection + subprotocol and extension metatdata and that provides the Sink+Stream.

This way, there is no double work/maintenance. Every layer is clearly defined. This new crate could supersede tokio-tungstenite and provide futures 0.3 types. If hyper is to high level for dealing with websocket handshakes over things other than TCP, maybe this can also be separated out into a crate that uses http and http-parse to do so and both hyper and tungstenite can reuse this logic?

Personally I'm working on providing AsyncRead/AsyncWrite on top of such a Sink/Stream of websocket messages so that wasm code and servers can conveniently communicate rust objects on a websocket connection framed with tokio-codec/futures-codec. This logic could potentially go in the same crate as the Sink/Stream.

@seanmonstar @agalakhov What do you think of this?

[agalakhov]

@najamelan Such a layered system was exactly that I wanted when I coded Tungstenite. I wanted to have a single, well-tested system that can work for both async (probably Tokio-based) and purely synchronous (maybe even microcontroller-based) projects.

[najamelan]

So to summarize how this could be:

• one handshake crate: takes a io::Read + io::Write and/or AsyncRead + AsyncWrite which are expected to receive a http upgrade request and metadata (such as which subprotocols the server accepts). It would returns a Result to a raw socket on which the handshake has been completed + metadata. This should probably also hold client side code for initiating the handshake.
• one codec crate that frames a raw socket and provides a Sink + Stream of websocket Messages and a synchronous equivalent.
• low level crates like hyper and (tokio-)tungstenite, as well as frameworks can both use these two crates, possibly deciding to handle Ping, Pong messages for the user. They can expose the Sink+Stream to end users.
• layers on top of this like proving AsyncRead + AsyncWrite which can work regardless of which web framework the user wants to use, and which can work on connections that aren't TCP thanks to a unified low level API.

[seanmonstar]

Hyper's server works with any AsyncRead + AsyncWrite. It just uses TCP if you use the convenient Server::bind constructor.

The warp framework built on top of hyper uses it's built-in upgrade support, and after a WS handshake, uses tungstenite afterwards.

[najamelan]

@seanmonstar Oh, sorry. I had a look a the impl of Server. That solves that than.