Authors

Torben Weis, University Duisburg-Essen
Joseph Gentle
Tad Glines
Chris Harvey, iotaWave.org
John Edstrom
Charles Goddard
Michael Rollins
Ian Roughley, Novell
Michael MacFadden, SOLUTE Consulting

Purpose and Motivation

• Keep it simple. We want developers to build their own wave servers and make them federate easily
• Keep it efficient. The HTTP-based protocol should not be slower than the XMPP version. In an ideal case it will even be faster
• Keep it secure: Provide the same (or higher) level of security as with the XMPP-based federation

Scope

This document shows how the logical message flow of wave federation can be implemented using an HTTPS-based transport. Furthermore, this document specifies how to do discovery, i.e. given a wave user ID (e.g. joe@example.com) how can we find the corresponding wave server for this user.

Discovery

When a wave server wants to send a message to a user it must find the corresponding wave server. For example a hosting-server encounters that user joe@example.com has been added to a wave. It needs to know how to talk to this user's wave server. The discovery protocol consists of three parts.

1. Determine or guess the host-name and port of the wave server
2. Try to fetch a capabilities document from this server using HTTPS. For example we suspect foo.com at port 443 to provide wave services. Using HTTPS we ask for the document https://foo.com:443/wave/fed/capabilities. This should return a document with a format as outlined below
3. Inspect the capabilities file. It describes, for example, which version of the wave federation protocol this user supports. Optionally, the wave server may have special security requirements or policies which are described in the capabilities file.

The protocol for this contains 3 options which are tried in the following order:

Transport

HTTP-based federation is REST-full. This implies that given the URL and the HTTP method, a server knows already what the purpose of the HTTP request is. Furthermore, the protocol is state-less, i.e. each HTTP call is an independent and atomic operation. All federation-specific URL pathes must be prefixed by /wave/fed. This simplifies server development since it provides an easy way to split wave requests from normal web server requests just by inspecting the URL.

Security

Messages

The following message types can be sent:
- POST Submit
- PUT WaveletUpdate
- GET GetSigner
- GET HistoryRequest

The semantics and contents of these messages is essentially the same as in the current federation protocol. Only PostSigner has been merged with the Submit message. Data that has formerly been encoded as XML does now become ProtoBuf data. XMPP-specific control message are gone for good.

The ProtoBuf message types used below are either part of the current Google Wave ProtoBuf definitions [TODO] or they are defined in this document. In the following the patterm <wavelet-id> corresponds to the wavelet name as specified in section 4 of [2].

A Submit is sent from a remote wave-server to a hosting wave-server with the intent to modify a wavelet. The remote-server is sending a wavelet operation and expects a response that tells whether the operation has been applied successfully or not.

The request payload is a ProtocolSubmitRequest which contains the signature and a ProtocolWaveletDelta (which is encoded as byte stream). The signature signs exactly this byte stream. So far, we could even tolerate a non-canonical serialization as long as this byte stream is persisted. In addition the message must be able to carry a ProtocolSignerInfo. This replaces the old PostSigner message. We integrate it as an option into Submit. This saves us one extra message and is more rest-ful, i.e. state-less.

message ProtocolSubmitRequest {
    required bytes delta = 1;
    repeated ProtocolSignature signature = 2;
    optional ProtocolSignerInfo signer = 3;
}

The response payload is a message of type

message ProtocolSubmitResponse {
    required int32 operations_applied = 1;
    optional string error_message = 2;
    optional protocol.ProtocolHashedVersion hashed_version_after_application = 3;

    optional int64 application_timestamp = 4;
}

Message: WaveletUpdate

A WaveletUpdate is sent by a hosting wave-server to a remote wave-server to inform him that a wavelet has been modified. The payload of the message is a set of wavelet operations

The payload is a ProtocolWaveletUpdate

message ProtocolWaveletUpdate {
    required string wavelet_name = 1;
    repeated ProtocolAppliedWaveletDelta deltas = 2;
    // Tells up to which version the deltas have been committed to persistent storage
    optional int64 commit_notice = 3;
}

Message: GetSigner

If a remote server receives a wavelet update which contains an unknown signer id, the remote server can ask the hosting server for the certificates corresponding to this signer id.
The signer-id is a hash. For the purpose of this protocol, the hash is base64 encoded.


GET http://hosting-server/wave/fed/signer/<signer-id>

The response payload is a message of type ProtocolSignerInfo, which contains the full certificate change corresponding to the signer-id


Message: GetHistory

A remote-server can ask the hosting server for the history of a wavelet. It has to specify the first delta v1 (inclusive) and the last delta v2 (exclusive) it wants to obtain. In addition the hashes of the respective wavelet versions are required. These hashes are SECURITY RELEVANT. If all users of a remote server have been kicked out of a wavelet, this remote server can still ask for the history up to the point where his last user has been kicked out. This is because this remote server knows the hashes of the wavelet up to this time.It does not know hashes of the wavelet after his last user has been kicked out.

- v1: An int64 encoded as a decimal number. Deltas starting at this version are to be included in the reply

- v2: An int64 encoded as a decimal number. The reply will only include deltas up to this version.

- v1hash: A base64 encoded hash value, corresponding to version v1 of the wavelet

- v2hash: A base64 encoded hash value, corresponding to version v2 of the wavelet
- limit: The maximum number of bytes the caller is willing to accept in the response.

The response payload is a message of type ProtocolWaveletHistory

message ProtocolWaveletHistory {
     repeated bytes deltas = 1;
     optional int64 truncated = 2;
     optional int64 commit_notice = 3;
}

This section of the specification discusses how servers can recover from failed communication attempts. Two cases have to be distinguished


- The calling server fails to establish a connection or send the complete request
- The calling server sent the complete request but it did not receive any response from the other wave server.


If GetSigner or GetHistoryRequest fail, nothing bad can happen, since they are idempotent. The calling server SHOULD try again after an interval of its choice.


If a remote-server submits a delta but does not get any response, it remains unclear whether the delta has been applied or not. The remote-server SHOULD re-submit its delta as soon as possible. The hosting-server MUST detect that this deltas has already been applied by comparing it to its accepted deltas. Thus, while the delta undergoes OT-transformation, the server must check if it has already accepted this delta. If the deltas have already been accepted, the hosting-server MUST send a valid submit response, but it MUST NOT send an additional wavelet update message. Thus, the hosting-server MUST make submission idempotent.


If a wavelet update could not be sent to a remote-server or if the hosting-server is not sure that the remote-server received the wavelet update, then the hosting-server SHOULD try again to send the wavelet update in an interval of its choice. If multiple wavelet update messages are pending for some remote-server, then hosting-server SHOULD only retry to send the latest of these messages. The remote-server can later use GetHistoryRequest to obtain the other deltas it has missed out. This reduces the queue size of pending wavelet update messages on the hosting-server.


If a server crashes and comes back up, it SHOULD ask all hosting-servers it has federated with for their capabilities file. This way these hosting-servers realize that the remote-server is back up. The hosting-server SHOULD then attempt to send a wavelet update to the remote-server. This wavelet-update consists only of the latest delta and the latest commit-notice. The remote-server can use GetHistoryRequest to obtain the other deltas it has missed out.

References