Search this site

Recent site activity
Protocol‎ > ‎Design Documents‎ > ‎

Client/server protocol

    Objective

    This design proposes a client/server protocol for Wave. The protocol allows rich clients to interact with wave in real-time, as opposed to the less live Wave Data API.

    Background

    The Google Wave service implements a client/server protocol based on a live view of a wave. The protocol mixes snapshot state with a stream of deltas over a "wave view" channel. This protocol proved to have some negative impacts on scalability and was never fully implemented in FedOne. The design presented here is both simpler and more robust.

    The existing protocol is partially documented here.

    Requirements

    Wave clients need to:
    • Discover what wavelets are accessible to them in a wave
    • Fetch up-to-date snapshots of wavelets
    • Stream live deltas for a wavelet, usually from a particular known version
    • Submit deltas to wavelets

    Some typical processes dictate a particular sequence of these services but clients should be free to interact with different sequences.

    The protocol as described is transport-agnostic. The typical implementation will be over an authenticated websocket connection. Authentication of the transport is beyond the scope of this design (see [link]).

    Design

    Overview

    The protocol is presented as a set of services. Clients may invoke these services at will - the protocol doesn't define a "correct" sequence, though particular sequences make sense for particular uses.

    To open a wave a client first fetches a snapshot of its wave view, which it may immediately render. It can then open individual delta channels for each wavelet in the view in order to receive live updates, and submit deltas.

    Transport

    The protocol is described as abstract requests and responses. A typical server (including Wave in a Box) will offer the protocol over websocket but parts are also available from servlets. The messages below are expressed as protocol buffers. On the wire these messages will be translated to JSON or binary protocol buffer as applicable.

    Fetch service

    The fetch service provides snapshots describing a client's current view of a wave. Purely as a bandwidth optimisation, the client may specify that it already has snapshots of some wavelets at some versions (e.g., from a previous fetch). If the server's current version matches the version the client provides then the snapshot is omitted from the response. The version hash is included to ensure that the client's version matches that committed to disk by the server (it's possible for a client to observe a version that "disappears" from history when a server crashes, but this is not a useful "known" snapshot).

    WaveletVersion {
        required WaveletId id,
        required ProtocolHashedVersion version,
    }

    FetchWaveViewRequest {
        required WaveId id;
        repeated WaveletVersion knownWavelet;
    }

    FetchWaveViewResponse {
        required int32 responseCode;
        repeated {
            required WaveletId id;
            optional WaveletSnapshot snapshot; // omitted if "known"
        }
    }

The response contains an entry for every wavelet to which the client currently has access (if a user is removed from a wavelet, that wavelet is no longer returned by the server). Each entry in the repsonse contains the current state of the corresponding wavelet (unless the client provided a hashed version that matches the wavelet's current state, in which case the snapshot is omitted).

An analogue of the fetch service is provided by the fetch servlet. This presents the same service, but with a different way of asking for it for web-based clients which speak HTTP. The servlet allows browser-side caching of results, HTTP compression, etc.

This service may be extended in the future. Specific improvements might include:
  • A streaming response allowing new wavelets to appear in the view
  • Returning historical snapshots when a user has been removed from a wavelet
  • Client provision of a filter over which wavelets to consider
  • Wavelet snapshot fetch at specific versions

Delta channel service

The delta channel service provides a uni-directional stream of deltas for a single wavelet, beginning at the delta applying at a client-specified version.

OpenWaveletChannelRequest {
    required WaveId waveid;
    required WaveletId waveletId;
    required ProtocolHashedVersion beginVersion;
}
CloseWaveletChannelRequest {
    required String channelId;
}

WaveletUpdate {
    required ProtocolWaveletDelta delta;
    required ProtocolHashedVersion resultingVersion;
    required int64 applicationTimpstamp;
}

// Repeated message
OpenWaveletChannelStream {
    // Channel id is provided only in the first message
    optional String channelId;

    // Second and subsequent messages contain either or both a delta and commitVersion
    optional WaveletUpdate delta;
    optional ProtocolHashedVersion commitVersion;

    // Last message contains only a terminator
    optional OpenWaveletChannelTerminator terminator;
}

// Terminating message
OpenWaveletChannelTerminator {
    required int32 responseCode;
    optional String errorMessage;
}

The first response contains only a delta channel id (to be attached to future delta submissions). Subquent responses contain contiguous deltas and interleaved or piggybacked with monotonic commit notifications.

As a bandwidth optimisation the server may filter a clients own deltas out of the delta stream. A client learns the version at which its delta applied in the submit response (see below) and can transform it appropriately. There is no guarantee of ordering between wavelet stream messages and delta submission responses.

When a client loses access to a wavelet the channel for that wavelet is terminated with a distinctive "lost access" response code and no further messages are received.

Future improvements might include:
  • Adding an end version so this can serve closed-range history requests

Delta submission service

Deltas are submitted to the application (not on the delta stream). The required channel id parameter implies that a delta channel must be opened and have provided the id before a submit may be attempted.

SubmitDeltaRequest {
    required WaveId waveId;
    required WaveletId waveletId;
    required ProtocolWaveletDelta delta;
    required String channelId;
}

SubmitDeltaResponse {
    // Both required if submit was successful
    optional ProtocolHashedVersion hashedVersionAfterApplication;
    optional long timestampAfterApplication;


    required int32 errorCode;
    optional String errorMessage;
}


Mechanisms
These services expose a number of mechanisms

Wavelet creation

A wavelet is created by submitting a delta to it. The metadata of that first delta provides the immutable wavelet metadata of creator and creation timestamp. A client can display the created wave optimistically, but must guess at the creation timestamp until the submit response is received.

The server may impose restrictions on the form of the first delta to a wavelet, such as requiring that the delta add its author as a participant to enable future deltas to clear access control.

A typical client process to create a new wave will be:
  1. Generates a wave id
  2. Open a delta channel against the conversation root wavelet and receive a channel id
  3. Submits deltas to the conversation root wavelet (and possibly per-user wavelet) adding itself as a participant and establishing the conversation manifest

Participant removal

As currently described, when a client loses access to a wavelet (they are removed from the wavelet or from the group giving them access to the wavelet) the delta stream for that wavelet closes and the wavelet will no longer appear in fetch requests. Essentially this means the client completely loses access to the data, including old, already-seen versions. This is not the desired end-goal, but an intermediate step.

Future changes to this protocol should ensure that a client who had access to a wavelet, observed the wavelet, and subsequently loses access should retain access to all versions of the wavelet up until they were removed. This requires that the fetch service can calculate this version for each wavelet a client ever had access to and return that snapshot.

If the fetch service is extended to support a streaming view so that new wavelets may come into view as the client gains access (e.g. is added to a private reply) then a client may lose and regain access to a wavelet (many times) during one view session. When a client re-gains access to a wavelet it will re-fetch the latest snapshot and re-open a wavelet channel against that version. This mechanism is much more efficient than streaming a potentially large history of deltas down the delta channel.
Comments