Architecture Design

Goal

The goal is to develop an architecture and with it a protocol to allow multiple users to collaborate on a CRDT based data structure in an end-to-end encrypted way.

Requirements

Actors

• User represents a person interacting with the content.
• Client represents the actual instance connecting to the service. A user can have one or multiple clients at the same time.
• Service represents the server responsible to receive, persist and deliver information to the clients.

Business Requirements

• The content must be end-to-end encrypted.
• The same user must be able to interact on the same document with multiple clients.
• Clients must not see each others IP addresses.
• When activated it must be possible to identify who wrote which content.
• The user must be able to start or stop sending and/or receiving updates and be able to send updates batched later.

System level Requirements

Data exchange

• Must support asynchronous exchange of data. This means participants don't have to be online at the same time, but still can exchange data.
• Must support real-time exchange incl. awareness features e.g. cursor position.
• The architecture must support clients that have to rebuild the CRDT based data structure from ground up.
• The architecture must support local-first clients. These clients can be offline for a while and only sync later once they are connected again.
• The architecture must support multiple CRDT implementations. In detail this means Secsync is a layer on top of a data type, where the operations are commutative. In particular Yjs and automerge should be supported.
• The architecture can, but must not be decentralized. Leveraging a centralized service is a viable option.

Security

• The content of a document must only be accessible to the participants.
• There are no limitations on meta data e.g. who created how many changes.

Authorization

The architecture should support two main use-cases:

• Every client is verifiable through a private-public keypair. The keypairs could come from any kind of Public-Key Infrastructure or Web of Trust system. The scenario here is close groups where the public keys are verified.
• Every client with access to the document ID can retrieve data and only with the shared secret can decrypt it e.g. www.example.com/doc/{id}#{pake of the shared key} would allow multiple anonymous participants to collaborate.

Eventual consistency

All clients receive the same set of content updates (possibly in different orders), and all clients converge to the same view of the document state as they receive the same set of control content updates.

Design Decisions

EphemeralMessages

The session ID + session counter are stored in the encrypted data.

The benefit of this design is that the id as well as the counter are not exposed to any MITM (man-in-the-middle) attacker nor the server.

To make this design work it's important that the sessionId is stored per client and not in one denormalized store per sessionId. Otherwise one client could increase the counter of another making their session basically invalid.

Process

• initialize -> proof and ask for proof
• validate proof -> respond with a proof
• validate proof
• message

Handling of missing messages and retry strategies

The snapshots are stored in order - old snapshots can be cleaned out (or at least the updates)

Usecases

• missing a update (e.g. update comes in not in order) -> the solution: have an incoming queue -> challenge: when to abort it
• getting a snapshot that doesn't apply to a known one -> precondition: what if it is an old snapshot? -> the solution: reconnect and ask the server for a new version of the document

Incoming queue

{
  [snaphotId]: {
    lastUpdatesPerAuthor: {
      [authorId]: {
        lastUpdate,
        updatesReceived
      }
    },
  }
}