A real-time collaborative rich text editor like Google Docs. Any number of tabs can edit the same document and see each other's changes, cursors and avatars. A tab survives a reload and a temporary disconnection without losing changes.
The application is ~2,6k lines of Go, 457 lines of templ and ~1k lines of TypeScript, not counting blank lines, comments, tests and generated code.
The documents directory stores 1 snapshot and 1 change log for each document.
flowchart LR
tabA["Tab A"] -->|"POST: change"| server["Server"]
tabB["Tab B"] -->|"POST: change"| server
server -->|"SSE: changes"| tabA
server -->|"SSE: changes"| tabB
server -->|"snapshot, log"| docs[("documents/")]
All editor JavaScript runs in the <docs-editor> Web Component, which wraps Quill.
Quill reports each edit as a Delta. The editor emits a docs-submit event. A Datastar data-on:docs-submit attribute turns the event into a POST request to POSTSubmit. The server puts changes from all tabs in 1 order. It sends each tab its missing changes over SSE with ExecuteScript. The script calls the editor's receive method. The editor merges the changes into its text, which may include changes that the server does not have yet. This design uses OT (operational transformation) with a central server.
The editor sends each change to POSTSubmit with the revision where it was made. The store transforms the change against all changes applied since that revision. It applies the result as the next revision and appends it to the document log. POSTSubmit then dispatches EventDocChanged with the document id. The broker delivers the event to the stream for every tab that has the document open. For each tab, OnDocChanged reads its missing changes from the store and passes them to the editor's receive method. The source tab also receives its change. This acknowledges the change.
sequenceDiagram
participant EA as Editor A
participant DA as Datastar A
participant P as POSTSubmit
participant S as store
participant N as broker
participant O as OnDocChanged
participant B as Tab B
Note over EA: Quill reports an edit as a Delta
EA->>DA: docs-submit event {client, seq, rev, delta}
DA->>P: POST /doc/{id}/submit
P->>S: Submit(change, rev)
Note over S: Transform against the changes after rev,<br/>apply as the next revision,<br/>append to the log
P->>N: Dispatch EventDocChanged
P-->>DA: 200 OK
N->>O: Deliver to the stream of Tab A
O->>S: Read the changes after the revision of Tab A
O->>DA: ExecuteScript over SSE
DA->>EA: receive(changes)
Note over EA: Own change acknowledged,<br/>send the buffer
N->>O: Deliver to the stream of Tab B
O->>S: Read the changes after the revision of Tab B
O->>B: ExecuteScript over SSE: receive(changes)
Note over B: Transform against its own<br/>unacknowledged changes, apply
The server sends the title, collaborators and cursors in the same way. POSTRename changes the title, and POSTCursor moves a cursor. StreamOpen adds the tab to the collaborators. StreamClose removes it. Each operation changes the store and dispatches EventDocChanged, which carries only the document id. StateDoc records the last state sent to a tab. OnDocChanged compares it with the store and sends the difference. It passes the changes and cursors to receive, then patches the avatars and the title as HTML. The _title signal gives the editor the title it stores in the browser.
EventListChanged reports a created, renamed, edited or deleted document. OnListChanged renders the document list again in every tab on /.
The code is in these directories:
| path | what |
|---|---|
app/ |
pages, actions, events and templates |
app/datapagesgen/ |
generated by datapages gen |
store/ |
documents in memory and on disk, submitted change transformations, open tabs and cursor positions |
delta/ |
the Delta operations of quill-delta, ported to Go |
richtext/ |
allowed document formats, repairs for concurrent changes and document HTML |
images/ |
uploaded images, 1 file per image named after the SHA-256 of its bytes |
templates/ |
the documents a new document can start from, and their pictures |
web/src/ |
the editor and styles that esbuild bundles into app/static/ |
cmd/server/ |
the entry point |
go run ./cmd/server -host 0.0.0.0:8080Open http://localhost:8080/. To edit from another device, open http://<this-machine>:8080/ on that device. Anyone who can reach the port can edit and delete documents. Without -host, the server listens only on localhost:8080. Use -dir to choose another document directory.
make npm-install
make devOpen http://localhost:7331/. Editing a file in web/src/ rebuilds editor.js, editor.css and style.css in app/static/, then reloads the page.
make testgo test ./... tests the Delta port, document rules, store and HTTP protocol. npm test in web/ runs 3 editors against a server model. It interleaves requests and streams at random, then checks that every editor ends with the server's document.
Edited text runs ahead of the server. Only the browser can merge changes from other users without losing keystrokes or moving the cursor. Datastar can replace an element or leave it unchanged, but it cannot merge changes into the element.
The page renders the document in Go with Quill's markup. The document appears before the bundle loads and stays in place when the editor replaces it.
A CRDT such as Yjs merges changes without a central order. It needs an id for every character and a tombstone for every deletion. This example uses a server to order changes, which keeps the document a plain Quill Delta that the server can check, repair and render as HTML.
A CRDT has these advantages:
- Several servers or browsers can accept and merge changes.
- A tab can rejoin after any number of changes.
- Editors can send changes without waiting for a reply.
- Duplicate or out-of-order changes do no harm.
- Cursors can stay anchored as the text changes.
A Delta is a flat list of formatted text runs. Each line break carries its line's formats. delta/ ports the compose, transform and transformPosition operations from quill-delta 5.1.0 to Go. The Go tests compare the port with quill-delta results in delta/testdata/quill-delta.json. web/vectors.mjs writes that file. Positions count UTF-16 code units, as they do in JavaScript.
The editor sends 1 change at a time with the tab's client id and a sequence number. It buffers text that the user types while the change is pending. The editor sends the buffer when the change returns.
A dropped event loses no data. The broker drops events only for a stream with a backlog. The next delivered event includes everything covered by the dropped events. When a stream reconnects, StreamOpen reads the editor revision from the rev signal and sends the missing changes. The editor keeps this signal current. The page keeps its stream open while the tab is hidden because the default reload on return would discard the undo history.
A change can arrive 2 times. Datastar retries a request that failed to connect. The editor resends a change if it has not returned after 10 seconds. The store skips a change if it has already applied its client id and sequence number.
GET gives each page load a random client id. The tab uses it to send changes. The server never sends it to another tab, which sees a name and color derived from the id.
2 valid concurrent changes can produce a document that Quill cannot show. One tab may make a line a heading while another makes it a list item, which puts 2 line types on 1 line. Concurrent changes can also put text formats on a code line. Quill then shows the line differently in each editor because the editors applied the changes in different orders. richtext.Repair computes a change that removes the conflict, and the store applies it as the next revision. The repair keeps and reapplies the last line type, which makes every editor use it.
The editor reports its selection with POSTCursor only when the server has all its changes. Only then do its positions match the last revision. The store moves every cursor with every change. It moves a cursor past an insertion at that cursor from the same editor, but keeps it before an insertion from another editor. This matches how Quill moves the cursor in each editor. Before drawing received cursors, the editor moves them through its own unacknowledged changes.
store/ keeps each document in 2 files. <id>.json is a snapshot. <id>.log contains 1 JSON line per change. After 200 logged changes, the store writes a new snapshot and keeps the last 100 changes in the log. A process crash loses no applied changes because the next start replays the log.
The store keeps the last 1000 changes of a document in memory. It uses them to transform changes made at earlier revisions. After a restart, this history contains the logged changes: all changes since the last snapshot and the 100 changes before it. A tab that falls further behind cannot catch up. Its editor becomes read-only and offers a reload. After the reload, the changes the server lacked go to a new document, as described in Disconnections.
A reload, closed tab or lost connection preserves changes. They merge when the store still has the history after the tab's last revision, up to 1000 changes. Otherwise the saved content becomes a new document.
Each tab saves its editor state to IndexedDB at most every 500 ms while it changes. The record contains the content, last revision, pending change with its sequence number, and input buffer. web/src/local.ts manages these records. An open tab holds a Web Lock named after its client id. An unlocked record belongs to a closed tab.
On load, the editor sends each closed-tab record with missing changes to POSTRestore. store.Restore processes the server changes after the saved revision. A matching client id and sequence number acknowledges the pending change. The store applies the remaining changes, and the streams deliver them as remote edits. The editor then deletes the record. If the store no longer has the saved revision, POSTRestore copies the saved content to a new document and links it in a toast.
After the first page load, the modules/offline service worker stores PageOffline and its files. WithDatastarJS serves Datastar from app/static/ because the CDN is unavailable offline. While offline, the worker serves PageOffline for any address it has not cached. At a document address, the editor resumes a closed-tab record with missing changes or uses the newest stored content. New changes stay in the record. When a HEAD request succeeds, the page reloads and restores the record. At other addresses, the page lists stored documents.
sequenceDiagram
participant E as Editor Web Component
participant I as IndexedDB
participant W as Service worker
participant D as Datastar
participant P as POSTRestore
participant S as store
participant N as broker
participant O as OnDocChanged
Note over E: The connection drops,<br/>the user keeps typing
E->>I: Save content, revision, pending change, buffer
Note over E: Reload while offline
W->>E: PageOffline for /doc/{id}/
E->>I: Read the closed-tab record
Note over E: Resume as that tab,<br/>new changes stay in the record
E->>I: Save the record
Note over E: A HEAD request succeeds,<br/>reload from the server
E->>I: Read the records of the document
Note over E: Lock each closed-tab record<br/>with missing changes
E->>D: docs-restore event {client, seq, rev, sent, buffer, title, content}
D->>P: POST /doc/{id}/restore
P->>S: Restore(pending)
alt rev within the history
Note over S: Skip the change with the saved client and seq,<br/>transform the rest against the changes after rev,<br/>apply as the next revisions
P->>N: Dispatch EventDocChanged
N->>O: Deliver to every stream of the document
O->>E: ExecuteScript over SSE: receive(changes)
else rev older than the history
P->>S: Create a document from the saved content
P-->>D: Toast with a link to the copy
end
P-->>D: ExecuteScript: restored(client)
D->>E: restored(client)
E->>I: Delete the record
IndexedDB records, Web Locks and the service worker require HTTPS or localhost. Over plain HTTP at http://<this-machine>:8080/, a reload loses changes missing from the server. Safari deletes site storage after 7 days of Safari use without interaction with the site.
The example runs 1 server with the in-memory broker. Multiple servers would need a shared store and 1 place that orders each document's changes. For example, they could use a database row whose revision each change advances with compare-and-swap.