Internet-Draft M. Toomim Expires: Jan 8, 2020 Invisible College Intended status: Proposed Standard R. Walker Invisible College July 8, 2019 The Braid Protocol: Synchronization for HTTP draft-toomim-braid-00 Abstract Braid is a proposal for a new version of HTTP that transforms it from a state *transfer* protocol into a state *synchronization* protocol. Braid puts the power of Operational Transform and CRDTs onto the web, improving network performance and robustness, and enabling peer-to-peer web applications. At the same time, Braid creates an open standard for the dynamic internal state of websites. Programmers can access state uniformly, whether local or on another website. This creates a separation of UI from State, and allows any user to edit or choose their own UI for any website's state. We have a working prototype of the Braid, and have deployed it with production websites. This memo describes the protocol, how it differs from prior versions of HTTP, and a plan to deploy it in a backwards-compatible way, where web developers can opt into the new synchronization features without breaking the rest of the web. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at https://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at https://www.ietf.org/shadow.html Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Synchronization . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Deployment and Upgrade Plan . . . . . . . . . . . . . . . . . 10 4. Proposed Changes to HTTP. . . . . . . . . . . . . . . . . . . 11 4.1. Linked JSON . . . . . . . . . . . . . . . . . . . . . . . 12 4.2. Generalized request/response . . . . . . . . . . . . . . . 13 4.3. Subscriptions . . . . . . . . . . . . . . . . . . . . . . 14 4.4. Versioning . . . . . . . . . . . . . . . . . . . . . . . . 15 4.4.1. Versions . . . . . . . . . . . . . . . . . . . . . . . . 16 4.4.2. Patches . . . . . . . . . . . . . . . . . . . . . . . . 17 4.4.3. Merge Types . . . . . . . . . . . . . . . . . . . . . . . 18 4.4.4. Acknowledgements . . . . . . . . . . . . . . . . . . . . 19 5. Network Messages . . . . . . . . . . . . . . . . . . . . . . 20 6. Security Considerations . . . . . . . . . . . . . . . . . . . 23 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 8. Copyright Notice . . . . . . . . . . . . . . . . . . . . . . 23 9. Author's Address . . . . . . . . . . . . . . . . . . . . . . 23 1. Introduction HTTP was initially designed to transfer static pages. If a page changes, it is the client's responsibility to issue another GET request. This made sense when pages were static and written by hand. However, today's websites are dynamic, generated from databases, and continuously mutate as their state changes. Now we need state *synchronization*, not just state *transfer*. Unfortunately, there is no standard way to synchronize. Instead, programmers write non-standard code; wiring together custom protocols over WebSockets and long-polling XMLHTTPrequests with stacks of Javascript frameworks. The task of connecting a UI with data is one that every dynamic website has to do, but there is no standard way to do it. ======= HTTP Websites ======= ====== Braid Websites ====== Today's websites are Braid generalizes HTTP and generated from multiple REST into a uniform standard layers of state across that synchronizes state multiple computers. Each within and between dynamic layer has a different API. websites. x Non-standard state API o Standard state API _Client__ / \ : o o o o : Webpage DOM o o o o State : \| \| : \| \| : x x : HTML Templates o o State : /| /| : /| /| : x x x x : JS Models o o o o State \ | | | | / | | | | | | | | | | | | o o o o - http:// - o o o o - braid:// - / | | | | \ | | | | : x x x x : Views o o o o State : | \| | : | \| | : x x x : Controllers o o o State : \ / \| : \ / \| : x x : Models o o State : \ / : \ / \.... x ../ Database o State Server Today's programmers have to On the braid, each piece of learn each API, and wire them state (o) has a URL; whether together, making sure that public, or internal to a changes to shared state client or server. Any state synchronize across all can be a function of other layers and computers. state, and dynamically recomputes when its dependencies change. Braid guarantees the network will synchronize. As the web becomes more dynamic and data-driven, the complexity of the non-standard Javascript stack grows, and an increasing amount of data is inaccessible to the open web. The result is a web which is open on the surface, but closed internally: websites can link to each other's *pages*, but cannot easily share each other's internal *state*. We can solve this by generalizing HTTP into a *synchronization* protocol, which replaces the complex Javascript stack, while providing new features, and making website internal state accessible anywhere desired, and realtime synchronized by default. We have a working prototype of the Braid protocol, and have deployed it with production websites. The prototype is implemented as a polyfill library, which adds Braid features to existing browsers and servers. This document describes the new protocol, how it differs from prior versions of HTTP, and a plan to deploy it in a backwards-compatible way, where web developers can opt into the new synchronization features without breaking the rest of the web. 2. Synchronization Braid incorporates the abilities of Operational Transform and CRDTs. These are approaches to solving *synchronization*. Synchronization is a problem that occurs whenever two or more computers or threads access the same state. Synchronization code is tricky to write, and can result in clobbers, corruptions, or race conditions. This is a challenging problem, which has seen a number of partial attempts in HTTP, such as e-tags, cache control, PATCH, JSON-diff, and SSE. Luckily, a set of maturing synchronization technologies (such as Operational Transform and CRDTs) can now automate and encapsulate synchronization within a library. They can synchronize arbitrary JSON data structures across an arbitrary set of computers that make arbitrary mutations, and consistently merge their edits into a valid result, without a central server, in the face of arbitrary network delays and dropouts. In other words, it is now possible to interact with state stored anywhere on a network as if it is a local variable, and program as if it is already downloaded and always up-to-date. Unfortunately, each synchronizer implements a different protocol, with a different set of features and tradeoffs. Braid proposes a common language for synchronizers, so that they can interoperate, and implements it as an extension to HTTP. This lets multiple synchronizers interoperate, if they agree on a way to consistently resolve ambiguities -- a *merge type*. We have run tests that succesfully interoperate a CRDT and OT system over the common Braid protocol. When applying synchronization to the web, we see the power of synchronization manifest in these concrete ways: - Caches update automatically and instantly, because servers promise to push changes to their subscribers. This obsoletes the `cache-control` and `refresh` headers, and the `max-age` heuristic. Users never need to force-clear their cache. - Updates go over the network as diffs, which can be much smaller than the resources they modify, significantly reducing network usage. - Web apps get an offline mode for free. Edits from multiple clients merge automatically once they come online. Network failures recover transparently. - *Reload* buttons in browsers become unnecessary, and can be removed for braid sites. Browsers automatically discover and display the most recent version on their own. - Web apps require roughly 70% less code to build (in our experiments), because programmers do not need web frameworks or custom logic to wire together a stack of server-state and client-state. This work is automated by the protocol. - Every