Synchronized contexts

A content model can be split across Vox applications: a PIM authors categories, a CMS authors category pages and manages the content releases, and a delivery application publishes both to a website. Each entity type is authored in one application only, and the others receive it as an external entity, see Share entities between Vox applications. Context synchronization lets the receivers read, calculate on and project the source's drafts, content releases and mergeable contexts, eg experiments, too, without authoring them. Every application can then preview the others' work, and the delivery application can publish the combined preview, with relations resolved across sources.

Application	Authors	Receives
PIM	Category	CategoryContentPage, the releases
CMS	CategoryContentPage, the releases	Category
Delivery	Nothing	Both entity types and the releases

One authority per context type

$release and each mergeable application context type have one authority: they're either owned by the application or synchronized from exactly one source. The applications agree which of them manages the releases, and the others receive those release entities with the same ids. There's no creation of releases in several applications and no mapping of ids. $draft has no members, so the drafts of every source and the local ones are all selected by $draft.

Authoring stays local. An external entity's cells at a draft, release or experiment are its source's data: they're never edited, merged, restored or scheduled where they're received, and a receiver never merges a member, also when its source deletes it. Receiving a release doesn't make the application a participant in it either: a PIM that only consumes the CMS's releases can preview them, but can't save its categories in them. A PIM that participates in them can, and the CMS publishes them with every participant's changes, see Shared releases.

Configuring a receiver

Context synchronization is configured per source, on its own and without UseAuthoring:

services.AddVox(options => options
    .UseContextSynchronization(sync => sync
        .Consume("pim", pim => pim
            .Drafts()
            .Entities<Category>())
        .Consume("cms", cms => cms
            .Drafts()
            .Releases<CmsRelease>()
            .ContextType<Experiment>()
            .Entities<CategoryContentPage>())));

• Drafts() receives the drafts of the source, and adds $draft to the model if authoring doesn't.
• Releases<TRelease>() binds an external entity type to $release. The type implements IVoxContentRelease, the release descriptor that the owned VoxContentRelease implements too, so Vox reads the status and openness of a release the same way whichever application manages it. A type generated from the source's contract of VoxContentRelease implements it. The application then has no releases of its own: call authoring.WithoutContentReleases() if it authors other content.
• ContextType<TContext>() receives the cells of an external mergeable application context type, eg the variants of an experiment. Configure its traits like the source's, eg options.ConfigureContextType<Experiment>(type => type.Mergeable()).
• Entities<TEntity>() receives the cells of an external entity type at the keys of the source's mergeable contexts. Its HasMergeableCells becomes true, so its drafts, releases and experiments are stored, read and calculated on like local ones, while IsAuthored stays false.

Startup rejects an external release type together with releases of the application's own, two sources of one context type, a supplied type that isn't external or isn't synchronized from the source, and an external mergeable context type that no source supplies. The external entity types with cells, and the context entity types, are stored by a provider with the ContextCells capability, eg Entity Framework.

A consuming application needs normal calculation and projection processing, but no publisher, merger, restore processor or release scheduler, and it has none.

What the source sends

The source opts a raw snapshot subscription in to contextual content by choosing the mergeable context types in SidecarContextTypes, eg ["$draft", "$release", "Language"]. Choosing $draft or $release needs the permission to read them. Its snapshots then carry:

• the cells at complete keys, eg {$draft}, {$release=autumn}, {$draft|$release=autumn}, {Experiment=hero-b} and {$draft|Language=sv}, where a cell at a key of mergeable contexts has every value of the entity there;
• the existence of the entity there, so an entity that only exists in drafts or releases is sent rather than deleted, with a cell at {} that says it isn't published, and an entity that's deleted in a release has a cell that says so;
• the scope: the context types the snapshot is complete for, so the receiver removes what's no longer there, eg after a draft is reverted or published, a release is published or deleted, or an experiment is promoted;
• the revision of the entity, see below, and the incarnations of the release and experiment members its keys have.

The source records a change event for every edit of a draft, a release or an experiment, which only the subscriptions that opted in to the context types of its key get. Other subscriptions and the search index get published changes only. A resynchronization sends the entities that only exist in drafts or releases, and the deletions of the context members that were deleted, so a receiver that missed a deletion retires the member.

Source conflict state, audit, publication records and schedules are never sent.

Ordering and atomic import

A snapshot carries a revision that changes with everything it exports, also with a draft edit that leaves the published version and time unchanged. It's increased in the transaction of every change of an entity with cells and read consistently with the snapshot, so a newer revision never labels older content. The receiver keeps the revision of each stream of each source, a stream being a subscription, and ignores a snapshot it already has or is past, also after the entity was deleted, so duplicates, reordered and late deliveries change nothing.

Every snapshot is imported in one transaction: the external entity, its cells, its existence cells and its stream's revision, and the changes that make what depends on them recalculate at the keys they changed. A failure leaves the previous revision intact, and the delivery is retried. The import replaces only the source's synchronized fields within the scope of the snapshot, so local enrichments, local calculated results and the cells delivered by another stream are kept. Two streams can't deliver the same context type of an entity.

A cell of a member that hasn't arrived yet is kept, and resolves once it arrives: the arrival of a release or an experiment variant, or a change of it, eg of its status, recalculates what's derived from the entities with cells at its keys. What happens to a cell of a context type the receiver doesn't have is a setting of the subscription, UnknownContextTypes, carried on its snapshots:

• Drop leaves the cell out, as before.
• Hold doesn't import the snapshot: its delivery fails and is retried until the receiver has the context type, so nothing is lost while it catches up with a new context type of the source.

Retired members

A member deleted at its source, eg a published or deleted release or a promoted experiment variant, arrives as the deletion of the external context entity, which deletes every cell keyed by it. A promoted variant's values arrive as published snapshots of the entities it changed. Every deleted context member is kept as a retired member, with its incarnation, so a late snapshot, a queued calculation or a resynchronization can't bring its cells back. A member that's created again with the same id, eg an experiment variant named hero-b again, is a new incarnation, and the cells of the earlier one are dropped.

Validating the source's traits

A receiver resolves the source's contexts with its own traits, which must be the source's: mergeability, restriction, existence, selection kind, portability, the stacking order of the shared context types, and the fallbacks, priorities and criteria of the members. The source's mergeable key is never read as a non-mergeable one. Send the source's manifest, GET api/contexts/manifest?includeRestricted=true there, to the receiver, signed like snapshots:

POST /vox/api/entity-sync/cms/manifest
X-Vox-Signature: sha256=...

It returns the differences as a validation problem, or 204 when the traits agree. A snapshot built with another manifest revision than the one validated last is logged as a warning. IVoxContextSourceManifestValidator validates a manifest in .NET.

Reading, calculating and projecting

The external entities' contextual cells resolve like local ones: context=$draft reads the drafts of every source and the local ones, context=$release:autumn the CMS's release, and they combine with languages, markets and the other contexts, eg context=$draft,language:sv. Relations across sources resolve at the same key, so the delivery application's calculation of a page's heading from its category reads the PIM's draft of the category and the CMS's draft of the page in $draft. Local calculated fields vary at these keys when they depend on contextual content; they stay derived values, and are calculated again when an import changes what they read.

A durable projection publication enumerates restricted contexts of external entity types only by explicit opt-in, as it does for local ones, eg [VoxEntityProjectionPublication(IncludeRestricted = ["$draft", "$release"])], so a delivery model with only external types publishes a combined preview.

Query and search keep filtering by published values. There's no contextual search index for preview.

Preview permissions

Selecting a restricted context type also works with its preview permission, vox:Preview:{contextType}, eg vox:Preview:$release or vox:Preview:$draft, which needs no permission to edit, publish or manage releases. Grant it to the API clients and users of a delivery application that only previews. It applies to context=, API client default selections, the restricted manifest, which only has the restricted context types the caller may select, subscription configuration, exports, relation traversal and projection endpoints. Restricted responses stay private and out of shared caches.

The admin offers the received draft, release and experiment selections, with the release preview banner, and shows external entities read-only in them. The Releases page lists synchronized releases read-only, with their status.

Eventual consistency

Each application imports atomically, but there's no transaction across databases. While a change travels from its source to a delivery application, eg a CMS page whose draft refers to a PIM category's new draft, the delivery application can show one source's new version next to the other's old one. That interval lasts until both snapshots have arrived and the calculations and projections that depend on them have run; it's usually seconds, and longer while a source or a transport is unavailable. A preview converges once everything has arrived; atomic activation of a combined preview in delivery sinks is a planned follow-up.

Participating in the source's releases

An application can participate in the releases and shared experiments of its source instead of only consuming them, eg the PIM saving its categories in the CMS's release, which the CMS then publishes, merges or discards together with every participant's changes, see Shared releases. A consumer is never a participant.