OpenPWA Evidence Layers

The OpenPWA Evidence Layers tell you, for any claim about a web platform feature, whether a source documents it, a demo shows it, or an OpenPWA run measured it on a real device. Each claim carries two labels: its layer (Support, Capability, or Behaviour) and the class of evidence behind it.

This is version 1.0, dated 2026-10-20, and the latest. The text as published that day is kept unchanged at version 1.0, later changes go in the version history, and the full paper adds the prior art, the references, and a naming appendix.

A Support claim says whether a browser ships the feature and since which version. A Capability claim says what the feature can do, and points to a demo you can run. A Behaviour claim says what happens on a named device, OS build, and launch context, as measured in an OpenPWA run. The three layers map to three evidence classes: documented, demonstrated, and measured. A measured result is attached to the support row it concerns as a dated condition, and the upstream source stays as it was. On openpwa.net, device-test results are published in the Device Lab.

Compatibility data already has notes and partial support, demo sites already show capabilities, and practitioners publish device notes. The framework names the separation between them, grades each claim by its evidence, and gives installed-app behaviour a layer of its own with its own evidence requirements. The full paper sets out the prior art and what is and is not new.

The three layers

A claim is one statement about a web platform feature that a reader could act on. The question it answers puts it in exactly one layer; what backs it gives it one evidence class.

	Layer 1, Support	Layer 2, Capability	Layer 3, Behaviour
Question	Is it supported, and since which version?	What can it do? Show me.	What exactly happens on this device, OS build, and context?
Sources	browser-compat-data (BCD), specifications, vendor documentation, release notes, and posts	OpenPWA’s runnable demos under /demo/	Dated runs of a published protocol on a named real device
Evidence class	documented	demonstrated	measured (native), or documented when a source describes the behaviour
Shown with	Source name and the date it was checked	Demo link, with wording that invites you to try it on your own device	Date, device, OS version and build, harness version, confidence label, and a link to the data
Where	Compatibility tables, reference availability banners	/demo/ and “Try it” links	Device Lab behaviour pages and footnotes on compatibility rows
Grading	The computed confidence described in the methodology (high, medium, low)	None; a demo shows that something is possible, and says nothing about support	A (instrumented read) or S (screenshot or recording)
Unit	feature × browser × version range	feature × demo	behaviour × device × OS build × context × date

Support

A support claim says that a feature is present, absent, partly present, behind a flag, or removed in a named browser and version range, as a source states it. If the source itself attaches a condition (“Home Screen web apps only”, “requires user activation”), the condition is part of the support claim, because it limits where the “yes” applies. Every support claim comes from a source, so support is documented by definition; OpenPWA relays and cross-checks these statements and does not author them. An OpenPWA measurement that bears on a support statement stays a behaviour record and is attached to the row as a relation.

Capability

A capability claim says that a feature can do a task, and is backed by a runnable page that tries it: requesting permission, setting a badge, sharing a file, subscribing to push. It claims only that the task is possible. What the demo does on your phone is your own observation. A demo run that OpenPWA records under the protocol yields a behaviour record; it does not make the capability claim any stronger.

Behaviour

A behaviour claim says what a feature does under stated conditions. That covers which input wins when two conflict, the default when one is missing, how a Home Screen app differs from a browser tab, timing and lifecycle, geometry and rendering, and pitfalls. A measured behaviour claim names a device, an OS version and build, a context, and a date. A behaviour that a source describes and OpenPWA has not run is shown as Documented by <source>; not measured by OpenPWA. In Safari 17.2, WebKit describes cookies being copied into a newly added Home Screen web app; that stays a documented behaviour claim until a run measures it.

Assigning a claim to a layer

Ask these questions in order and stop at the first “yes”.

1. Does the claim say what happens when the feature is used, or how the result depends on device, OS build, launch context, or state (values, precedence, defaults, timing, lifecycle, rendering, prompts)? Then it is a Layer 3, Behaviour claim. It is measured if OpenPWA ran it under the protocol, and documented if a source states it.
2. Does it say the feature can do a task and point to a runnable page, with no recorded run? Then it is a Layer 2, Capability claim, demonstrated.
3. Does it say whether, where, or since when the feature is available (presence, version, flag, partial, a condition the source states)? Then it is a Layer 1, Support claim, documented. If the claim came from an OpenPWA run and not from a source, go back to step 1: it is a behaviour record attached to the row.
4. If none of these applies, it is not a platform claim. Label it as advice, history, or opinion, or remove it.

Split a mixed statement (“supported since 16.4, and fires only in Home Screen apps”) into one claim per layer. A version number from a run (“works on 18.6.2”) is a behaviour claim about one build and does not establish version_added. If a demo fails on your device, that is your observation; it becomes an OpenPWA claim only once it is reproduced under the protocol.

Three classes of evidence

A documented claim is stated by a source outside OpenPWA: a specification, vendor documentation, release notes, a vendor engineering post, a vendor bug tracker, or BCD. It is shown with the source and the date it was checked. BCD rows count as documented even when BCD collected the data automatically, because OpenPWA did not observe them. Third-party notes such as practitioner blogs and forums are also documented, with lower confidence under the high, medium, and low rule in the methodology.

A demonstrated claim is backed by a runnable OpenPWA demo that exercises the capability. It is shown with the demo link and wording that invites you to try it on your own device. The label means the demo exists and was built to exercise the feature. It makes no promise that the feature works on any particular device.

A measured claim comes from an OpenPWA run of a published protocol on a named real device. Only runs published with confidence A or S carry the label.

What a measured result records

A run counts as measured only if it records every item in this list:

• an environment block: device name and model identifier, OS version and build, release channel, browser version, language and region, non-default flags, content blockers or VPN state, and the inspecting host if one was used;
• the date and time, with time zone, for the run and for each read;
• the context: tab, Home Screen app, or both; fresh state or not; first launch or relaunch;
• the protocol and harness: the test id, the published steps, and the harness version with a SHA-256 hash for each file the run loaded;
• the expected result and its source, written down before the step;
• the raw data: instrumented reads as files, screenshots, and recordings, each listed with a hash;
• a verdict (as expected, differs, inconclusive, or could not run) and a confidence label.

The iOS 18.6.2 study ran before these requirements were written and does not meet all of them. Its report and the paper list the gaps. Raw reads carry tool-hash sidecars only from notebook entry N15 onwards. Run-time hash snapshots exist for some harness files, not all. The environment record lacks feature flags, VPN and iCloud Private Relay state, the identity of an installed Safari Web Extension, and the Settings language and region.

Confidence labels within measured

Label	Meaning	Published as measured?
A, instrumented read	The output of a published probe script, read from the page over a debugging protocol (for example the WebKit remote inspector) and stored verbatim as JSON. The value is the probe’s own output, not a model’s reading of it.	Yes
S, screenshot or recording	A capture analysed afterwards: pixel values, video frames, capture times.	Yes
O, observer report	The operator’s verbal report only.	No. It stays in the notebook and may support an A or S result, never stand alone.

Labels combine. “A+S” means an instrumented read and a capture agree; “S+O” means a capture with the operator’s report as context. A published result names its weakest essential component.

How a measured result relates to a support row

A behaviour result is attached to the support row it bears on, with one of three relations.

Relation	When it applies	Example
agrees	The measurement matches the row.	After setAppBadge(3) the Home Screen icon showed a badge, and BCD lists the Badging API as supported.
adds-condition	The row holds under a condition the source does not state.	On iOS 18.6.2 window.pushManager also exists in a Safari tab, and with a subpath-scoped worker it does not return that worker’s subscription.
differs	The measurement contradicts the source.	notificationclick fired in Home Screen apps where BCD records it as unsupported (see the worked example).

The row keeps its source, status, version, and confidence. Under the framework, the relation and its condition are shown in the row, dated and linked to the run. A differs relation appears in the row itself as well as in any footnote, and becomes a candidate for an upstream report such as a BCD issue or a WebKit bug. When a later run differs from an earlier one, both stay visible, each with its build. In version 1.0 a measurement never changes a row’s status (principle P8).

Twelve principles

P1. Every claim has a layer and an evidence class. Both are shown as text, so a reader does not depend on colour to see them.

P2. Classes are never upgraded by wording. “Tested”, “verified on device”, “in practice”, and their Chinese equivalents are used only for published measured runs.

P3. Behaviour is versioned per OS build and context. A behaviour claim names device, OS version and build, context, and date. Each build gets its own row.

P4. Measurements are dated, reproducible, and come with raw data. The protocol, harness version and file hashes, environment block, raw reads, and captures are all published in sanitised form.

P5. Expectations are written before the step. The expected result and its source are recorded first. A result that differs from the expectation is a finding.

P6. Records beat reports. When an operator’s report conflicts with a read or a capture, the record is preferred and the conflict is logged as an anomaly. A result backed only by an observer is never published as measured.

P7. Corrections are append-only. To correct a wrong entry, add a new dated entry that cites the old one. Retracted readings are listed as “do not cite” next to the result they were retracted from.

P8. Layer 3 conditions layer 1; it does not silently overwrite it. In version 1.0 a measurement never changes a support row’s status, version, or confidence. A future rule that allows it would first be published as a new major version of this framework, and only then applied.

P9. Scope is stated. Every result gives its sample size (devices, builds, runs), what was not tested, and known threats to validity.

P10. Novelty is checked, replication is labelled. A result that confirms a published description is labelled a replication and cites that description. A result is called new only when no prior description was found, and the search is described.

P11. The observer protocol protects the record. When a person operates the device and an instrument reads it, these rules are part of the method. Define every term before using it. Ask one question per message. Confirm a step was done before asking for its result. Leave the screen untouched for 20 to 30 seconds while an instrumented read runs. Do not write test markers through browser automation. In the iOS 18.6.2 study the safaridriver automation window had its own website data, separate from the owner’s Safari profile (notebook N06).

P12. Harness bytes and installed apps are frozen between comparable runs. Once a run has used a harness version, that version is never moved, redirected, or edited. The one exception is a privacy or security fix, published as a dated erratum with the old and new hashes. New versions get new, non-overlapping scopes. Test apps installed for a before-and-after study keep their scope and start URL until the after run is done.

Worked example with notificationclick on iOS

The service worker notificationclick event on iOS sits in all three layers at once, and each layer says something different about it.

Support, documented by BCD

BCD records api.ServiceWorkerGlobalScope.notificationclick_event as version_added: false for Safari on iOS (ServiceWorkerGlobalScope.json, github.com). BCD updates are assisted by mdn-bcd-collector (github.com), a web service that loads generated feature-detection tests as pages in a browser. In the issue cited next, a BCD contributor says the collector checks 'onnotificationclick' in self. On iOS 18.6.2 that check was false in a worker controlled by a Safari tab, which fits a “no” collected in a tab context; the study did not establish which collector run produced the row. A BCD issue opened in April 2024 (github.com) reports that the event does fire. On openpwa.net the row stays as BCD states it.

Capability, demonstrated by the push demo

The push permission page asks for notification permission and subscribes your browser to push, the steps that come before any notification tap. Try it on your device. The page does not show a notification or fire notificationclick, so it demonstrates push and tells you nothing about the click event on your phone.

Behaviour, measured in Home Screen apps

On an iPhone 16 Pro with iOS 18.6.2 (build 22G100), on 2026-10-20, notificationclick fired on six of six taps in a web app added to the Home Screen, through both addEventListener and the onnotificationclick property, and focus() on the existing window succeeded (confidence A). In a worker controlled by the Home Screen app, 'onnotificationclick' in self was true. Every tap had a live window. Taps with no live page, which comment 35 on WebKit bug 268797 (bugs.webkit.org) reports as failing, were not tested. The relation to the BCD row is differs. Under the framework it is shown in the row with the condition and the run date, and the row’s status does not change (P8). The result and its raw data are on the behaviour page notificationclick in Home Screen apps, with the study’s data package.

How to cite

Cite the version you read. For version 1.0:

OpenPWA Editorial Team. The OpenPWA Evidence Layers: support, capability and behaviour as separate claims about web platform features. Version 1.0, 2026-10-20. openpwa.net. https://openpwa.net/about/evidence/v1.0/

In running text, “OpenPWA Evidence Layers v1.0” is enough. To cite one definition, add its fragment, for example #p8 or #decision-procedure; the fragments are the same on every version page and in both languages.

The framework text is licensed under Creative Commons Attribution 4.0 International (creativecommons.org). You may copy, translate, adapt, or extend it, including commercially, on the licence’s terms, which include giving credit, linking to the licence, and indicating changes. Using the framework as a method needs no licence. To adopt it, a site or dataset labels each claim with its layer and evidence class, publishes the requirements it applies for “measured”, and names the version it follows. It does not need OpenPWA’s labels, data model, or harness.

Device steps performed by an OpenPWA operator on an iPhone 16 Pro. AI agent sessions read page state over USB, kept the notebook and drafted this text; automated gates and an AI fact-check checked it. No person reviews each entry.