OpenPWA Evidence Layers 1.0

This is the frozen text of the OpenPWA Evidence Layers (Support, Capability, Behaviour). Version 1.0, 2026-10-20. Under the framework’s version policy, a frozen version page is not rewritten; a factual error found later is shown in a dated erratum box at the top and the text below stays as published. The latest version is at /about/evidence/, and the version history lists every version. The full paper adds the prior art, the references, and a naming appendix.

Markdown download of the full paper: openpwa-evidence-layers-v1.0.md (English), SHA-256 f23b1e63109bae469707e0a6c98dc3ab4e26a80e04e76d258437d89d03869c2f.

The OpenPWA Evidence Layers separate three kinds of claim about a web platform feature. Support says whether a browser ships the feature and since which version. Capability says what the feature can do, shown by a demo you can run. Behaviour says what happens on a named device, OS build, and launch context, measured by an OpenPWA run. Each layer has its own evidence class (documented, demonstrated, measured), and the framework asks that every claim carry both its layer and the class of evidence actually behind it. Under the framework, measured results attach to support rows as dated conditions and do not overwrite the upstream source. On openpwa.net, device-test results are published in the Device Lab. None of these ingredients is new on its own: compatibility data has notes and partial support, demo sites show capabilities, and practitioners publish device notes. The framework formalises and names their separation, grades each claim by its evidence, and treats installed-app behaviour as a layer with its own evidence requirements. The full paper sets out the prior art and what is and is not new.

The three layers

A claim is one statement about a web platform feature that a reader could act on. It belongs to exactly one layer, decided by the question it answers, and carries one evidence class, decided by what backs it.

	Layer 1, Support	Layer 2, Capability	Layer 3, Behaviour
Question	Is it supported, and since which version?	What can it do? Show me.	What exactly happens on this device, OS build, and context?
Sources	browser-compat-data (BCD), specifications, vendor documentation, release notes, and posts	OpenPWA’s runnable demos under /demo/	Dated runs of a published protocol on a named real device
Evidence class	documented	demonstrated	measured (native), or documented when a source describes the behaviour
Shown with	Source name and the date it was checked	Demo link, with wording that invites you to try it on your own device	Date, device, OS version and build, harness version, confidence label, and a link to the data
Where	Compatibility tables, reference availability banners	/demo/ and “Try it” links	Device Lab behaviour pages and footnotes on compatibility rows
Grading	The computed confidence described in the methodology (high, medium, low)	None: a demo shows possibility, not support	A (instrumented read) or S (screenshot or recording)
Unit	feature × browser × version range	feature × demo	behaviour × device × OS build × context × date

Support

Support claims state presence, absence, partial presence, flag status, or removal of a feature in a named browser and version range, as a source states it. Availability conditions that the source itself states (“Home Screen web apps only”, “requires user activation”) belong to the support claim, because they bound where the “yes” applies. Support is documented by definition: OpenPWA relays and cross-checks support statements and does not author them. When an OpenPWA measurement bears on a support statement, the measurement stays a behaviour record and is attached to the row as a relation.

Capability

Capability claims say that a feature can accomplish a task, backed by a runnable page that attempts it: requesting permission, setting a badge, sharing a file, subscribing to push. The claim is about possibility. What the demo does on your phone is your observation. A demo run that OpenPWA records under the protocol produces a behaviour record, not a stronger capability claim.

Behaviour

Behaviour claims say what a feature does under stated conditions: precedence when inputs conflict, defaults when an input is missing, differences between a Home Screen app and a browser tab, timing and lifecycle, geometry and rendering, and pitfalls. A measured behaviour claim names a device, an OS version and build, a context, and a date. A behaviour that a source describes but OpenPWA has not run is shown as Documented by <source>; not measured by OpenPWA. WebKit’s description of the cookie copy into new Home Screen web apps in Safari 17.2 is such a claim until a run measures it.

Assigning a claim to a layer

Apply the steps in order; the first “yes” decides.

1. Does the claim say what happens when the feature is used, or how its result depends on device, OS build, launch context, or state (values, precedence, defaults, timing, lifecycle, rendering, prompts)? Layer 3, Behaviour. Then ask whether OpenPWA ran it under the protocol (measured) or a source states it (documented).
2. Does it say that the feature can do a task and point to a runnable page, with no recorded run? Layer 2, Capability, demonstrated.
3. Does it say whether, where, or since when the feature is available (presence, version, flag, partial, a source-stated availability condition)? Layer 1, Support, documented. If the claim came from an OpenPWA run rather than a source, return to step 1: it is a behaviour record attached to the row.
4. Otherwise it is not a platform claim (advice, history, opinion). Label it as such or remove it.

Mixed statements (“supported since 16.4, and fires only in Home Screen apps”) are split into one claim per layer. A version number from a run (“works on 18.6.2”) is a behaviour claim about one build and does not establish version_added. A demo that fails on your device is your observation; it becomes an OpenPWA claim only when reproduced under the protocol.

Three classes of evidence

Documented. A source outside OpenPWA states the claim: a specification, vendor documentation, release notes, a vendor engineering post, a vendor bug tracker, or BCD. It is shown with the source and the date it was checked. BCD rows count as documented even where BCD’s own data came from automated collection, because OpenPWA did not observe them. Third-party notes (practitioner blogs, forums) are documented with lower confidence under the site’s existing high, medium, and low rule.

Demonstrated. A runnable OpenPWA demo exercises the capability. It is shown with the demo link and wording that invites you to try it on your own device. The label asserts that the demo exists and is designed to exercise the feature. It does not assert that the feature works on any particular device.

Measured. An OpenPWA run of a published protocol on a named real device recorded the result. Only runs published with confidence A or S carry this label.

What a measured result records

A run qualifies as measured when it records all of the following:

• an environment block: device name and model identifier, OS version and build, release channel, browser version, language and region, non-default flags, content blockers or VPN state, and the inspecting host if one was used;
• the date and time, with time zone, for the run and for each read;
• the context: tab, Home Screen app, or both; fresh state or not; first launch or relaunch;
• the protocol and harness: the test id, the published steps, and the harness version with a SHA-256 hash for each file the run loaded;
• the expected result and its source, written down before the step;
• the raw data: instrumented reads as files, screenshots, and recordings, each listed with a hash;
• a verdict (as expected, differs, inconclusive, or could not run) and a confidence label.

The iOS 18.6.2 study ran before these requirements were written and does not meet all of them. Its report and the paper state the gaps: raw reads carry tool-hash sidecars only from notebook entry N15 onwards; run-time hash snapshots exist for some harness files, not all; and the environment record lacks feature flags, VPN and iCloud Private Relay state, the identity of an installed Safari Web Extension, and the Settings language and region.

Confidence labels within measured

Label	Meaning	Published as measured?
A, instrumented read	The output of a published probe script, read from the page over a debugging protocol (for example the WebKit remote inspector) and stored verbatim as JSON. The value is the probe’s output, not a model’s interpretation.	Yes
S, screenshot or recording	A capture analysed afterwards: pixel values, video frames, capture times.	Yes
O, observer report	The operator’s verbal report only.	No. It stays in the notebook and may support an A or S result, never stand alone.

Labels combine: “A+S” means an instrumented read and a capture agree; “S+O” means a capture with the operator’s report as context. A result is published with its weakest essential component named.

How a measured result relates to a support row

Under this framework, a behaviour result is attached to the support row it bears on, with one of three relations:

• agrees: the measurement matches the row. After setAppBadge(3) the Home Screen icon showed a badge, and BCD lists the Badging API as supported.
• adds-condition: the row holds under a condition the source does not state. On iOS 18.6.2 window.pushManager also exists in a Safari tab, and in two Home Screen apps whose subpath-scoped worker held a push subscription, window.pushManager.getSubscription() returned null.
• differs: the measurement contradicts the source. notificationclick fired in Home Screen apps where BCD records it as unsupported (see the worked example).

The row keeps its source, status, version, and confidence. The framework asks that the relation and its condition be shown in the row, dated and linked to the run; a differs relation is shown in the row itself, not only in a footnote, and becomes a candidate for an upstream report (a BCD issue, a WebKit bug). When a later run differs from an earlier one, both stay visible, each with its build. In version 1.0 a measurement never changes a row’s status (principle P8).

Twelve principles

P1. Every claim has a layer and an evidence class. Readers can see both, in text, not only by colour.

P2. Classes are never upgraded by wording. “Tested”, “verified on device”, “in practice”, and their Chinese equivalents are reserved for published measured runs.

P3. Behaviour is versioned per OS build and context. A behaviour claim names device, OS version and build, context, and date. Different builds give different rows.

P4. Measurements are dated, reproducible, and come with raw data. Protocol, harness version and file hashes, environment block, raw reads, and captures are all published in sanitised form.

P5. Expectations are written before the step. The expected result and its source are recorded first. “Differs from expected” is a finding.

P6. Records beat reports. When an operator’s report conflicts with a read or a capture, the record is preferred and the conflict is logged as an anomaly. Observer-only results are never published as measured.

P7. Corrections are append-only. A wrong entry is corrected by a new dated entry that cites the old one. Retracted readings are listed as “do not cite” next to the result they were retracted from.

P8. Layer 3 conditions layer 1; it does not silently overwrite it. In version 1.0 a measurement never changes a support row’s status, version, or confidence. Any future rule that lets it do so is published as a new major version of this framework before it is applied.

P9. Scope is stated. Sample size (devices, builds, runs), what was not tested, and known threats to validity are part of every result.

P10. Novelty is checked, replication is labelled. A result that confirms a published description is labelled a replication and cites the description. Only results with no prior description found are called new, and the search is described.

P11. The observer protocol protects the record. When a person operates the device and an instrument reads it, the protocol rules are part of the method: define every term before using it, ask one question per message, confirm a step was done before asking for its result, leave the screen untouched for 20 to 30 seconds while an instrumented read runs, and do not write test markers through browser automation. In the iOS 18.6.2 study the safaridriver automation window had its own website data, separate from the owner’s Safari profile (notebook N06).

P12. Harness bytes and installed apps are frozen between comparable runs. A harness version is never moved, redirected, or edited once a run has used it, except to fix a privacy or security defect, which is published as a dated erratum with the old and new hashes. New versions get new, non-overlapping scopes. Test apps installed for a before-and-after study keep their scope and start URL until the after run is done.

Worked example with notificationclick on iOS

The same feature can sit in all three layers at once. The service worker notificationclick event on iOS shows each layer saying something different, and each label saying what stands behind it.

Support, documented by BCD

BCD records api.ServiceWorkerGlobalScope.notificationclick_event as version_added: false for Safari on iOS (ServiceWorkerGlobalScope.json, github.com). BCD updates are assisted by mdn-bcd-collector (github.com), a web service that runs generated feature-detection tests as pages loaded in a browser. A BCD contributor states in the issue below that the collector checks 'onnotificationclick' in self. That check was false in a worker controlled by a Safari tab on iOS 18.6.2, which is consistent with the “no” coming from a tab context; the study did not establish which collector run produced the row. A BCD issue opened in April 2024 (github.com) reports that the event does fire. On openpwa.net the row stays as BCD states it.

Capability, demonstrated by the push demo

The demo’s push permission page asks for notification permission and subscribes your browser to push, which are the steps that come before any notification tap. Try it on your device. It does not show a notification or fire notificationclick itself, so it demonstrates the push capability and says nothing about the click event on your phone.

Behaviour, measured in Home Screen apps

On an iPhone 16 Pro with iOS 18.6.2 (build 22G100), on 2026-10-20, notificationclick fired on six of six taps in a web app added to the Home Screen, through both addEventListener and the onnotificationclick property, and focus() on the existing window succeeded (confidence A). In a worker controlled by the Home Screen app, 'onnotificationclick' in self was true. Every tap had a live window. Taps with no live page, which comment 35 on WebKit bug 268797 (bugs.webkit.org) reports as failing, were not tested. The relation to the BCD row is differs. Under this framework it is shown in the row with the condition and the run date, and the row’s status does not change (P8). The result and its raw data are on the behaviour page notificationclick in Home Screen apps, with the study’s data package.

How to cite

Cite the version you read. For version 1.0:

OpenPWA Editorial Team. The OpenPWA Evidence Layers: support, capability and behaviour as separate claims about web platform features. Version 1.0, 2026-10-20. openpwa.net. https://openpwa.net/about/evidence/v1.0/

Short form in running text: “OpenPWA Evidence Layers v1.0”. To cite one definition, add its fragment, for example #p8 or #decision-procedure; the fragments are the same on every version page and in both languages.

The framework text is published under Creative Commons Attribution 4.0 International (creativecommons.org): you may copy, translate, adapt, or extend it, including commercially, on the licence’s terms, which include giving credit, linking to the licence, and indicating changes. Using the framework as a method needs no licence. A site or dataset adopts it by labelling each claim with its layer and evidence class, publishing the requirements for “measured” that it applies, and naming the version it follows; it need not use OpenPWA’s labels, data model, or harness.

Device steps performed by an OpenPWA operator on an iPhone 16 Pro. AI agent sessions read page state over USB, kept the notebook and drafted this text; automated gates and an AI fact-check checked it. No person reviews each entry.