OpenPWA Device Lab
Published
The OpenPWA Device Lab measures what web apps do on a real device, under a named OS build, in a stated context (Safari tab or Home Screen app). The first run’s data package publishes the protocol, the notebook, the raw reads, the probe scripts with their hashes and the screenshots, and assigns each finding an evidence label (A/S/O). Someone with a comparable device can repeat a step and compare the result with the published reads.
Where the Lab sits in the Evidence Layers
Section titled “Where the Lab sits in the Evidence Layers”Support (layer 1) describes whether a feature exists and since which version. It comes from browser-compat-data and specifications, along with vendor release notes. A live demo runs an API on your device to demonstrate capability (layer 2). To measure behaviour (layer 3), an operator runs a published protocol on a named device and records what happened. The OpenPWA Evidence Layers define these terms.
A support table can say that Safari on iOS 16.4 added Web Push for Home Screen web apps (Web Push for Web Apps on iOS and iPadOS, webkit.org). It cannot say whether notificationclick reaches a worker that a Safari tab controls, which colour the status-bar strip takes when the page and the manifest disagree, or whether a cookie set in Safari is still there when the Home Screen app first opens. Those are behaviour questions, and each one gets a behaviour page with its own results.
The published protocol sets how a device result reaches the compatibility data: it corroborates a row and leaves its source intact, whether that is a vendor document, BCD entry or specification, and a result that contradicts those sources does not overwrite the row; it is listed as a discrepancy and becomes a candidate for a dated note on the page.
Devices behind the results
Section titled “Devices behind the results”The published results come from one device and one operator.
| Device | OS and build | Channel | Runs |
|---|---|---|---|
| iPhone 16 Pro (iPhone17,1) | iOS 18.6.2 (22G100), Safari 18.6 | release | 2026-10-20 |
One OpenPWA operator performs the device steps. AI agent sessions on a Mac read page state over USB through the WebKit remote inspector. They also keep the notebook and draft the pages. An iPad may be added later for a small subset of tests (multitasking windows and the Add to Home Screen sheet); no iPad result exists. Each result names the device and build tested.
Latest run
Section titled “Latest run”Latest run: 2026-10-20, iPhone 16 Pro, iOS 18.6.2 (22G100), 13 behaviours
The study report treats the 2026-10-20 run as the baseline for an iOS 27 rerun and notes that the phone will be upgraded to iOS 27, after which these measurements cannot be repeated on this phone. The roadmap lists the planned reruns.
The demo and what it proves
Section titled “The demo and what it proves”The demo at /demo/ runs individual web app APIs on your own device. It demonstrates layer 2 capability by showing what your device does when you try it. That is your observation, not a measurement by OpenPWA. The 2026-10-20 run used the demo pages as test subjects alongside the lab’s own test pages, and the behaviour pages publish its reads with their context and time.
| Demo page | What it lets you try | Related behaviour pages |
|---|---|---|
| Push | permission request, subscription, a local notification | notification-permission-prompt, notificationclick-in-home-screen-apps, declarative-web-push-subscription-scope |
| Badging | navigator.setAppBadge() and clearAppBadge() |
app-badge-on-home-screen |
| Install | whether the page sees an install prompt API | install-apis-on-ios |
| Share | navigator.share() and canShare() |
no behaviour page |
| File handling | launchQueue and declared file handlers |
none measured; listed on the roadmap |
Sections of the Lab
Section titled “Sections of the Lab”- Behaviours: one page per measured behaviour, with the question, the steps, the results per OS build and what each result backs.
- Tests: the protocol index, with every test id, its duration, its harness pages and the behaviours it feeds.
- Roadmap: planned tests and reruns, with their status and reason.
- Run guides: how to run the tests on an iPhone, set up Web Inspector, name screenshots and send results.
- Field notes: the studies written from Lab runs.
- Data:
results.jsonand the study data packages, along with the harness hash lists. - Harness: harness versions and the frozen v1 file list with its hashes.
- Methodology: the protocol and the A and S confidence labels, along with threats to validity.
- Cite: citation strings and the fragment rule for result rows.
Device steps performed by an OpenPWA operator on an iPhone 16 Pro. AI agent sessions read page state over USB, kept the notebook and drafted this text; automated gates and an AI fact-check checked it. No person reviews each entry.