Device Lab test protocol

A test is a short, numbered procedure (T0, T1, and so on) that runs on one device against published harness pages. Results that answer the same question are collected on a behaviour page. A suffix marks a variant: -WI needs a Mac with Web Inspector attached, T4b is the status-bar style variant of T4, and T5b repeats the Add to Home Screen check from a third-party browser.

How a test is structured

Before running a step, record its expected result and the source of that expectation. Then record what the screen and page showed. A difference from the expectation counts as a finding. The protocol keeps each test under five minutes and a whole checklist under an hour; a test that runs long is stopped and recorded as incomplete with the reason.

Fresh state

A test that says “fresh state” starts with no trace of an earlier run. The protocol gives the steps for an iPhone: remove the Home Screen icon (touch and hold, Remove App, Delete) before adding it again, and clear the site’s data in Settings > Apps > Safari > Advanced > Website Data (older releases put it under Settings > Safari > Advanced). Apple describes the Website Data control in Delete your Safari history, cache, and cookies on iPhone (apple.com). A test that depends on earlier state, such as a cookie written in Safari before Add to Home Screen, says which state it needs.

Tab and Home Screen app

A page in a Safari tab and the same page launched from its Home Screen icon can behave differently. In the 2026-10-20 run, a Home Screen web app with a manifest ran in its own process, apart from Safari, with its own storage, while a page with no manifest and no icon opened from the Home Screen in Safari’s process with Safari’s storage (findings, F1). Each read is labelled with its context, tab or app, and a result says which context it covers. Harness versions are listed on Harness.

Recording and confidence

Record each test with the same fields: result (as-expected, differs, could-not-run with the reason, or incomplete with the reason), what was observed in one or two factual sentences with on-screen text quoted in the UI language, the key values, the screenshot file names and the time taken. Each result carries a confidence label, as defined in the study report (section 5.4): A for an agent-measured instrumented read stored in raw/ (a Web Inspector read, or a WebDriver read for some tab values), S for a screenshot or recording captured by the operator and analysed afterwards, and O for the operator’s verbal report alone. A combination such as A+S means independent sources agree.

Test list

Tests are indexed by id, duration, platform, harness page for each version, and related behaviours. Harness v1 pages sit under /lab/device-tests/ and are noindex; open them on the device under test.

Protocol index

Test	Name	Time	Platforms	Harness pages	Behaviours
T0	Web Inspector setup and user agent	5 min	iOS	Harness v1: PWA Lab	None.
T1	Page with no manifest added to the Home Screen	5 min	iOS	Harness v1: t1/no-manifest	Home Screen launch without a manifest
T1-WI	Display-mode values of the manifest-less icon in Web Inspector	2 min	iOS	Harness v1: t1/no-manifest	Home Screen launch without a manifest
T2	Install the demo and launch it	4 min	iOS	Demo page: /demo/	Standalone viewport and display mode
T3	Home Screen icons	4 min	iOS	Harness v1: PWA Lab, t3/manifest-only	Home Screen icon selection
T4	Status bar area in installed apps	6 min	iOS	Harness v1: t4/baseline, t4/fixed, t4/sticky, t4/safe-area, t4/threshold, t4/white-top, t4/theme-red, t4/bg-green, t4/dynamic	Status bar colour source
T4-18c	Manifest theme_color against meta theme-color (apps M1, M2, M3)	part of T4	iOS	Harness v1: t4/theme-m1, t4/theme-m2, t4/theme-m3	Status bar colour source
T4b	apple-mobile-web-app-status-bar-style apps	5 min	iOS	Harness v1: t4/sbs-default, t4/sbs-black, t4/sbs-black-translucent	Black-translucent status bar geometry
T5	Display mode in a tab and in the app	3 min	iOS	Harness v1: PWA Lab Demo page: /demo/	Standalone viewport and display mode
T5b	Add to Home Screen from a third-party browser	3 min	iOS	Demo page: /demo/	None.
T6	Storage and cookies, Safari and the Home Screen app	5 min	iOS	Harness v1: PWA Lab, t3/manifest-only Demo page: /demo/	Storage isolation and cookie copy
T7	Notification permission and push subscription	5 min	iOS	Harness v1: PWA Lab, t7/perm Demo page: /demo/push/	Notification permission prompt, Declarative Web Push and subscription scope
T8	notificationclick: exposure, handlers and a notification tap	5 min	iOS	Harness v1: PWA Lab	notificationclick in Home Screen apps, Worker event handlers in a tab and in an app, Page lifetime after a swipe-away
T9	Install APIs absent on iOS	2 min	iOS	Harness v1: PWA Lab Demo page: /demo/	Install APIs on iOS
T10	Declarative Web Push detection	2 min	iOS	Harness v1: PWA Lab Demo page: /demo/push/	Declarative Web Push and subscription scope
T11	Badging in the installed app	3 min	iOS	Demo page: /demo/badging/	App badge on the Home Screen icon

Before and after a run

Compare the files you load with the published hashes. Harness v1 lists the SHA-256 of each file as served in /lab/harness/v1/SHA256SUMS.txt. The study report asks rerunners to compare served files with recorded hashes because the lab can change, and it records one change during the 2026-10-20 run: a deploy at about 08:28 UTC changed lab.js, and pages that were already open kept the cached earlier copy (section 10, harness drift). The iPhone and iPad guide has a script for the comparison.

Device steps performed by an OpenPWA operator on an iPhone 16 Pro. AI agent sessions read page state over USB, kept the notebook and drafted this text; automated gates and an AI fact-check checked it. No person reviews each entry.