Run Device Lab tests on iPhone and iPad

A Device Lab run on an iPhone follows the protocol published with the iOS 18.6.2 baseline of 2026-10-20, with a hash check taken from the reproducibility notes in the study report (section 13). Take a hash snapshot before the first tap and check it again after the last step. You need the iPhone, Safari, about an hour, and for the tests marked -WI a Mac with Safari and a USB cable (Set up Web Inspector).

Before the first step

Before opening a test page, take a hash snapshot and fill in the environment block.

Take the pre-run hash snapshot

Harness v1 lists the SHA-256 of each of its files as served in SHA256SUMS.txt. Hash what the server sends you and compare it with that list, so you know whether the server is still sending the frozen files. This checks the server, not the copy a page already open on the phone is running (see the next section). On a Mac or Linux shell:

curl -s https://openpwa.net/lab/harness/v1/SHA256SUMS.txt | grep -v '^#' |
while read -r expected file; do
  actual=$(curl -s --compressed "https://openpwa.net/${file%index.html}" | shasum -a 256 | cut -d' ' -f1)
  if [ "$actual" = "$expected" ]; then echo "ok       $file"; else echo "CHANGED  $file"; fi
done | tee pre-run-snapshot.txt

Every line should read ok. A CHANGED line means the served file is not the frozen one; record it in your notes and check the harness v1 page for an erratum. If you will also use the demo, hash the /demo/ pages you load into the same file; the demo is not frozen, so keep its hashes with your notes.

Know that served files can change during a run

The report asks rerunners to compare served files with the recorded hashes because the lab can change. During the 2026-10-20 run a deploy at about 08:28 UTC changed lab.js, and pages that were already open kept running the cached earlier copy, which is served with max-age=14400 (section 10, harness drift). A changed hash on the server therefore does not show which copy a page ran. Repeat the snapshot at the end, and if a line changed, record the time you saw it and treat later reads as possibly from either copy.

Fill in the environment block

Copy the environment block into your notes. Fill it in for each device and session. For the model number, open Settings > General > About and tap the part number to the right of Model Number (Find the model number of your iPhone, iPad, or iPod touch, apple.com); Apple lists the models in Identify your iPhone model (apple.com).

- date / time / time zone:
- device: <marketing name> (<model number>)
- OS: iOS <version> (<build>)  [Settings > General > About > iOS Version]
- release channel: release | public beta | developer beta | RC
- Safari: <version>; user agent from T0-WI: <navigator.userAgent>
- device language / region:
- Safari feature flags changed from default: none | <list>
- content blockers / VPN / iCloud Private Relay: <on or off>
- Mac for Web Inspector: <macOS version, Safari version> | not used

Leave Safari’s feature flags as they are (Settings > Apps > Safari > Advanced > Feature Flags). If any is already non-default, list it in the block.

Run the tests

Work through the test list in its order, so that nothing is installed twice. Each test page states its expectation and the source of that expectation; read it before you tap.

1. Reset where the test asks for fresh state. Touch and hold the test’s Home Screen icon, choose Remove App, then Delete. Clear the site’s data in Settings > Apps > Safari > Advanced > Website Data (the protocol gives Settings > Safari > Advanced for earlier releases). Apple documents the Website Data control in Delete your Safari history, cache, and cookies on iPhone (apple.com).
2. Open the harness page in a Safari tab, for example the lab app at /lab/device-tests/ or the T7 permission app at /lab/device-tests/t7/perm/. Read the panel, or read the page from the Mac for a -WI test.
3. Tap Share, then Add to Home Screen, and keep the prefilled name unless the test says otherwise. Take a screenshot of the add sheet when the test asks for one.
4. Launch the app from its icon, not from Safari. Repeat the read. In the 2026-10-20 run a Home Screen web app with a manifest ran in its own process, while a page with no manifest and no icon opened from the Home Screen in Safari (findings, F1), so tab and app results are recorded separately.
5. Record the result with the protocol’s fields: result (as-expected, differs, could-not-run with the reason, incomplete with the reason), what you observed in one or two sentences with on-screen text quoted in the UI language, the values the test asks for, the screenshot file names and the time taken.
6. Stop at five minutes. A test that runs longer is recorded as incomplete with the reason, and the run moves on.

Ask for one thing at a time and name the screen element exactly (“the strip with the time, signal and battery”). In the baseline run, several verbal reports answered a different question than the one asked; the reads and screenshots caught them, and the notebook records each one.

Evidence and privacy

Screenshots follow the naming and redaction guide. Before you keep or send anything:

• record only a push subscription’s endpoint host, omitting the endpoint path and keys (lab.js and the T7 permission app print the host only, for example web.push.apple.com);
• crop or blank other apps’ notifications, contacts, the carrier name, the location and other sites’ tab titles;
• keep identifiers out: no serial number, UDID, Apple Account or phone number.

Close the run

Run the hash snapshot again and compare it with the first one; note any changed line as described above. Record which tests ran, which differed from expectations, and which could not run. To share your results with OpenPWA, follow Send your results.

iPad

No iPad result is published, and iPadOS was outside the 2026-10-20 run. The protocol notes that iPadOS differs in the Add to Home Screen UI and in multitasking windows, plans an iPad subset of tests in the same week as the iPhone run, and writes the OS in the citation as iPadOS <version>. Fill in a separate environment block for the iPad.

Device steps performed by an OpenPWA operator on an iPhone 16 Pro. AI agent sessions read page state over USB, kept the notebook and drafted this text; automated gates and an AI fact-check checked it. No person reviews each entry.