# OpenPWA device-test protocol Owner decision 2026-10-20: the owner runs real-device checks on their own iPhone / iPad (iOS 26/27), about 1 hour per Safari release. This file says how to run and record them and how results reach the site. Each release gets its own checklist in this folder (`YYYY-MM-.md`). The first one is `2026-10-ios27.md`. ## Why this exists openpwa.net is a derived source today. Its methodology counts editor-run tests as `low` confidence, and it has no first-hand observations. Gatekeepers (WebKit Bugzilla, BCD, spec threads, Wikipedia) want primary evidence. A dated, reproducible observation on a named device is the smallest thing that changes that. The checks also stop us repeating unverified third-party claims (the iOS 27 status-bar blur and the 27.2 icon fallback, for example). ## Ground rules 1. **One test, under 5 minutes.** If a test runs long, stop and record "incomplete" plus why. A whole checklist stays under 60 minutes, and optional tests are marked. 2. **Record what you saw, not what you expected.** Each test lists the expected result (from sources) so you know what to look for. The observed field is filled from the screen. "Differs from expected" is a finding, not a mistake. 3. **Fresh state where the test says so.** Remove the Home Screen icon (long-press → Remove App → Delete) before re-adding it. To clear website data: Settings → Apps → Safari → Advanced → Website Data (iOS 18+ path; older: Settings → Safari → Advanced → Website Data). 4. **No personal data in evidence.** Crop or skip screenshots that show other apps' notifications, contacts, the carrier name or the location. Push subscriptions: record the endpoint **host only** (e.g. `web.push.apple.com`), never the full endpoint or keys. 5. **Don't change Safari feature flags** (Settings → Apps → Safari → Advanced → Feature Flags). If any flag is already non-default, record it in the environment block. 6. **Nothing is posted from here.** Results feed datasets and drafts through a working session (see "How results flow" below), never directly to a public thread. ## Environment block (fill once per device per session) ``` - date / time / timezone: 2026-10-__ __:__ CST - tester: owner - device: (), Settings → General → About → Model Name / Model Number (tap to toggle) - OS: iOS / iPadOS (). Settings → General → About → iOS Version. Write "build n/a" if it isn't shown. - release channel: release | public beta | developer beta | RC - Safari: same as the OS version (Safari ships with iOS). UA string from T0-WI: - device language / region: - Safari feature flags changed from default: none | - Content blockers / VPN / Private Relay: - Mac for Web Inspector: | not used ``` ## Screenshots - Take: side button + volume up. Transfer by AirDrop to the Mac. - Store originals outside version control (a private, git-ignored working folder). - Name: `____.png`, e.g. `2026-10-24_iphone16pro_ios27.0.1_T4_1.png`. Use the same slug everywhere for the same device. - Reference screenshots by file name in the test's `screenshots:` field. ## Recording a test (fields used in every checklist) ``` - result: as-expected | differs | could-not-run () | incomplete () - observed: - values: - screenshots: - time taken: - notes: ``` When a whole checklist is done, fill its **Summary** table and set `status: complete`. ## How results flow into the site and drafts Results never change the site directly. A working session (not the daily routine) picks them up: 1. **Citation string** (use exactly this form everywhere): `OpenPWA device test, , , iOS ()` e.g. `OpenPWA device test, 2026-10-24, iPhone 16 Pro, iOS 27.0.1 (24A…)`. On iPad: `iPadOS `. 2. **Datasets** (`site/src/data/compat/*.json`): if a row's claim was observed, add the test as a supporting source on that row with the citation string. Under the current methodology an editor-run test alone is `low`; it corroborates and does not replace a vendor, BCD or spec source. `verified_at` changes only on rows the test actually covered. If the observation **contradicts** BCD or vendor docs, don't overwrite the row. Add it to the checklist's "Discrepancies" section; it becomes a candidate for a BCD issue or WebKit bug (both parked until the owner approves accounts) and for a dated note on the page. 3. **Reference pages** (e.g. `/reference/platforms/ios-by-version/`, `/reference/installation/ios-add-to-home-screen/`): a dated "Observed on …" sentence with the citation string, only for behaviour the sources don't already state. The page must go through the normal gates and the deploy rule (from `main` only). 4. (Internal editorial-workflow steps omitted from the public copy.) Methodology follow-up (owner decision, not yet made): adding a visible "tested" evidence class to `/about/methodology/` would let these results raise confidence instead of only corroborating. That is Phase 2 in PLAN.md and is out of scope here. ## Cadence - Each Safari / iOS point release (x.y): rerun the core tests (T1, T2, T4, T5, T6, T7, T9, T10 in the first checklist), about 1 hour. Upcoming: iOS 27.2 (public ~10-26 to 10-27; RC 24B115 on 10-19), iOS 27.3 (reported for December). - x.y.z bug-fix releases: rerun only tests whose subject the release notes mention. - iPad: run the iPad subset from the checklist in the same week if time allows (iPadOS differs for Add to Home Screen UI and multitasking windows). ## Web Inspector (needed by tests marked -WI) One-time setup, about 5 minutes; it needs the Mac and a USB cable for the first pairing: 1. iPhone: Settings → Apps → Safari → Advanced → **Web Inspector** on. 2. Mac Safari: Settings → Advanced → **Show features for web developers** (shows the Develop menu). 3. Connect the iPhone by cable, unlock it and tap Trust if asked. 4. Mac Safari → Develop → ``: open Safari tabs and running Home Screen web apps are listed; service workers may be listed separately. Click one to open its console. If a target isn't listed, record `could-not-run (not inspectable)` for that sub-step and move on.