Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .claude/reflections/2026-09-09-ui-test-tcc-evidence-on-clones.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# 2026-09-09 - TCC Evidence Read From the Wrong Simulator

## What happened

`AccessKeyBackupSmokeTests.testAccessKeyBackup_saveToPhotos` failed in the full `AllTargets` plan run (`Expected "Allow" Button to be hittable within 30.0s`) but passed alone. The base "iPhone 17" simulator's `TCC.db` still held `kTCCServicePhotosAdd = 2` from the earlier isolated run, unchanged by the plan run, which read as "the reset in `setUp` did not clear the add-only Photos row". Two hypotheses followed: `resetAuthorizationStatus(for: .photos)` skips `kTCCServicePhotosAdd`, or the reset is ignored while the previous test's app instance is still alive.

## The misstep

The row was unchanged because the plan run never touched that device. `AllTargets.xctestplan` marks `FlipcashTests` parallelizable, so `xcodebuild test -testPlan AllTargets` boots `Clone 1 of iPhone 17` and runs every target there, UI tests included (`CoreSimulator.log`: "Boot requested: Clone 1 of iPhone 17"). The clone is deleted after the run, taking its `TCC.db` and `tccd` log with it. Both hypotheses were built on evidence from a simulator the failing test never ran on.

Measured on the base device with a `tccd` log stream and a `TCC.db` poller, the reset deleted the `kTCCServicePhotosAdd` row and the alert appeared in every configuration tried: the test alone, the whole class with the app pre-launched, and the test right after a fresh install where `tccd` logs `bundleRecordWithBundleIdentifier failed ... -10814` and still publishes the delete. The one measurable difference was time-to-alert: about 6s on the base device versus 12.5s on a clone running next to a single parallel unit-test target.

## Resolution

Harness-only change: `BaseUITestCase.setUp` terminates the app before resetting permissions, and `allowSystemAlertIfNeeded(orUntil:)` waits up to 60s for either the springboard "Allow" button or the app moving on without one, tapping Allow only when it shows. The exact failing run was not reproduced; the fix covers the slow-alert path that was measured and the stale-grant path that was hypothesized.

## Lesson

Before reading simulator state as evidence, confirm which simulator the test ran on. With parallel testing on, look for "Clone N of <device>" in `~/Library/Logs/CoreSimulator/CoreSimulator.log` and the xcodebuild log; if the run used a clone, the base device's `TCC.db`, containers, and logs say nothing about it. Capture from inside the run instead: `xcrun simctl spawn <udid> log stream --predicate 'process == "tccd"'` on the base device for `-parallel-testing-enabled NO` runs, and `xcresulttool get test-results activities` for tap timings on any run.
1 change: 1 addition & 0 deletions .claude/reflections/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ A log of situations where things got out of hand. Each entry documents the issue
- [2026-03-02 - SendCashOperation Verified State Regression](2026-03-02-send-cash-verified-state-regression.md): Path 2 (transfer) called `getVerifiedState()` from cache instead of reusing the state already resolved in Path 1 (message send). New currencies weren't in cache → `missingVerifiedState` → bill instantly disappeared.
- [2026-05-09 - Camera Isolation Masked by @preconcurrency](2026-05-09-camera-isolation-masked-by-preconcurrency.md): Step 2 of the package restructure was tagged "Risk: low" because stripping `@MainActor` while adding `defaultIsolation` is semantically a no-op. The required tools-version bump to 6.2 enabled a runtime isolation check that caught a years-old bug `@preconcurrency import AVKit` had been hiding — instant crash on camera start.
- [2026-07-11 - Regression Test at the Wrong Layer](2026-07-11-regression-test-wrong-layer.md): PR #472's regression test asserted diff shape (the fix's layer) instead of the crash path; the same UIKit crash shipped again two days later as 6a522ee. Crash repros must drive a windowed view, read live cells, and be observed failing first.
- [2026-09-09 - TCC Evidence Read From the Wrong Simulator](2026-09-09-ui-test-tcc-evidence-on-clones.md): Diagnosed a Photos-permission UI test flake from the base simulator's `TCC.db`, but the `AllTargets` plan runs UI tests on a throwaway clone; measured on the device the test actually used, the reset worked and the alert was just slower.
12 changes: 7 additions & 5 deletions FlipcashUITests/Smoke/AccessKeyBackupSmokeTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -87,16 +87,18 @@ final class AccessKeyBackupSmokeTests: BaseUITestCase {
dialog.buttons["View Access Key"].tap()

// Tap "Save to Photos"
waitAndTap(app.buttons["Save to Photos"])
let saveButton = app.buttons["Save to Photos"]
waitAndTap(saveButton)

// Allow Photos access via the system permission dialog
let springboard = XCUIApplication(bundleIdentifier: "com.apple.springboard")
waitUntilHittableAndTap(springboard.buttons["Allow"])
// Allow Photos access via the system permission dialog. The alert can
// take well over ten seconds on a freshly cloned simulator under a full
// plan run, and it never appears at all when the reset in `setUp` left
// access granted, so wait for whichever comes first.
allowSystemAlertIfNeeded { !saveButton.exists }

// The button should transition to success state (checkmark).
// We verify the button is no longer showing the original title,
// indicating it transitioned to .success state.
let saveButton = app.buttons["Save to Photos"]
let disappeared = saveButton.waitForNonExistence(timeout: 10)
XCTAssertTrue(
disappeared,
Expand Down
27 changes: 27 additions & 0 deletions FlipcashUITests/Support/BaseUITestCase.swift
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,13 @@ class BaseUITestCase: XCTestCase {
app.launchArguments.append("--beta-flags=\(enabledBetaFlags.joined(separator: ","))")
}

// The previous test's app instance can still be running here. The reset
// API only promises that a running app "might" be terminated during the
// reset, so stop it first and reset against a known-dead process.
if !resetPermissions.isEmpty {
app.terminate()
}

for permission in resetPermissions {
app.resetAuthorizationStatus(for: permission)
}
Expand Down Expand Up @@ -268,6 +275,26 @@ class BaseUITestCase: XCTestCase {
waitUntilHittableAndTap(springboard.buttons["Allow"])
}

/// Taps the springboard "Allow" alert if it shows up, or returns as soon as
/// `settled` reports the app moved on without one; fails when neither
/// happens within `timeout`.
func allowSystemAlertIfNeeded(timeout: TimeInterval = 60, orUntil settled: () -> Bool) {
let springboard = XCUIApplication(bundleIdentifier: "com.apple.springboard")
let allowButton = springboard.buttons["Allow"]
let deadline = Date().addingTimeInterval(timeout)

while Date() < deadline {
if allowButton.exists, allowButton.isHittable {
allowButton.tap()
return
}
if settled() { return }
Thread.sleep(forTimeInterval: 0.25)
}

XCTFail("Neither the system \"Allow\" alert nor the expected app state appeared within \(Int(timeout))s")
}

/// Everything legible on screen, for failure messages.
func visibleText() -> String {
app.staticTexts.allElementsBoundByIndex
Expand Down
Loading