Skip to content

Make Espresso waits say what they are waiting for - #64

Merged
parawanderer merged 1 commit into
mainfrom
fix/instrumented-test-flakes
Aug 12, 2026
Merged

parawanderer merged 1 commit into
mainfrom
fix/instrumented-test-flakes

Conversation

@parawanderer

Copy link
Copy Markdown
Owner

Fixes the instrumented job failing on main after #62.

Three CI failures in a row, all the same shape, none of them product bugs.

What was wrong

The waiting helper was copied into three test classes. A flake was found and
fixed in one; the identical code in the others kept failing. That is literally
what happened — the keyboard-over-the-button fix went into one login test, and
CI then failed on its sibling.

The retry was also wrong in a way copying could only spread. Retrying until
nothing throws cannot tell two opposite situations apart:

  • the tap missed — the soft keyboard was still over the button — and must be
    retried
  • the tap worked, and what it started has already torn the screen down, so
    Espresso throws NoActivityResumedException from the very same call

Retrying the second turns one success into fifty failures. That is what broke
the post-merge run: theConfiguredServerIsWhatTheSignInActuallyUses signs in
immediately, so the click that worked also ended the activity.

The fix

One Eventually class, two methods, and picking the wrong one is the bug:

  • Eventually.check(...) — assertions and waiting for views. Espresso waits for
    the main thread and nothing else, while every step here runs on an Rx
    scheduler.
  • Eventually.perform(what, tookEffect, action) — anything that changes the
    screen. The caller states what "it worked" means, usually a fake having been
    called, and it stops the moment that holds regardless of what was thrown.
Eventually.perform("the sign in button", () -> apple.timesCalled("login") > 0,
        () -> onView(withId(R.id.login_button_main)).perform(click()));

Documented in AGENTS.md, along with two rules from the same failures: one
ViewAction per perform when the action may finish the flow, and a GONE
view still matches withId.

Verification

Two consecutive full runs on the managed device, 106 tests, no failures. Since
the whole point is that this used to pass locally and fail in CI, the run on
this PR is the one that counts.

🤖 Generated with Claude Code

Three CI failures in a row, all the same shape and none of them product bugs.

The waiting helper was copied into three test classes. Fixing a flake in one
left the identical code in the others failing, which is exactly what happened:
the keyboard-over-the-button fix went into one login test and CI then failed on
its sibling. It is one class now.

The retry itself was also wrong. Retrying until nothing throws cannot tell "the
tap missed, try again" from "the tap worked and has already torn the screen
down" - Espresso reports the second as NoActivityResumedException from the same
call, so the retry turned one success into fifty failures. That is what broke
the post-merge run on main.

Eventually.perform takes a condition saying what "it worked" means, usually a
fake having been called, and stops the moment that holds regardless of what was
thrown. Eventually.check keeps the old behaviour for assertions, where retrying
until it holds is right.

Documented in AGENTS.md with the two rules that came out of the same failures:
one ViewAction per perform when the action may finish the flow, and a GONE view
still matches withId.

Two consecutive full runs green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@parawanderer
parawanderer merged commit d04a600 into main Aug 12, 2026
3 checks passed

This branch was previously deployed

1 inactive deployment
Android Build — 2aad24b9 Deployed Aug 12, 2026 by parawanderer via build #59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant