Skip to content

Add host-tools/testrunner.sh to run the full test cycle in one step - #346

Open
iarif4u wants to merge 2 commits into
WordPress:masterfrom
iarif4u:add/host-tools-testrunner
Open

iarif4u wants to merge 2 commits into
WordPress:masterfrom
iarif4u:add/host-tools-testrunner

Conversation

@iarif4u

@iarif4u iarif4u commented Oct 2, 2026 •

Copy link
Copy Markdown

Fixes #260. Fixes #259 (creates the host-tools/ directory; @aialvi agreed to the handover in #260).

Purpose

Hosts say that running the runner is tedious (#257). This adds a ready-made script that runs the full cycle in one step, after the runner is set up.

What host-tools/testrunner.sh does

  1. Goes to the runner directory: the parent of host-tools/, or WPT_RUNNER_DIR.
  2. Stops with a clear message if .env does not exist.
  3. Stops if another run is still active (flock), so cron runs cannot overlap.
  4. Runs git pull --ff-only origin master. If the update fails, it warns and continues. Set WPT_SKIP_UPDATE=1 to skip the update.
  5. Loads .env and runs prepare.php, test.php, report.php and cleanup.php with WPT_PHP_EXECUTABLE, the same PHP that the runner uses.

Differences from the example in the issue and the README:

  • The report runs when tests fail. test.php exits with 1 when any test fails. A plain set -e script would then skip the report, so failures would never be reported.
  • Cleanup always runs: after a failed prepare, a failed test, or a stopped run (trap).
  • Exit status is 1 when any step fails, so cron mail or monitoring can see it.

The README "Script en Bash" section now points to this script. The systemd example is not changed.

Testing

I used stub prepare/test/report/cleanup.php files that exit 0 or 1:

Case Result
No .env Error, exit 1
All steps pass All 4 run, exit 0
test.php fails Report and cleanup still run, exit 1
prepare.php fails Test and report skipped, cleanup runs, exit 1
cleanup.php fails Exit 1
Second run while first is active "Another test run is still active", exit 1
SIGTERM during test Cleanup runs
git pull fails Warning, run continues

Contributed at WordCamp Contributor Day.

Updates the runner, loads .env, and runs prepare, test, report and
cleanup. The report still runs when tests fail, cleanup always runs,
and a lock stops overlapping runs from cron.

Fixes WordPress#260. Creates the host-tools directory from WordPress#259.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: iarif4u <iarif4u@git.wordpress.org>
Co-authored-by: ramonfincken <ramon-fincken@git.wordpress.org>
Co-authored-by: aialvi <aialvi@git.wordpress.org>
Co-authored-by: kittenkamala <amykamala@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@ramonfincken

Copy link
Copy Markdown

!!!!
Please refrain from using our hardcode paths @iarif4u

The examples hardcoded /home/wptestrunner. Use /path/to/phpunit-test-runner
instead, and note that the script finds the runner from its own location.
@iarif4u

iarif4u commented Oct 2, 2026

Copy link
Copy Markdown
Author

Thanks @ramonfincken, fair point. I took /home/wptestrunner from the existing README and the #260 example, but new scripts should not suggest one host's layout. Fixed in 41575c3: the examples now use /path/to/phpunit-test-runner, and the comment notes that the script finds the runner from its own location. I made the same change in #349 (387f77b).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make testrunner.sh in host-tools directory create host-tools directory in root folder

2 participants