rasuvaeff/property-testing-phpunit

PHPUnit adapter for the property-testing engine: a PropertyTesting trait with a fluent forAll()->check() API over the framework-agnostic runner. Generate hundreds of random inputs per test, find the failing one, and shrink it to a minimal counterexample you can actually read — inside an ordinary PHPUnit TestCase.

Using an AI coding assistant? llms.txt contains a compact API reference you can share with the model.

Part of the property-testing family

Package	Use it when
`rasuvaeff/property-testing-core`	You drive the engine yourself: a custom harness, CI guard, CLI checker, or another framework adapter
`rasuvaeff/property-testing-testo`	You test with Testo — drop-in replacement for the frozen `rasuvaeff/property-testing` with the same `#[Property]` attribute
`rasuvaeff/property-testing-phpunit` (this package)	You test with PHPUnit — the `PropertyTesting` trait with the fluent `forAll()->check()` API

Requirements

PHP 8.3+
phpunit/phpunit ^11.5 || ^12.0 || ^13.0
rasuvaeff/property-testing-core ^1.0

PHPUnit 13 requires PHP 8.4.1 or newer. On PHP 8.3, Composer resolves a compatible PHPUnit 11 or 12 release.

Installation

composer require --dev rasuvaeff/property-testing-phpunit

No configuration is needed: mix the trait into a TestCase and call forAll() from a test method.

Usage

Map each property-body parameter to a generator, configure the run with the fluent chain, and hand the property to check(). The engine generates random arguments, runs the closure the configured number of times, and on the first failure shrinks the counterexample to a minimal one:

use PHPUnit\Framework\TestCase;
use Rasuvaeff\PropertyTesting\Gen;
use Rasuvaeff\PropertyTesting\PhpUnit\PropertyTesting;

final class SortPropertyTest extends TestCase
{
    use PropertyTesting;

    public function testSortIsIdempotent(): void
    {
        $this->forAll(['values' => Gen::arrayOf(Gen::int())])
            ->runs(300)
            ->check(static function (array $values): void {
                sort($values);
                $once = $values;
                sort($values);

                self::assertSame($once, $values);
            });
    }
}

The closure's parameter names select the generators, exactly like a #[Property] method signature does under the Testo adapter. On failure the test fails with the engine's message:

Property falsified after 12 successful run(s); seed=7382910
  Original: values=[20, 82, 44, 43, 29, 47, 29, 0, … +4 more]
  Shrunk:   values=[0, 0, 0, 0, 0, 0] (7 shrink step(s), 29 trial(s))
  Changed:  values=[20, 82, 44, …] -> [0, 0, 0, 0, 0, 0]

Reproduce the exact run by pinning the reported seed: ->seed(7382910).

The fluent chain

forAll() returns a PropertyCheck; every setter returns it for chaining, and check() runs the property.

Method	Meaning
`id(string)`	Names the property, replacing the id derived from the calling method. Keys the corpus and the events, and is the display name — required when `forAll()` runs inside a closure
`runs(int)`	Successful checks to complete (default 100). Discarded runs do not count
`seed(int)`	Pins the random phase for reproduction. Also disables corpus replay — the pinned run wins
`maxShrinks(int)`	Cap on accepted shrink steps; `0` disables shrinking
`maxDiscards(int)`	Cap for the discard budget and the skip budget. Left unset the two differ: `runs * 10` for discards, `runs` for environmental skips
`timeoutMs(int)`	Wall-clock deadline for a single run — exceeding it fails with `DeadlineExceededException`
`budgetMs(int)`	Wall-clock budget for the whole random phase — running out fails with `TimeBudgetExceededException`
`examples(array)`	Fixed positional argument tuples run before the random phase; a failing example short-circuits, unshrunk
`listeners(...)`	`PropertyListener` observers of the engine's lifecycle events
`shrink(ShrinkMode)`	How hard to minimise: `Full` (default), `Off` (report the input as generated), `Bounded` with a budget
`shrinkBudgetMs(int)`	Wall-clock budget for the descent — the one knob that costs determinism, since how far it gets depends on how long the body takes
`phases(array)`	Stages to perform (`Phase::Examples`, `Corpus`, `Random`, `Shrink`) — a subset trades coverage for time on purpose
`derandomize(bool)`	Derives an unset seed from the property id instead of drawing one; an explicit `seed()` still wins
`path(string)`	Replays a recorded shrink descent instead of searching for it; needs the seed that produced it
`edgeCases(EdgeCases)`	`None` turns off the numeric boundary bias — for a property the edges only cost runs
`auto(bool = true)`	Derives generators from the closure's signature for every parameter the `forAll()` map does not cover; the map becomes partial overrides. Off by default, and stays off
`throws(string)`	The exception class every trial must throw — a trial that throws it passes, one that does not (or throws another class) fails and shrinks. The property-level replacement for `expectException()`, which never sees a throw from inside the body
`exhaustive(bool = true)`	Walk the whole parameter domain instead of sampling it when every generator is `Enumerable` and the product fits the budget; otherwise the phase samples and a warning says why. `runs()` is ignored when it walks — see the core README
`exhaustiveBudget(int)`	The largest domain `exhaustive()` walks (default 10 000)
`flakyReplays(int)`	Re-executions of the minimised counterexample (default 2); one that passes marks it flaky, with a `Flaky:` line in the failure. `0` disables — see flaky detection
`searchRuns(int)`	Bodies the targeted search may execute after the random phase, for a body that calls `Target::maximize()`/`minimize()` (default 0 — no search) — see targeted search

Every setter validates its argument at the call, with the property's name in the message: runs(0) throws Property "testSortIsIdempotent": runs must be greater than or equal to 1 (the same for maxShrinks/maxDiscards/flakyReplays/searchRuns below 0 and for timeoutMs/budgetMs/shrinkBudgetMs/exhaustiveBudget below 1). A path() without a seed() — in either order — or a PROPERTY_PATH without a seed is refused by check() the same way. The forAll() map is checked by check() too, before the engine runs: a value that is not an ArbitraryInterface throws Property "…": forAll() expects array<string, ArbitraryInterface>, got int for key "x", and a key that is not a parameter of the closure throws Property "…": forAll() covers "vlaue", which is not a parameter of the property — with or without auto(), because a typoed entry would otherwise run green in whatever domain the real parameter has.

Auto-derived generators (`auto()`)

When the closure's parameters are fully described by their types, the forAll() map can go away entirely: auto() derives a generator for every uncovered parameter from the closure's own signature via Gen::forParameters() — the @param psalm type when the closure carries a docblock (int<1, 300>, non-empty-string, list<T>, 'a'|'b'), the native type otherwise. A docblock over a closure is legal PHP, and reflection sees it in every natural placement:

$this->forAll()
    ->auto()
    ->check(
        /**
         * @param int<1, 300> $base
         * @param int<1, 86400> $cap
         */
        function (int $base, int $cap): void {
            self::assertLessThanOrEqual(86_400, $cap);
        },
    );

The forAll() map becomes the overrides and may be partial — the escape hatch for domains no psalm type can express (a float range, a dependent pair built with Gen::flatMap()):

$this->forAll(['multiplier' => Gen::floatBetween(1.0, 4.0)])
    ->auto()
    ->check(
        /** @param int<1, 40> $attempt */
        function (float $multiplier, int $attempt): void { /* … */ },
    );

Rules worth knowing — verbatim the Testo adapter's (#[Property(auto: true)]):

Strictly opt-in. A bare int or float derives its full native domain, and only the property's author knows whether that is the intended one. Annotate or override anything narrower.
A type the deriver cannot read (a bare array, mixed, an untyped or variadic parameter) fails with an error naming the function and the parameter — never a silently widened guess.
A forAll() key that is not a parameter of the closure is an error, with auto() as without it; under auto() merge semantics would otherwise silently replace a typoed entry with a signature-derived generator.
A full map plus auto() is legal — auto derives nothing.
There is deliberately no PROPERTY_AUTO environment variable: the environment dials the suite, while auto() changes what one property's arguments mean.

Naming a property (`id()`)

id() names the property:

$this->forAll(['values' => Gen::arrayOf(Gen::int())])
    ->id('sort::idempotent')
    ->runs(300)
    ->check(static function (array $values): void { /* … */ });

Left out, the name is derived from the calling method — which is right for a test method and wrong for a closure, because PHP has no stable name for one. On PHP 8.3 every closure of a class is {closure}, so two properties in one file share a corpus key and overwrite each other's recorded counterexample; from 8.4 the name is {closure:/path/File.php:19}, so inserting a line above the property orphans yesterday's entry. Nothing throws — the corpus just stops replaying the failure it exists to replay.

So call id() whenever forAll() runs inside a closure rather than directly in a test method (Pest's it() and test() are the common case). The id keys the regression corpus and every event, and it also becomes the display name, so one string identifies the property in the corpus, in the events and in the printed output.

Expected exceptions (`throws()`)

expectException() cannot work inside a check() closure: the executor observes every throw before PHPUnit's own expectation mechanism does, so the expected exception would falsify the property. throws() states the expectation where it is checked:

$this->forAll(['width' => Gen::intBetween(1, 5000), 'minWidth' => Gen::intBetween(2, 5000)])
    ->runs(200)
    ->throws(ImageUploadException::class)
    ->check(static function (int $width, int $minWidth): void {
        $validator->validate(imageWithWidth($width), profileWithMinWidth($minWidth));
    });

Semantics, per trial:

Throws the class (a subclass matches) — the trial passes.
Returns normally — the trial fails with Expected <class> to be thrown, but it was not, and the input shrinks like any other counterexample.
Throws another class — that throw is the failure, exactly as it would be without throws().
A failed assertion is never the expected throw. PHPUnit's AssertionFailedError extends \RuntimeException, so throws() refuses a class it is an instance of — \RuntimeException, \Exception, \Throwable — with a failed assertion is an instance of it; name the exception the body throws. A failing assertSame() inside the body fails the trial whatever class was expected.
markTestSkipped()/markTestIncomplete() still skip the run, and an Assume::that() discard still discards it: the environment's verdict about the run is never a pass earned by throwing.

A class that is not a Throwable is rejected immediately — a typoed name would otherwise falsify every run without a word of explanation.

How results map onto PHPUnit

The check is the assertion: every outcome counts one assertion on the running TestCase, so a property whose body asserts nothing is never marked risky — neither when it passes nor when it is falsified (a failing property is reported once, as a failure). Only a property whose every run was skipped registers none: that test is skipped, not checked.
Every failing outcome (falsified, gave up, unmet coverage, deadline, budget, generation failure, failing example, replayed regression) surfaces as one AssertionFailedError whose message is the engine's own — seed, original and shrunk arguments, shrink statistics — and whose previous is the engine exception (PropertyViolationException, GaveUpException, RegressionViolationException, …).
Assume::that() is a discarded run inside the property, retried by the engine — never a skipped PHPUnit test.
markTestSkipped() / markTestIncomplete() inside the body skip that run (a discard); when every run skipped, the skip is rethrown and PHPUnit reports the test as skipped or incomplete. Partly skipped runs spend a budget of their own, separate from maxDiscards: a skip is not a discard, and when that budget runs out the message names the environment rather than advising narrower generators. Unlike an Assume::that() discard, a skip says nothing about the input, so a recorded regression whose replay only skipped stays in the corpus instead of being pruned.
expectException() does not see the body's exception: the engine catches it as the run's failure. Declare the expectation with throws() instead, or assert on the exception inside the body.
setUp() runs once per test, not per generated input — a property is one test method with one check().
With a data provider, the corpus id carries the data set name (Class::method with data set "large"), so the sets do not replay — and prune — each other's regressions. The unstable-id warning is printed only when a corpus is in use, once per id.

Environment overrides

Byte-for-byte parity with the Testo adapter — one contract across adapters:

Variable	Effect
`PROPERTY_RUNS`	Positive integer that overrides every property's run count (dial runs up in CI)
`PROPERTY_SEED`	Integer seed for any property without an explicit `seed()` (replay a whole suite). An explicit `seed()` still wins
`PROPERTY_VERBOSE`	Enables the trace of every run's generated arguments and each accepted shrink step. `''` is unset; `0`, `false`, `off`, `no` (case-insensitive, trimmed) are off; anything else is on.
`PROPERTY_DB`	Directory path enabling the regression corpus, or a `redis://host[:port][/db][?prefix=key-prefix]` DSN (`rediss://` for TLS) for a corpus shared between CI and developers. Unset means off, nothing is written
`PROPERTY_PHASES`	Comma-separated stage list (`examples,corpus,random,shrink`, case-insensitive) that overrides `phases()` — an unknown name throws rather than skipping a stage. `examples,corpus` is the fast pull-request gate
`PROPERTY_DERANDOMIZE`	Derives every unset seed from the property id, making a whole suite reproducible without editing it. Same switch words as `PROPERTY_VERBOSE`: `''` is unset, `0`/`false`/`off`/`no` are off, anything else is on
`PROPERTY_PATH`	A recorded shrink descent (`CounterExample::$path`) replayed instead of searched for. Needs the seed that produced it; an explicit `path()` wins. It describes one failure, so run it with `--filter` on that one test — every other property would report the path as stale
`PROPERTY_EDGE_CASES`	`mixin` or `none` (case-insensitive) — the numeric boundary bias for the whole suite, overriding `edgeCases()`. An unknown value throws
`PROPERTY_EXHAUSTIVE`	Turns exhaustive mode on for every property whose domain fits its budget, overriding `exhaustive()`; the same switch words as `PROPERTY_DERANDOMIZE`
`PROPERTY_SEARCH_RUNS`	Non-negative integer overriding `searchRuns()` for the whole suite — a bigger search budget on a nightly, or `0` to switch it off. A malformed value throws

PROPERTY_DB takes either a directory or a Redis DSN:

PROPERTY_DB=/tmp/corpus                           vendor/bin/phpunit   # one machine
PROPERTY_DB=redis://127.0.0.1:6379                vendor/bin/phpunit   # shared
PROPERTY_DB=redis://redis:6379/2?prefix=suite-a:  vendor/bin/phpunit   # shared server, database 2, own prefix
PROPERTY_DB=rediss://redis.example.com            vendor/bin/phpunit   # TLS

The DSN has the shape everything else gives it (the IANA registration, predis, Symfony): the path is the database index, the key prefix is the prefix query parameter, rediss:// is TLS. The pre-0.6 form with the prefix in the path (redis://host/suite-a:) is refused with the new spelling in the message. The value is parsed by the engine's CorpusFactory, shared with the Testo adapter.

A directory remembers a counterexample for whoever owns it — in CI, a machine deleted when the job ends. The Redis form is the same corpus, in the same document, shared. It needs ext-redis or predis/predis; neither installed is an error rather than a silent fall back to the filesystem. A PROPERTY_DB with any other scheme — a rediss:// typo, another backend — is likewise an error, never a directory named after the scheme. Credentials in the DSN (redis://user:pass@host) are rejected rather than silently dropped; configure Redis AUTH out of band.

The corpus format is exactly the one rasuvaeff/property-testing 2.8 wrote — a corpus recorded under Testo (or under 2.x) replays here and vice versa. On falsification the minimal input is recorded; the next run replays recorded failures first (unless seed() pins the property) and reports a still-red one as a RegressionViolationException; a green one is pruned.

Distribution and discards

Classify::label()/when()/cover() work inside the property body. When a classified property passes, the adapter prints the label distribution:

Property "testSortKeepsEveryElement" distribution: long 39% (77/200), short 61% (123/200)

A property that discards more than 90% of its attempts (via Assume::that()) gets a warning suggesting narrower generators.

Beside the distribution line, the adapter prints what else the engine measured: each Classify::tabulate() table with its tag shares and the pairs hit together (Property "…" table features: compressed 40% (80/200), retried 15% (30/200); together: compressed & retried 10% (20/200)), whether exhaustive mode walked the domain (enumerated its whole domain of 24 input(s), stdout) or why it sampled instead (stderr), and the search report of a body that targets something (search: 100 evaluation(s); delay max 58210 (7 improvement(s))). PROPERTY_VERBOSE also logs every TargetImproved event with the input that scored it.

These diagnostics — the distribution on stdout, the discard warning and the unstable-id warnings on stderr — are written straight to the process streams while PHPUnit is printing its progress dots, so each one starts with a newline of its own (\n + line + \n) rather than being glued to the end of ....F... The Testo adapter, which has no progress row, prints the same lines without the leading newline. The streams and the exact format are diagnostics for a human reading the terminal, not a contract: grep for the Property "<name>" prefix, not for a byte offset.

Why no `#[Property]` attribute?

PHPUnit's public extension/event API observes test execution but offers no stable contract for intercepting and re-invoking a test method many times — which is exactly what a property attribute must do. This adapter deliberately does not depend on PHPUnit internals; the fluent API needs only the documented surface. An attribute may appear later, only if it can be built on the documented extension API of the supported majors.

Generators

The full generator catalog (Gen::int() … Gen::subset(), Gen::regex(), Gen::commands(), Gen::draw(), Shrinkable, writing your own ArbitraryInterface, stateful/model-based testing) is the engine's API, documented in the core README. Everything there is usable from a check() closure as-is.

Public API of this package

Type	Role
`Rasuvaeff\PropertyTesting\PhpUnit\PropertyTesting`	The trait a `TestCase` mixes in; `forAll()` is its single entry point
`Rasuvaeff\PropertyTesting\PhpUnit\PropertyCheck`	The fluent builder: resolves the chain and the environment into a core `PropertyDefinition`, runs the engine, maps the structured result onto PHPUnit

VerboseListener, PhpUnitTrialExecutor, the PropertyCheck constructor and its output() seam are @internal. The environment variables and the PROPERTY_DB DSN are parsed by the engine (EnvironmentOverrides, CorpusFactory), so they mean the same thing under the Testo adapter.

Security

Generated values are pseudo-random (seeded MT19937), not cryptographic. Seeds are not secrets — they are printed in failure output by design. Treat PROPERTY_DB corpus files as test artifacts: they contain generated inputs verbatim, so do not point the variable at a directory that gets published.

Examples

See examples/ — a complete property-based TestCase:

vendor/bin/phpunit examples/SortPropertyTest.php

Development

make install     # composer install (Docker)
make build       # validate + normalize + require-checker + cs + psalm + tests
make cs-fix      # apply code style
make mutation    # infection mutation testing

Tests run through PHPUnit (composer test is phpunit), not Testo.

License

BSD-3-Clause

Name		Name	Last commit message	Last commit date
.github		.github
examples		examples
src/PhpUnit		src/PhpUnit
tests		tests
.editorconfig		.editorconfig
.gitattributes		.gitattributes
.gitignore		.gitignore
.php-cs-fixer.php		.php-cs-fixer.php
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
LICENSE.md		LICENSE.md
Makefile		Makefile
README.md		README.md
README.ru.md		README.ru.md
composer-require-checker.json		composer-require-checker.json
composer.json		composer.json
infection.json5		infection.json5
llms.txt		llms.txt
phpunit.xml		phpunit.xml
psalm.xml		psalm.xml
rector.php		rector.php

Navigation Menu

Repository files navigation

rasuvaeff/property-testing-phpunit

Part of the property-testing family

Requirements

Installation

Usage

The fluent chain

Auto-derived generators (`auto()`)

Naming a property (`id()`)

Expected exceptions (`throws()`)

How results map onto PHPUnit

Environment overrides

Distribution and discards

Why no `#[Property]` attribute?

Generators

Public API of this package

Security

Examples

Development

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

rasuvaeff/property-testing-phpunit

Part of the property-testing family

Requirements

Installation

Usage

The fluent chain

Auto-derived generators (auto())

Naming a property (id())

Expected exceptions (throws())

How results map onto PHPUnit

Environment overrides

Distribution and discards

Why no #[Property] attribute?

Generators

Public API of this package

Security

Examples

Development

License

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Auto-derived generators (`auto()`)

Naming a property (`id()`)

Expected exceptions (`throws()`)

Why no `#[Property]` attribute?