0.3.0: an object is a case, its type its kind; how a case is run stated with it

.github/workflows/ci.yml

@@ -53,6 +53,7 @@ jobs:
           set -euo pipefail
           mcpp build
           mcpp test -p mcpplibs.mtest
+          mcpp test -p mcpplibs.mtest.manifest
           mcpp test -p mtest.mtest
           mcpp test -p conformance
       - name: Every implementation passes the conformance suite
@@ -67,7 +68,7 @@ jobs:
         run: |
           set -euo pipefail
           mtest=$(find target -type f -name mtest -path '*/bin/mtest.mtest/*' | head -1)
-          programs=$(find target -type f -path '*/bin/*' \( -name expectations -o -name companions \))
+          programs=$(find target -type f -path '*/bin/*' \( -name expectations -o -name companions -o -name suites \))
           "$mtest" run $programs --junit target/mtest-junit.xml
           "$mtest" gate --set full
           "$mtest" ci annotate
@@ -107,7 +108,7 @@ jobs:
       - name: The tests as images
         run: |
           set -euo pipefail
-          for member in mcpplibs.mtest mtest.mtest conformance; do
+          for member in mcpplibs.mtest mcpplibs.mtest.manifest mtest.mtest conformance; do
             mcpp test -p "$member" --target "${{ matrix.arch }}-openkal"
           done
       - name: The script adapters on this system

.github/workflows/release.yml

@@ -2,7 +2,7 @@ name: Release
 
 # A tag vX.Y.Z: CI on the commit, then a GitHub release of the engine's package for every
 # system and its okb, with their checksums and the version's notes from CHANGELOG.md. The
-# source archive of the tag is what mcpp-index's packages (mcpplibs.mtest, .gtest, .capi)
+# source archive of the tag is what mcpp-index's packages (mcpplibs.mtest, .manifest, .gtest, .capi)
 # name; xim's `mtest` takes the package.
 
 on:

AGENTS.md

@@ -27,7 +27,8 @@ protocol. This file states the core ideas and rules; the design and its record a
 | Member | Role | Depends on |
 |---|---|---|
 | `lib` | the C++ library: cases, expectations, `bench`, `cts`, `e2e` | `mcpplibs.os`, JSON |
-| `engine` | the `mtest` command and its service | `mcpplibs.os`, `mcpplibs.cmdline`, JSON, TOML |
+| `manifest` | the static case manifest: the cases a source file declares, read from its text | nothing |
+| `engine` | the `mtest` command and its service | `manifest`, `mcpplibs.os`, `mcpplibs.cmdline`, JSON, TOML |
 | `conformance` | the protocol's conformance suite | `lib` |
 | `adapters/*` | GoogleTest, Bash, Python | the framework adapted, JSON |
 | `capi` | `include/mtest.h` for any language with a C FFI | JSON |

CHANGELOG.md

@@ -1,5 +1,41 @@
 # Changelog
 
+## [0.3.0] - 2026-11-08
+
+A case states in code what it is and what it needs, and the runner acts on that statement and
+records what it did.
+
+- An object is a case: `UTest`, `E2E`, `CTS` and `Bench` (and `Suite`, of no kind) take a name, any
+  attributes and a body. Making the object records them; a runner lists the object as one case,
+  schedules it by its attributes and runs its body when it is selected. Cases the body declares are
+  under its name, have its attributes and add their own, and can be selected alone; the body's other
+  statements run only when one of them does. A case that runs only as the scope of a selected one is
+  reported only when a statement of its own fails (P3.4). `suite` remains as the spelling of 0.2.
+- Attributes for how a case is run: `timeout{}`, `isolate{}`, `retries{n}` and `diagnose{.env, .limit}`
+  (a failing case runs once more with that environment; its output is kept and named by
+  `mtest explain`). P3.3 gains `timeout`, `isolation`, `retries` and `diagnose`; the Bash adapter
+  (`--timeout`, `--isolate`, `--retries`, `--diagnose-env`, `--diagnose-timeout`) and the C ABI
+  (`mtest_case_timeout`, `mtest_case_isolate`, `mtest_case_retries`, `mtest_case_diagnose`) state
+  them too.
+- `shared<T>`: an object the cases of a body share, made when one first uses it; the `fixture` event
+  reports which case made it and how long that took.
+- A case that states no kind is of its program's: `mtest.toml`'s, else its directory's
+  (`tests/unit`, `tests/e2e`, `tests/cts`, `tests/bench`; `docs/layout.md`), else unit. The library,
+  the Bash adapter and the C ABI no longer state `unit` for a case that names no kind.
+- `[cases."<glob>"]` takes the library's attributes in the same words (`clauses`, `artifacts`,
+  `isolation`, `retries`, `diagnose`, `skip`, `expect_fail` beside `kind`, `tags`, `resources`,
+  `timeout`); what the program states comes first (in 0.2 the file took precedence).
+- The engine runs a case that states isolation in a process of its own, ends a process that runs one
+  case at that case's time limit and gives a process of several at least the sum of theirs, retries
+  a case as often as it states, records each program's start and what it held (`timeline` of
+  `--report-json`), and records skips and expected failures `mtest.toml` states.
+- The static manifest is a package of its own, `mcpplibs.mtest.manifest`, which a tool that reads
+  tests (the engine, an editor) depends on without linking the library a test program links. It
+  reads the suite types, a suite type the file derives from one
+  (`class Fixture : public CTS`), a suite's attributes, `resource`, `clause` and `artifact`;
+  `mtest list --static` gives a case the kind and attributes `mtest list` gives it.
+- Discovery globs match `**/` as no directory too (`tests/**/*.sh` matches `tests/a.sh`).
+
 ## [0.2.0] - 2026-11-08
 
 - `[discover.<name>]`: every file a glob matches is a program run by a command (`{path}`,

README.md

@@ -38,21 +38,27 @@ int main() {
 
 ```toml
 [dev-dependencies]
-mcpplibs.mtest = "0.2.0"
+mcpplibs.mtest = "0.3.0"
 ```
 
 A failure names its place and both sides (`expected 2, actual 3`). Under a runner the same program
 speaks the protocol; alone it prints one line per case.
 
-A file may declare suites and no `main`; mtest's entry begins it (mcpp 2026.11.8.1 and later; an
-earlier mcpp is told `[test] main = { from = "mtest/main" }`):
+A file may declare its cases as objects and no `main`; mtest's entry begins it (mcpp 2026.11.8.1
+and later; an earlier mcpp is told `[test] main = { from = "mtest/main" }`). Each object is one
+case a runner lists, selects and schedules by the attributes it states (kind by its type,
+resources, time limit, isolation, retries, a diagnosis); the cases its body declares are selectable
+within it ([lib/docs/writing-tests.md](lib/docs/writing-tests.md)):
 
 ```cpp
 import mcpplibs.mtest;
 using namespace mcpplibs::mtest;
 
-suite parsing{"parsing", [] {
-    "an empty line"_test = [] { expect(eq(parse(""), 0)); };
+UTest empty{"an empty line", [] { expect(eq(parse(""), 0)); }};
+
+E2E cli{"cli", resource{"serial:home"}, timeout{std::chrono::minutes{2}}, [] {
+    "prints its version"_test = [] { /* ... */ };
+    "reads its configuration"_test[retries{1}] = [] { /* ... */ };
 }};
 ```
 

README.zh-CN.md

@@ -37,20 +37,23 @@ int main() {
 
 ```toml
 [dev-dependencies]
-mcpplibs.mtest = "0.2.0"
+mcpplibs.mtest = "0.3.0"
 ```
 
 失败时给出位置与两侧的值（`expected 2, actual 3`）。在运行器之下，同一程序按协议报告；单独运行时每个用例输出一行。
 
-文件可以只声明 suite 而不写 `main`，由 mtest 的入口作为起点（mcpp 2026.11.8.1 及以后；更早的 mcpp 需写
-`[test] main = { from = "mtest/main" }`）：
+文件可以只以对象声明用例而不写 `main`，由 mtest 的入口作为起点（mcpp 2026.11.8.1 及以后；更早的 mcpp 需写
+`[test] main = { from = "mtest/main" }`）。每个对象是一个用例，运行器按它声明的属性（类型即类别，以及资源、时限、隔离、重试、诊断）列举、选择与调度；其函数体声明的用例可在其中单独选择（[lib/docs/writing-tests.md](lib/docs/writing-tests.md)）：
 
 ```cpp
 import mcpplibs.mtest;
 using namespace mcpplibs::mtest;
 
-suite parsing{"parsing", [] {
-    "an empty line"_test = [] { expect(eq(parse(""), 0)); };
+UTest empty{"an empty line", [] { expect(eq(parse(""), 0)); }};
+
+E2E cli{"cli", resource{"serial:home"}, timeout{std::chrono::minutes{2}}, [] {
+    "prints its version"_test = [] { /* ... */ };
+    "reads its configuration"_test[retries{1}] = [] { /* ... */ };
 }};
 ```
 

adapters/gtest/mcpp.toml

@@ -3,7 +3,7 @@
 # own listener. A program's tests stay GoogleTest's (`TEST`, `EXPECT_EQ`); the package supplies `main`:
 #
 #   [dev-dependencies]
-#   mcpplibs.mtest.gtest = "0.2.0"
+#   mcpplibs.mtest.gtest = "0.3.0"
 #
 # mcpp 2026.11.8.1 and later begin a test file that defines no `main` here by itself (`test-main`);
 # an earlier mcpp is told `[test] main = { from = "gtest/main" }`.
@@ -13,7 +13,7 @@
 [package]
 namespace   = "mcpplibs.mtest"
 name        = "gtest"
-version     = "0.2.0"
+version     = "0.3.0"
 description = "GoogleTest speaking the mtest protocol: a test main that lists, selects and reports GoogleTest's cases"
 license     = "Apache-2.0"
 

adapters/gtest/src/protocol.cpp

@@ -127,7 +127,7 @@ int mcpplibs_mtest_gtest_main(int argc, char** argv) {
                   {"protocol", 1},
                   {"level", 1},
                   {"framework", "googletest"},
-                  {"version", "0.2.0"},
+                  {"version", "0.3.0"},
                   {"isolation", {"process", "case"}},
                   {"guide", "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/mcpp-community/mtest/blob/main/adapters/gtest/docs/"
                             "writing-tests.md"}});

adapters/script/mtest.sh

@@ -38,7 +38,7 @@ _mtest_list() {   # a JSON array of the arguments
 _mtest_event() { [ -n "$_mtest_report" ] && printf '{%s}\n' "$1" >> "$_mtest_report"; return 0; }
 _mtest_ms() { local now; now=$(date +%s%N 2>/dev/null || echo 0); echo $(( (now - _mtest_started) / 1000000 )); }
 
-_mtest_event '"e":"program","protocol":1,"level":3,"framework":"mtest.sh","version":"0.2.0","isolation":["process","case"]'
+_mtest_event '"e":"program","protocol":1,"level":3,"framework":"mtest.sh","version":"0.3.0","isolation":["process","case"]'
 
 _mtest_selected() {
     [ -z "$_mtest_select" ] && return 0
@@ -50,17 +50,26 @@ _mtest_selected() {
     return 1
 }
 
-# mtest_case ID FUNCTION [--tag T] [--kind K] [--clause C] [--resource R] [--artifact A] [--expect-fail REASON] [--skip REASON]
+# mtest_case ID FUNCTION [--tag T] [--kind K] [--clause C] [--resource R] [--artifact A] [--timeout SECONDS]
+#            [--isolate] [--retries N] [--diagnose-env NAME=VALUE] [--diagnose-timeout SECONDS]
+#            [--expect-fail REASON] [--skip REASON]
+# A case that states no kind is of the program's (its directory's, docs/layout.md).
 mtest_case() {
     local id="$1" fn="$2"; shift 2
-    local tags=() clauses=() resources=() artifacts=() kind=unit xfail="" skip=""
+    local tags=() clauses=() resources=() artifacts=() kind="" limit="" xfail="" skip=""
+    local isolate="" retries="" diagnose=() diagnose_limit=""
     while [ $# -gt 0 ]; do
         case "$1" in
             --tag) tags+=("$2"); shift 2 ;;
             --kind) kind="$2"; shift 2 ;;
             --clause) clauses+=("$2"); shift 2 ;;
             --resource) resources+=("$2"); shift 2 ;;
             --artifact) artifacts+=("$2"); shift 2 ;;
+            --timeout) limit="$2"; shift 2 ;;
+            --isolate) isolate=1; shift ;;
+            --retries) retries="$2"; shift 2 ;;
+            --diagnose-env) diagnose+=("$2"); shift 2 ;;
+            --diagnose-timeout) diagnose_limit="$2"; shift 2 ;;
             --expect-fail) xfail="$2"; shift 2 ;;
             --skip) skip="$2"; shift 2 ;;
             *) echo "mtest_case: unknown option $1" >&2; shift ;;
@@ -69,7 +78,19 @@ mtest_case() {
     _mtest_seen+=("$id")
     if [ "$_mtest_mode" = list ]; then
         local ev
-        ev="\"e\":\"case\",\"id\":$(_mtest_json "$id"),\"name\":$(_mtest_json "$id"),\"file\":$(_mtest_json "${BASH_SOURCE[1]:-$0}"),\"line\":${BASH_LINENO[0]:-0},\"kind\":$(_mtest_json "$kind"),\"tags\":$(_mtest_list "${tags[@]}"),\"clauses\":$(_mtest_list "${clauses[@]}"),\"resources\":$(_mtest_list "${resources[@]}"),\"artifacts\":$(_mtest_list "${artifacts[@]}")"
+        ev="\"e\":\"case\",\"id\":$(_mtest_json "$id"),\"name\":$(_mtest_json "$id"),\"file\":$(_mtest_json "${BASH_SOURCE[1]:-$0}"),\"line\":${BASH_LINENO[0]:-0},\"tags\":$(_mtest_list "${tags[@]}"),\"clauses\":$(_mtest_list "${clauses[@]}"),\"resources\":$(_mtest_list "${resources[@]}"),\"artifacts\":$(_mtest_list "${artifacts[@]}")"
+        [ -n "$kind" ] && ev+=",\"kind\":$(_mtest_json "$kind")"
+        [ -n "$limit" ] && ev+=",\"timeout\":$limit"
+        [ -n "$isolate" ] && ev+=',"isolation":"case"'
+        [ -n "$retries" ] && ev+=",\"retries\":$retries"
+        if [ ${#diagnose[@]} -gt 0 ] || [ -n "$diagnose_limit" ]; then
+            local env_fields="" kv
+            for kv in ${diagnose[@]+"${diagnose[@]}"}; do
+                [ -n "$env_fields" ] && env_fields+=","
+                env_fields+="$(_mtest_json "${kv%%=*}"):$(_mtest_json "${kv#*=}")"
+            done
+            ev+=",\"diagnose\":{\"env\":{$env_fields}${diagnose_limit:+,\"timeout\":$diagnose_limit}}"
+        fi
         [ -n "$skip" ] && ev+=',"expect":"skip"'
         [ -n "$xfail" ] && [ -z "$skip" ] && ev+=',"expect":"fail"'
         _mtest_event "$ev"

capi/include/mtest.h

@@ -25,13 +25,23 @@ extern "C" {
 void mtest_init(const char* framework, int level);
 /* Whether the runner asked for the list of cases (MTEST_MODE=list). */
 int mtest_listing(void);
-/* A case of the list (P3). `kind` is unit, e2e, cts or bench; NULL is unit. */
+/* A case of the list (P3). `kind` is unit, e2e, cts or bench; NULL states none, and the runner
+   gives the program's (its directory's, mtest docs/layout.md). */
 void mtest_case(const char* id, const char* file, int line, const char* kind);
 /* An attribute of the case last listed: a tag, a clause, a resource, an artifact (level 3). */
 void mtest_case_tag(const char* tag);
 void mtest_case_clause(const char* clause);
 void mtest_case_resource(const char* resource);
 void mtest_case_artifact(const char* artifact);
+/* How long the case last listed may run, in seconds (P3.3). */
+void mtest_case_timeout(double seconds);
+/* The case last listed runs in a process of its own (P3.3). */
+void mtest_case_isolate(void);
+/* The case last listed runs again, alone, up to `count` times after failing (P3.3). */
+void mtest_case_retries(int count);
+/* After the case last listed fails, the runner runs it once more with NAME=VALUE added to its
+   environment, within `seconds` (0: its own limit) (P3.3). Each call adds a variable. */
+void mtest_case_diagnose(const char* name, const char* value, double seconds);
 /* The case last listed is expected to fail (level 3). */
 void mtest_case_expect_fail(void);
 /* Whether the case runs (MTEST_SELECT); every case runs when the runner selected none (P4). */

capi/mcpp.toml

@@ -4,7 +4,7 @@
 [package]
 namespace   = "mcpplibs.mtest"
 name        = "capi"
-version     = "0.2.0"
+version     = "0.3.0"
 description = "mtest's C ABI: the protocol's events for any language with a C FFI"
 license     = "Apache-2.0"
 

capi/src/mtest.cpp

@@ -25,6 +25,10 @@ struct pending_case {
     event body;
     std::vector<std::string> tags, clauses, resources, artifacts;
     bool expect_fail = false;
+    double timeout = 0;
+    bool isolated = false;
+    int retries = 0;
+    std::optional<event> diagnose;
 };
 
 struct state {
@@ -73,6 +77,14 @@ struct state {
             p.body["artifacts"] = p.artifacts;
             if (p.expect_fail)
                 p.body["expect"] = "fail";
+            if (p.timeout > 0)
+                p.body["timeout"] = p.timeout;
+            if (p.isolated)
+                p.body["isolation"] = "case";
+            if (p.retries > 0)
+                p.body["retries"] = p.retries;
+            if (p.diagnose)
+                p.body["diagnose"] = *p.diagnose;
             write(p.body);
         }
         pending.clear();
@@ -100,7 +112,7 @@ void mtest_init(const char* framework, int level) {
              {"protocol", 1},
              {"level", std::clamp(level, 1, 3)},
              {"framework", text(framework)},
-             {"version", "0.2.0"},
+             {"version", "0.3.0"},
              {"isolation", {"process", "case"}}});
 }
 
@@ -113,12 +125,15 @@ void mtest_case(const char* id, const char* file, int line, const char* kind) {
     std::lock_guard g(s.lock);
     s.flush_pending();
     s.seen.insert(id ? id : "");
-    s.pending.push_back({{{"e", "case"},
-                          {"id", text(id)},
-                          {"name", text(id)},
-                          {"file", text(file)},
-                          {"line", line},
-                          {"kind", kind ? kind : "unit"}}});
+    event body = {{"e", "case"},
+                  {"id", text(id)},
+                  {"name", text(id)},
+                  {"file", text(file)},
+                  {"line", line}};
+    // A kind not stated is the runner's to give (the program's, by its directory).
+    if (kind)
+        body["kind"] = kind;
+    s.pending.push_back({std::move(body)});
 }
 
 namespace {
@@ -143,6 +158,41 @@ void mtest_case_artifact(const char* value) {
     attribute(&pending_case::artifacts, value);
 }
 
+void mtest_case_timeout(double seconds) {
+    auto& s = the();
+    std::lock_guard g(s.lock);
+    if (!s.pending.empty())
+        s.pending.back().timeout = seconds;
+}
+
+void mtest_case_isolate(void) {
+    auto& s = the();
+    std::lock_guard g(s.lock);
+    if (!s.pending.empty())
+        s.pending.back().isolated = true;
+}
+
+void mtest_case_retries(int count) {
+    auto& s = the();
+    std::lock_guard g(s.lock);
+    if (!s.pending.empty())
+        s.pending.back().retries = count;
+}
+
+void mtest_case_diagnose(const char* name, const char* value, double seconds) {
+    auto& s = the();
+    std::lock_guard g(s.lock);
+    if (s.pending.empty())
+        return;
+    auto& d = s.pending.back().diagnose;
+    if (!d)
+        d = event{{"env", event::object()}};
+    if (name)
+        (*d)["env"][name] = text(value);
+    if (seconds > 0)
+        (*d)["timeout"] = seconds;
+}
+
 void mtest_case_expect_fail(void) {
     auto& s = the();
     std::lock_guard g(s.lock);

conformance/mcpp.toml

@@ -8,7 +8,7 @@
 [package]
 namespace   = "mcpplibs.mtest"
 name        = "conformance"
-version     = "0.2.0"
+version     = "0.3.0"
 description = "The conformance suite of the mtest protocol, for every implementation of its program side"
 license     = "Apache-2.0"
 

conformance/tests/protocol.cpp

@@ -156,8 +156,18 @@ int main() {
         std::set<std::string> listed;
         for (const auto& c : of(r, "case"))
             listed.insert(c.value("id", ""));
+        // P3.4: a case declared inside a case may be listed through the case that declares it.
+        auto listed_or_declared = [&](const std::string& want) {
+            for (auto at = want.size(); at != std::string::npos; at = want.rfind('/', at - 1)) {
+                if (listed.contains(want.substr(0, at)))
+                    return true;
+                if (at == 0)
+                    break;
+            }
+            return false;
+        };
         for (const auto& want : fixture_ids(level(r)))
-            expect(listed.contains(want)) << "not listed: " << want;
+            expect(listed_or_declared(want)) << "not listed: " << want;
     };
     "P3.2 identifiers are distinct and stable"_test[kind::cts][clause{"P3.2"}] = [] {
         const auto a = run("list");