0.2.0: discovered groups, case attributes, shards, the coverage map and the static case manifest

.agents/docs/2026-11-08-mtest-design.md

@@ -1,7 +1,7 @@
 # mtest：面向 Agent 时代的通用测试系统
 
 **日期：** 2026-11-08
-**状态：** 第二版，M1–M5 已决定；0.1.0（PR #1）实现协议 v1、C++ 库、引擎与服务、适配与一致性套件
+**状态：** 第二版，M1–M5 已决定；0.1.0（PR #1）实现协议 v1、C++ 库、引擎与服务、适配与一致性套件；0.2.0 实现 mcpp 顶层方案 W8 的 mtest 部分：`[discover.<name>]` 组、`[cases]` 与宿主清单中的用例属性、`--shard`、覆盖映射（`docs/coverage.md`）、静态用例清单（`mcpplibs.mtest.manifest`、`mtest list --static`）。构建插件形式的静态清单不采用：mcpp 以 `[package.metadata.mtest]` 保存工具的表，引擎直接读取清单与测试源，无须每次构建运行构建程序，IDE 与引擎读同一份
 **仓库：** `mcpp-community/mtest`；包的命名空间 `mcpplibs.mtest`
 **上游：** [mcpp 2026-11-08-ecosystem-885-supplement-design.md](https://github.com/mcpp-community/mcpp/blob/main/.agents/docs/2026-11-08-ecosystem-885-supplement-design.md) S7（mcpp 的测试协议）
 

CHANGELOG.md

@@ -1,5 +1,27 @@
 # Changelog
 
+## [0.2.0] - 2026-11-08
+
+- `[discover.<name>]`: every file a glob matches is a program run by a command (`{path}`,
+  `{name}`), with the keys of `[programs.*]`; tags and resources are read from each file's first
+  lines (`tags_from`, `resources_from`), so a directory of scripts is a suite without a line of
+  configuration per script. `kind` states the kind of an L0 program's one case.
+- `[cases."<glob>"]`, and a host's `[package.metadata.mtest.cases."<glob>"]` in mcpp.toml, state the
+  tags, resources, kind and timeout of the cases a glob matches, taking precedence over what a
+  program lists.
+- `--exclude-program GLOB` leaves the programs a glob matches out of a selection; globs take character classes (`[0-9]`, `[!x]`).
+- `--shard I/N` selects the I-th of N parts: each program, the longest first, to the part with the
+  least expected time; every machine with the same store computes the same parts.
+- The coverage map (`docs/coverage.md`): `mtest run --coverage` records, for each case, the sources
+  its run executed, read from GCC's `.gcda` counters and a compile database; affected selection
+  reads it, and `mtest coverage show|import` and `mtest store import` carry it between machines.
+- The static case manifest: `mcpplibs.mtest.manifest` reads the cases a source file declares
+  (`"name"_test`, their attributes, named suites, nested cases) from its text, and
+  `mtest list --static` lists a project's test files with it, building and running nothing.
+- A set that selects by change (`pr`) selects every case when the change cannot be read (a shallow clone
+  without its base), and says so; before, it selected none of the affected cases.
+- A failing case's entry in a run's answer names its kind.
+
 ## [0.1.0] - 2026-11-08
 
 The first release.

README.md

@@ -38,7 +38,7 @@ int main() {
 
 ```toml
 [dev-dependencies]
-mcpplibs.mtest = "0.1.0"
+mcpplibs.mtest = "0.2.0"
 ```
 
 A failure names its place and both sides (`expected 2, actual 3`). Under a runner the same program

README.zh-CN.md

@@ -37,7 +37,7 @@ int main() {
 
 ```toml
 [dev-dependencies]
-mcpplibs.mtest = "0.1.0"
+mcpplibs.mtest = "0.2.0"
 ```
 
 失败时给出位置与两侧的值（`expected 2, actual 3`）。在运行器之下，同一程序按协议报告；单独运行时每个用例输出一行。

adapters/gtest/docs/writing-tests.md

@@ -15,7 +15,7 @@ TEST(Parsing, Fields) {
 }
 ```
 
-- The manifest: `[dev-dependencies] mcpplibs.mtest.gtest = "0.1.0"`. A file defines no `main` (mcpp
+- The manifest: `[dev-dependencies] mcpplibs.mtest.gtest = "0.2.0"`. A file defines no `main` (mcpp
   2026.11.8.1 or later). An earlier mcpp needs `[test] main = { from = "mcpplibs.mtest.gtest/main" }`
   and `compat.gtest = "1.18.0"` beside it, so that GoogleTest's own `gtest_main.cc` stays out of the
   program.

adapters/gtest/mcpp.toml

@@ -3,7 +3,7 @@
 # own listener. A program's tests stay GoogleTest's (`TEST`, `EXPECT_EQ`); the package supplies `main`:
 #
 #   [dev-dependencies]
-#   mcpplibs.mtest.gtest = "0.1.0"
+#   mcpplibs.mtest.gtest = "0.2.0"
 #
 # mcpp 2026.11.8.1 and later begin a test file that defines no `main` here by itself (`test-main`);
 # an earlier mcpp is told `[test] main = { from = "gtest/main" }`.
@@ -13,7 +13,7 @@
 [package]
 namespace   = "mcpplibs.mtest"
 name        = "gtest"
-version     = "0.1.0"
+version     = "0.2.0"
 description = "GoogleTest speaking the mtest protocol: a test main that lists, selects and reports GoogleTest's cases"
 license     = "Apache-2.0"
 

adapters/gtest/src/protocol.cpp

@@ -127,7 +127,7 @@ int mcpplibs_mtest_gtest_main(int argc, char** argv) {
                   {"protocol", 1},
                   {"level", 1},
                   {"framework", "googletest"},
-                  {"version", "0.1.0"},
+                  {"version", "0.2.0"},
                   {"isolation", {"process", "case"}},
                   {"guide", "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/mcpp-community/mtest/blob/main/adapters/gtest/docs/"
                             "writing-tests.md"}});

adapters/script/mtest.sh

@@ -38,7 +38,7 @@ _mtest_list() {   # a JSON array of the arguments
 _mtest_event() { [ -n "$_mtest_report" ] && printf '{%s}\n' "$1" >> "$_mtest_report"; return 0; }
 _mtest_ms() { local now; now=$(date +%s%N 2>/dev/null || echo 0); echo $(( (now - _mtest_started) / 1000000 )); }
 
-_mtest_event '"e":"program","protocol":1,"level":3,"framework":"mtest.sh","version":"0.1.0","isolation":["process","case"]'
+_mtest_event '"e":"program","protocol":1,"level":3,"framework":"mtest.sh","version":"0.2.0","isolation":["process","case"]'
 
 _mtest_selected() {
     [ -z "$_mtest_select" ] && return 0

adapters/script/mtest_report.py

@@ -50,7 +50,7 @@ def _event(**fields):
             f.write(json.dumps(fields, ensure_ascii=False) + "\n")
 
 
-_event(e="program", protocol=1, level=3, framework="mtest_report.py", version="0.1.0", isolation=["process", "case"])
+_event(e="program", protocol=1, level=3, framework="mtest_report.py", version="0.2.0", isolation=["process", "case"])
 
 
 def _where(depth=2):

capi/mcpp.toml

@@ -4,7 +4,7 @@
 [package]
 namespace   = "mcpplibs.mtest"
 name        = "capi"
-version     = "0.1.0"
+version     = "0.2.0"
 description = "mtest's C ABI: the protocol's events for any language with a C FFI"
 license     = "Apache-2.0"
 

capi/src/mtest.cpp

@@ -100,7 +100,7 @@ void mtest_init(const char* framework, int level) {
              {"protocol", 1},
              {"level", std::clamp(level, 1, 3)},
              {"framework", text(framework)},
-             {"version", "0.1.0"},
+             {"version", "0.2.0"},
              {"isolation", {"process", "case"}}});
 }
 

conformance/mcpp.toml

@@ -8,7 +8,7 @@
 [package]
 namespace   = "mcpplibs.mtest"
 name        = "conformance"
-version     = "0.1.0"
+version     = "0.2.0"
 description = "The conformance suite of the mtest protocol, for every implementation of its program side"
 license     = "Apache-2.0"
 

docs/commands.md

@@ -17,17 +17,26 @@ acts on. Exit status: `0` done, `1` a test or a gate failed, `2` a usage error,
 | `--program GLOB` | the programs whose name matches |
 | `--case GLOB` | cases by identifier, or `program::identifier`; `*` and `?` |
 | `--tag T`, `--kind K`, `--exclude-tag T` | by tag; by kind (`unit`, `e2e`, `cts`, `bench`) |
+| `--exclude-program GLOB` | the programs whose name matches are left out |
 | `--set S` | a set ([sets.md](sets.md)) |
 | `--since COMMIT`, `--changed PATH` | the cases a change affects: a case's `file:` artifacts, its program's `sources` |
 | `--budget 60s` | the most useful cases that fit: those that failed lately, then the quickest |
+| `--shard I/N` | the I-th of N parts: each program, the longest first, to the part with the least expected time, the same on every machine with the same store |
 
-`plan` says why each case is in the selection, and runs nothing.
+A change affects a case whose last measured run executed a changed file (the coverage map,
+[coverage.md](coverage.md)), whose `file:` artifacts it touches, or whose program's `sources` it
+touches; a case that states nothing it reads, and that the map does not know, is affected.
+
+`plan` says why each case is in the selection, and runs nothing. `list --static` reads the test
+files `[static] globs` names (a host's own tests by default) and gives the cases each declares with
+this library, their places, kinds and tags and the attributes of `[cases]`, building and running
+nothing; a built program's own listing is the authority (`lib/src/manifest.cppm`).
 
 ## Running
 
 | Command | Does |
 |---|---|
-| `run` | runs the selection; `--jobs N`, `--seed N`, `--fail-fast`, `--retries N`, `--isolate` (a process per case), `--timeout 5m`, `--env K=V`, `--run-under CMD`; `--junit`, `--html`, `--report-json FILE` |
+| `run` | runs the selection; `--jobs N`, `--seed N`, `--fail-fast`, `--retries N`, `--isolate` (a process per case), `--timeout 5m`, `--env K=V`, `--run-under CMD`; `--junit`, `--html`, `--report-json FILE`; `--coverage` measures the coverage map |
 | `rerun` | the cases that failed in the last run |
 | `smoke` | the `smoke` set |
 | `golden` | a run with `MTEST_GOLDEN=update`: golden files are written, not compared |
@@ -59,6 +68,7 @@ A run is named by its identifier, `last`, or `last-N`.
 | Command | Does |
 |---|---|
 | `quarantine list\|add\|remove --case REF [--until DATE] [--reason TEXT]` | cases whose failures do not fail a run, each until a date (14 days by default) |
-| `store gc\|verify\|import --from DIR` | the store's retention, its consistency, another store taken in (a CI artifact) |
+| `store gc\|verify\|import --from DIR` | the store's retention, its consistency, another store taken in (a CI artifact), with its coverage map when that is newer |
+| `coverage show\|import --from FILE` | the coverage map affected selection reads, or another taken in |
 | `ci status OWNER/REPO#N...` | the checks of pull requests, failing ones with their links (by the GitHub CLI) |
 | `ci annotate`, `ci collect --from DIR` | a run's failures as GitHub annotations and a job summary; a CI store taken in |

docs/config.md

@@ -17,14 +17,46 @@ resources = ["serial:db"]               # what it holds alone (P7.1)
 tags      = ["slow"]
 timeout   = "10m"
 isolation = "process"                   # or "case": a process per case
+kind      = "e2e"                       # the kind of an L0 program's one case (default unit)
 
 [discover]
 globs = ["build/tests/*"]               # every executable that matches is a program
 
+# A group: every file the globs match (`*`, `?`, `[0-9]`, `[!x]`) is a program, run by the command ({path}: the file,
+# absolute; {name}: its path from the root), with the keys of [programs.*]. Each word after a
+# prefix on the file's first 64 lines is a tag (with the tag prefix given) or a resource.
+[discover.e2e]
+globs          = ["tests/e2e/[0-9]*.sh"]
+command        = ["bash", "{path}"]
+kind           = "e2e"
+timeout        = "20m"
+tags_from      = { "# requires:" = "requires:" }   # `# requires: gcc elf` -> requires:gcc, requires:elf
+resources_from = ["# resources:"]                   # `# resources: serial:home`
+
 [host]
 kind = "mcpp"                           # the tests of `mcpp test` (mcpp 2026.11.8.1+)
 args = ["--workspace"]
 
+# What the cases a glob matches (`program::id` or `id`) are, stated beside the code; it takes
+# precedence over what a program lists. A host states the same table in its own manifest:
+# mcpp.toml's [package.metadata.mtest.cases."<glob>"].
+[cases."parsing/*"]
+tags      = ["fast"]
+resources = ["serial:db"]
+kind      = "unit"
+timeout   = "30s"                       # an L0 program's
+
+# The test files `mtest list --static` reads; with host.kind = "mcpp", mcpp.toml's [test] discover
+# (else tests/**/*.cpp).
+[static]
+globs = ["tests/**/*.cpp"]
+
+# The coverage map a run under `--coverage` measures (coverage.md).
+[coverage]
+objects  = "target/x86_64-linux-gnu/0123abcd/obj"  # the instrumented objects (.gcno, .gcda)
+database = "target/x86_64-linux-gnu/0123abcd/compile_commands.json"
+max_age  = "7d"                         # an older map is not read: every case is then affected
+
 [sets.fast]                             # sets.md
 tags    = ["fast"]
 include = ["smoke"]

docs/coverage.md

@@ -0,0 +1,48 @@
+# The coverage map
+
+Affected selection asks which cases a change can reach. A program's `sources` and a case's `file:`
+artifacts answer it where a program states them. An end-to-end case states neither: it runs a
+program built from most of a repository, and without more knowledge every change affects it. The
+coverage map is that knowledge, measured: for each case, the source files its run executed.
+
+## Measuring it
+
+The programs under test are built with GCC's instrumentation (`--coverage`, or `-fprofile-arcs`),
+and `mtest.toml` names their objects and a compile database that maps each object to its source:
+
+```toml
+[coverage]
+objects  = "build/obj"
+database = "build/compile_commands.json"
+max_age  = "7d"
+```
+
+`mtest run --coverage` gives each program of the run a directory of its own (`GCOV_PREFIX`, with
+`GCOV_PREFIX_STRIP` set so that each `.gcda` file lies at its object's path below it); the files a
+program's run wrote record which of its objects ran. A `.gcda` file records a counter that ran when
+one of its arc records holds a non-zero count; GCC 12 and later write a run of zero counters as a
+negative length. Each object that ran names its source through the database, and the map is written
+to the store:
+
+```json
+{"format": "mtest-coverage/1", "commit": "<sha>", "time": "<ISO 8601>", "epoch": 1794100000,
+ "cases": {"<program>::<case>": ["src/a.cpp", "src/b.cpp"]}}
+```
+
+The cases of one program share its directory, so the map is per program; `--isolate` measures each
+case in its own process and directory.
+
+## Reading it
+
+`--changed`, `--since` and the `pr` set read the store's map when it is no older than `max_age`. A
+case the map knows is affected when the change touches a file its run executed, a `file:` artifact,
+or a source of its program; a case the map does not know is decided as before, and one that states
+nothing it reads is affected. A map older than `max_age` is not read, so a stale map widens the
+selection and never narrows it.
+
+## Sharing it
+
+A map is measured where the instrumented build runs, on a schedule of the project's choosing (a
+nightly run of the `full` set), and travels as a file: `mtest coverage import --from FILE` takes one
+in, `mtest store import --from DIR` takes a store's when it is newer, and `mtest coverage show`
+prints what the store holds and whether it is read.

engine/mcpp.toml

@@ -10,7 +10,7 @@
 [package]
 namespace   = "mtest"
 name        = "mtest"
-version     = "0.1.0"
+version     = "0.2.0"
 description = "The mtest engine: run, schedule, store, explain and analyse tests of any language"
 license     = "Apache-2.0"
 authors     = ["mcpp-community"]
@@ -29,10 +29,9 @@ kind = "bin"
 main = "src/main.cpp"
 
 [dependencies]
+mcpplibs.mtest      = { path = "../lib" }   # the static case manifest (mcpplibs.mtest.manifest)
 mcpplibs.os         = "0.4.0"
 mcpplibs.cmdline    = "0.0.2"
 nlohmann.json       = "3.12.0"
 marzer.tomlplusplus = "3.4.0"
 
-[dev-dependencies]
-mcpplibs.mtest = { path = "../lib" }