Repository navigation
Module test hardening: tests/<lib>/ suites, conversion module, export fixes - #5
Draft
ZheFeng7110 wants to merge 35 commits into
Draft
ZheFeng7110 wants to merge 35 commits into
ZheFeng7110 wants to merge 35 commits into
Conversation
… transform_reduce include guard The prefix heuristic saw no headers for conversion (its headers sit at the top of boost/), so the library had no module and its casts were exported by range and graph. EXTRA_HEADERS gives it its three headers, ownership and include edges; pointer_cast.hpp is smart_ptr's so conversion does not claim smart_ptr's names. range, graph and statechart now import the conversion module instead of re-exporting the casts. core's header set adds boost/core/detail/string_view.hpp, the only header of the documented boost::core::string_view; no module exported the alias before (json exported only basic_string_view). patches/algorithm.patch: transform_inclusive_scan.hpp reused the include guard of transform_reduce.hpp, so transform_reduce was missing from the algorithm module and from any TU that includes both headers. mcpp.toml also gains [dev-dependencies] compat.doctest and the [test] discover globs of the test hardening.
…octest suites Each library of the hardening gets tests/<lib>/ (smoke.cpp is the former tests/<lib>.cpp); examples*.cpp port the official examples, api*.cpp the documented components, custom_types.cpp the user-type suite, and include_*.cpp the macro API as plain #include TUs. coverage.toml maps every example and documentation snippet of the 1.92.0 release to a test case or a reason (tools/ci/test_coverage.py scan/link/check). First libraries: algorithm, any, callable_traits, charconv, circular_buffer, compat, conversion, core, crc, dynamic_bitset, variant2. tests/dll/ and tests/test/utf/ are test projects of their own (plugins; the compiled Boost.Test form), excluded from the root [test] discover.
…container_hash, convert The converter headers (stream, lexical_cast, strtol, printf, charconv) are not self-contained, so the standalone-compile filter had dropped them and the convert module exported convert() without a converter. They now follow <boost/convert.hpp> in an explicit GMF ([convert] in export_policy.toml); spirit.hpp stays header-only (it would make Spirit's whole surface convert's). Four boost::parameter internals that graph and heap exported move to convert, which now reaches Boost.Parameter first. patches/convert.patch makes boost::throw_on_failure inline constexpr: as a namespace-scope const object it had internal linkage and could not be exported. New suites: uuid (every documented component), container_hash (official examples; described classes through the module and as headers), convert (examples grouped by the default converter they define, user converters).
…l examples patches/bimap.patch turns the anonymous-namespace references of boost/bimap/support/lambda.hpp into inline constexpr (as lambda's _1), so the bimap module exports them. The examples run as doctest cases whose output must equal the original program's, built from upstream/boost alone with clang++; examples that combine bimap with assign, lambda, property_map, range or serialization use those modules, and the foreach / typeof / xpressive ones are a pure-include TU. api.cpp covers the inline tutorial code, custom_types.cpp user keys, comparators, hashers, tags and info payloads.
…an output oracle bloom: the tutorial section by section, every subfilter, user hashers and allocator; genome.cpp runs the k-mer example on a FASTA fixture instead of the 140 MB download. describe: all non-fmt examples (the json and serialization ones with the BOOST_DESCRIBE_* macros included before the module imports) and the class/enum pages over user types. container_hash: every customization point (hash_value by ADL, boost::hash specialization, is_range / is_tuple_like / is_unordered_range). Ported example programs compare their output with an oracle: the same ported code compiled from upstream/boost headers alone with clang++. test_coverage.py link also maps <page>#example_<stem> sections to the example's case.
…ch kind, user types/algorithms/flavors)
…isitation, stats as headers, user types)
patches/container.patch: boost/container/pmr/devector.hpp reused the include guard of pmr/vector.hpp, so the module (whose GMF includes devector first) never exported boost::container::pmr::vector / vector_of. The 19 examples (output oracle), the documentation's inline code (hub, devector, stable_vector, pmr, const-reference parameters, null iterators), user types, options and a user memory_resource. doc_extended_allocators is recorded as a gap: the package does not build the dlmalloc TUs.
…ings); regex suite patches/regex.patch (M5) had moved the function-local static tables of regex_traits_defaults.hpp to namespace-scope arrays with internal linkage. In a TU that imports the regex module, gcc gives those tables a local, uninitialized copy: get_default_error_string() returned nullptr and every invalid expression threw std::logic_error (string from null) instead of regex_error; collating names ([[.space.]]) failed alike. The tables are inline constexpr now: external linkage and COMDAT, so the original multiple-definition problem stays solved. Suite: 17 examples (fixture files on the command line, fixed stdin, grep over program_options), the reference pages' inline examples, user traits, iterators, allocators, formatters and grep callbacks.
…url suite patches/url.patch: Boost.URL's character sets and grammar rules (grammar::alnum_chars, digit_chars, unreserved_chars, uri_rule, authority_rule, ...) were namespace-scope constexpr objects, which have internal linkage and cannot be exported; the url module lacked all 19. They are BOOST_INLINE_CONSTEXPR now, as upstream already spells absolute_uri_rule. Suite: the six command-line examples (output oracle, documented arguments, fixtures), the documentation's unit files (snippets, grammar, 3_urls) run through a doctest shim of Boost.URL's test macros, and user rules, charsets and string tokens.
patches/json.patch: BOOST_JSON_INLINE_VARIABLE declared object_kind, array_kind and string_kind as references in an anonymous namespace; they are inline constexpr now, so the json module exports them. container: allocator_traits::construct uses the global operator new(std::size_t, void*, boost_container_new_t); a consumer that instantiates it (pmr::vector<json::value>::resize) could not see it. The policy exports the tag and the placement operators, and gen_exports.py accepts namespace = "" (the global namespace) for [[<lib>.extra]]. Suite: the json documentation's code files (test/snippets.cpp and doc_*.cpp, which the pages include) through a doctest shim, the examples (output oracle; the parse_into* timing benchmarks excluded), user types via tag_invoke and contexts, described types (describe headers + json module).
…temp paths) The MSVC ABI compresses one empty base only, so two sizeof checks of empty_value hold elsewhere; demangled names are matched in a form both ABIs print; describe's string_to_enum output (a typeid name) is compared off MSVC only; and the ported command-line examples name their fixtures relative to the temporary directory, which they make current, since the output prints the path (macOS's temporary directory is not /tmp).
The decimal facade's GMF declared no std::formatter specializations: boost/decimal/format.hpp provides them only when __cpp_lib_format is defined, and nothing had included <version> yet. The GMF now starts with <version>, and unexported aliases in the purview name the six specializations so a consumer's std::format reaches them. Suite: the examples (output oracle; configuration macros taken from the headers that define them), the six type pages section by section, cmath / charconv / strings / cfenv / format pages (several of the pages' names and values differ from 1.92 and are noted), and user generic code, a money type and decimal keys.
…pages) - asio: the self-contained examples (cpp11/14/20) and the network ones over loopback in one process (servers in a background thread against a harness client, clients against their ported server), the overview pages and the requirements pages as user types (completion token, buffer sequences, executor, services, random-access devices); 171 examples covered, 18 excluded with reasons (ssl, spawn, windows, fork, icmp, multicast, ...) - asio module: export experimental::use_promise (use_promise.hpp is not self-contained upstream, so the standalone filter had dropped it) and the stream_errc / socket_errc namespaces (using-declarations only) - regex: the POSIX C API, basic/extended syntax and error_type pages - convert: the remaining pages, plus a transforming converter (other.qbk) - test_coverage.py: quickbook code blocks are indented by any amount, not 4+ spaces; inventories rescanned
Progress per library, where the implementation departs from the design, the methods used (example ports with an output oracle, loopback harness), the wrapper defects the tests found, the open CI failures and the remaining work.
…ming)
container: the `using ::operator new;` export added for the json docs
re-exports every global operator new; on clang, libc++'s aligned allocation
inside another module's instantiation (tests/statechart.cpp) became
ambiguous. The export is gcc-only now; on clang src/json.cppm names the
placement operator in its purview so it stays reachable from its GMF.
asio: the deferred examples print no-error through ex_message() ("Success";
the C library's text differs on musl, macOS and Windows); the actor benchmark
passes its token 500k times instead of 50M (it timed out on loaded CI
runners); type_erasure/main, connect_pair and the fd-passing pair are
compiled only where asio has POSIX stream descriptors / socketpair /
sendmsg, and the doc case for connect_pair skips Windows.
decimal: `near` is a macro of <windows.h> (renamed close_to); the
decimal_fast128_t -> long double check is skipped where long double has
113 bits — an upstream bug, reproduced from upstream/boost alone with
clang -mlong-double-128.
json: the doc files no longer #include <array> after the imports (MSVC STL
headers cannot be re-included after an import reached them).
…ndows case names The local/ examples (UNIX domain sockets) are compiled on POSIX systems only: Windows has AF_UNIX stream sockets but bind fails on the CI hosts (WSAEOPNOTSUPP). The pipe that stands in for stdin is POSIX-only in every network file (it returned a descriptor as int). prioritised_handlers' output depends on whether the accept has completed when the queue first runs, so it is not compared. On Windows each case's name goes to stderr before it runs, to tell which case ends a process without a report.
boost/beast/core/error.hpp declares `namespace errc = boost::system::errc;`. The generator collects entities and a using-declaration cannot name a namespace alias, so the policy re-declares it in the module. Suite: the HTTP and WebSocket examples over the loopback interface (output oracle; harness clients written with Beast: a GET of a document root's index.html, one WebSocket message), the reference pages' inline code over beast::test::stream (primer messages, body types, serializer and parser stream operations, chunked encoding with the page's chunk-callback output, the RFC 6455 handshake key/accept pair, the Body and BuffersGenerator concepts), and user types: a Body with reader and writer, Fields with a user allocator, a DynamicBuffer, and a next layer with its own teardown, async_teardown and beast_close_socket overloads. SSL examples need OpenSSL, the coroutine ones asio::spawn (Boost.Context) and two clients public internet services; they are excluded.
…sed_4 The served index.html is written in binary mode (Windows wrote \r\n and the Content-Length differed). The local-socket probe stays open like the TCP one: on macOS an accept of a connection the peer already closed fails with EINVAL and the beast async-local servers stop. tutorial/timer5 runs two threads whose timers are due at the same instant, so its output is not compared; composed_4 ends the process on windows-llvm-msvc (0xC0000374) and is POSIX-only until that is understood. The oracle reruns a program whose free port was taken in between.
…t suite boost/cobalt.hpp includes boost/cobalt/main.hpp, which defines the program's main as a friend of main_promise. From the module's GMF that definition reached every importer and clang rejected a consumer's own `int main(int, char**)`. The facade has cobalt.hpp's headers except main.hpp (policy gmf); a co_main program includes <boost/cobalt/main.hpp>. patches/cobalt.patch: use_op, use_task, this_coro::allocator / cancelled / initial and the cobalt::io protocol constants (tcp_v4, udp, local_stream, ...) were namespace-scope constexpr objects (internal linkage, not exported); they are inline constexpr. cobalt::this_coro re-exports the asio::this_coro names its using-directive brings in (policy extra). Suite: the examples (output oracle against the cobalt sources; co_main becomes a task<int> run by cobalt::run, echo_server on a free port), the documentation's code (background, benchmarks, design, primer, tour, push generator) and user types: an awaitable, an interruptible awaitable in race, a cobalt::op, move-only results through task/promise/generator/channel/join, and a user executor for cobalt::spawn.
The examples talk to broker.hivemq.com; here a small MQTT 5 broker of the test harness (CONNECT, PUBLISH at QoS 0-2, SUBSCRIBE with wildcards and retained messages, UNSUBSCRIBE, PING, DISCONNECT; TCP or WebSocket) listens on a free port passed as argv. The programs that run until SIGINT stop as on SIGINT after their first message or fifth completion. The broker's CONNACK carries a property: the client reads a property-less CONNACK's remainder with a zero-size read, which a websocket::stream completes only on the next frame. The documentation's snippets run against the same broker; user types: a stream layer with its async_shutdown, a logger, an authenticator. The TLS examples need OpenSSL and are excluded.
…xes found by the suites - geometry: GMF without adapted/boost_fusion.hpp (fusion and tuple adaptation both visible are ambiguous), with io/svg/svg_mapper.hpp; the facade imports algorithm; clang reaches the placement operator new through the facade; core drops std::size (ambiguous with boost::size) - variant.patch: name the struct in has_result_type.hpp (gcc 16 ICE on every rtree use) - interprocess.patch: creation tags, lock options, offset_type_alignment and ec_table as BOOST_INLINE_CONSTEXPR (TU-local otherwise) - math.patch: float/double/long_double_constants and the nonfinite facet flags inline constexpr; cubic_roots / line_search_policies name std::sin / fabs - config_cstdlib.patch + libs.json: boost/cstdlib.hpp in the config module, exit_success / exit_failure / ... as BOOST_INLINE_CONSTEXPR - dll: boost::dll::fs is a using-directive upstream, invisible to importers; policy extra re-declares the filesystem names the documentation uses - cobalt: sctp protocols only where IPPROTO_SCTP exists (Windows)
Official examples ported with output oracles, partitioned so that no two examples with conflicting registrations share a program; they run in the temporary directory (svg / data files). api_reference / api_index cover the documentation pages, custom_types the traits and registration macros.
Examples in four ported programs (each case restores the classic global locale; plots and data files go to the temporary directory); outputs with seeds, timings, libm digits or library message text are not compared. Excluded: examples needing GMP / quadmath / Eigen / lodepng, ./.temps/ of the upstream test tree, and double_exponential (crashes as published). Policy- macro examples and the C99/TR1 pages are gaps. doc_* cover the documentation pages; custom_types a user real type, error policy and functors.
Each doc example (libs/test/doc/examples) and libs/test/example program is a test program: the original main renamed, a harness that runs it with the documented command line and compares the output (times, source locations normalised) with a clang oracle and the expected exit status. test_coverage.py treats a file defining main as the case "program". custom_types covers the printing, dataset, fixture, precondition, formatter and tolerance points.
Seven plugin packages (kind = "shared") built next to the test programs; examples.cpp runs the importers against them, self.cpp the tutorials that import from the program itself (-rdynamic), custom_types user types across the boundary. The package is linked shared (Mach-O / PE refuse a static copy reached by the program and the plugins). tutorial8_static is a gap: this_line_location() is static inline upstream.
… spsc A doctest listener reports a case still running after 60 s (and prints case names on Windows), so a CI timeout names the hanging example. The connecting peer of the composed_* examples does not count a connection reset before any data: it landed in the backlog of the previous server's closing acceptor and left the next server waiting (composed_6 under load). cobalt's spsc passes 100 thousand values, not 100 million (the CI timeout).
The 'example coverage' job runs tools/ci/test_coverage.py check; group A of each host runs mcpp test in tests/dll and tests/test/utf. docs/architecture describes tests/<lib>/ and the two sub-projects.
- Boost.Test and dll custom types: namespace app, not user (macOS <sys/proc.h> declares struct user); numeric_limits (min)/(max) (windows.h) - geometry/api_reference: close_to, not near (Windows macro) - math examples: outputs compared on the oracle's platform (x86_64 Linux) only; elsewhere long double's width and libm digits differ - math docs: tgamma(5000) overflows in every type (aarch64's long double); the ooura cosine tolerance; daubechies_scaling with an explicit grid; naive_monte_carlo with the lambda's type, as the page writes it - asio parallel_group examples: output compared only where POSIX stream descriptors exist (an empty main elsewhere) - Boost.Test programs: normalise colour codes, 'unknown location', the system's memory-fault text and MSVC's class-key in type names; example83's std::rand output not compared; sizeof(long) and SIGALRM outputs not compared on Windows - cobalt/doc names each case on Windows (heap corruption ends the process)
- cobalt spsc: its awaitable queue races (await_suspend stores the executor and handle unsynchronised with the other thread's post); it crashed on aarch64 and can hang. Excluded with the reason. - cobalt/doc asio-awaitable / a-minimal-executor and math daubechies_scaling end the process on windows-llvm-msvc only (heap corruption / SIGSEGV, like asio's composed_4): skipped there, an open issue in the design record. - math ooura cosine: the result must lie within the integrator's own error estimate (with the values in the report)
…uble - cobalt/doc ends with heap corruption on windows-llvm-msvc at a different case each run (a-minimal-executor, race): the whole program is skipped there (a test-suite decorator), an open issue in the design record - ooura_fourier_cos<double> returns NaN on aarch64 Linux (its nodes are computed in the 113-bit long double): that check is skipped there
…n Windows yet - asio/api_overview: read the clock before constructing the timer (its expiry counts from construction; a loaded runner measured < 5 ms) - dll/self tutorial9: the function type spelt as the original's typedef (clang-cl misparsed the alias) - CI: tests/dll is not run on Windows: the package built shared does not export its compiled sources on PE (undefined boost::filesystem / boost::system symbols); open issue in the design record
gcc@16.1.0 on Linux arm64 targets aarch64-linux-musl with a static C library; a static program cannot load the plugin shared libraries.
§11 records the progress (every library's todo cleared), the deviations (dll linkage, Boost.Test as whole programs), the wrapper-layer issues the suites found, the CI timings (§9 Q3: no further test grouping) and the open issues, which move to todos/module-tests-open-issues.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
.agents/docs/2026-11-09-module-test-hardening-design.md(draft: work in progress, opened to run CI on all hosts).Wrapper changes
mcpplibs.mwrap.boost.conversion). Its headers sit at the top ofboost/, so the prefix heuristic saw no headers and the casts were exported byrange/graph.EXTRA_HEADERSintools/boost_common.pygives it its header set, ownership and include edges;pointer_cast.hppis owned bysmart_ptr.boost::core::string_viewis exported by core (its only header isboost/core/detail/string_view.hpp; no module exported the alias before).cnv::cstream,cnv::lexical_cast,cnv::strtol,cnv::printf,cnv::charconv): the converter headers are not self-contained and had been dropped by the standalone-compile filter.algorithm.patch(upstream include-guard bug hidtransform_reduce),convert.patch(throw_on_failurehad internal linkage),bimap.patch(_key/_datawere TU-local).Tests
tests/<lib>/for 34 libraries; doctest from mcpp-index (compat.doctest,[dev-dependencies]).coverage.tomlper library maps every example and documentation snippet of the 1.92.0 release to a test case or a reason;tools/ci/test_coverage.py scan|link|check.tests/dll/andtests/test/utf/are test projects of their own.Done so far: algorithm, any, bimap, bloom, callable_traits, charconv, circular_buffer, compat, container_hash, conversion, convert, core, crc, describe, dynamic_bitset, uuid, variant2. Remaining libraries and the CI steps for the sub-projects and the coverage check follow in this PR.