Skip to content

test: pin the recipe and inequality-constraint problems (strict xfail) - #374

Draft
lachlangrose wants to merge 1 commit into
masterfrom
test/recipe-and-inequality-xfail
Draft

lachlangrose wants to merge 1 commit into
masterfrom
test/recipe-and-inequality-xfail

Conversation

@lachlangrose

@lachlangrose lachlangrose commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Tests that pin two problems I found while looking for examples to write (#373). Both are strict xfails, so the suite stays green and they fail loudly (XPASS) when the problem is fixed. Independent of the other open PRs (based on master).

1. Recipes do not store a feature's build parameters

tests/unit/modelling/test_recipe_reproduces_field.py

GeologicalModel.to_recipe_json / from_recipe_json are provisional API. The existing recipe tests check feature names, the data and the stratigraphic column, but not the field. A feature entry in the recipe contains only name, kind, faults and regions, so the build parameters are lost and the rebuilt model uses the defaults.

model built with rebuilt field vs original (max abs difference, field range about 560)
defaults 0 (test passes)
nelements=2000 27
interpolatortype="PLI" 12
regularisation=10 60

API.md requires a user-facing example before a provisional symbol can be promoted to stable. An example now would imply the recipe reproduces a model, which is only true for defaults.

Diagnosis (LoopStructural/modelling/core/_model_serializer.py)

The build parameters are only the first of several problems. All were found by running a round trip.

  1. Writer (to_recipe_dict): a feature entry is name, kind, faults and a hardcoded regions: []. Nothing else.
  2. Reader (from_recipe_dict): ignores kind. Every entry is rebuilt with create_and_add_foliation, so a fault comes back as a plain foliation (in a small fault model the strata change by 0.21 on a field of range 10 and fault displacement by 0.49).
  3. feature_recipe_kind: tests isinstance(..., GeologicalFeature) and StructuralFrame before the subclasses, so a fault is written as structural_frame (the "fault" branch is unreachable, FaultSegment subclasses StructuralFrame) and an unconformity as foliation.
  4. Models with an unconformity cannot be rebuilt at all: from_recipe_json raises ValueError: Could not recreate feature '__strati_unconformity'.
  5. group_mapping of the stratigraphic column is lost ({'Group_0': 'strati'} becomes {}); unit order survives.
  6. A feature built with data= (not taken from model.data) is rebuilt from model.data filtered by feature_name: a half-size subset gives a difference of 20.5.
  7. Where the parameters are: feature.builder.build_arguments holds regularisation, damp, solver, tol, etc., but its nelements is the realised element count (17,732 for a request of 10,000), and interpolatortype is not in it (it is on the interpolator). Replaying build_arguments as-is is wrong (differences 6 to 25); replaying them with the originally requested nelements reproduces the field exactly (0.0 in all four cases I tried). The requested nelements is not retained anywhere today.

Not tested: fold frames, folded foliations (their fold functions are callables, which JSON cannot hold), intrusions, domain faults.

Suggested fix (not in this PR)

All creation funnels through GeologicalModel.create_and_add_feature(feature_type, name, **params), so record the requested JSON-safe parameters there (interpolatortype, nelements, tol, regularisation, solver, ...), write them in the recipe, and have the reader replay create_and_add_feature(kind, name, **params) in order, resolving faults by name. Also write group_mapping. Until a kind is supported, to_recipe_dict should raise a clear error rather than write an entry that rebuilds wrongly.

2. Inequality constraints

packages/loop_interpolation/tests/test_inequality_enforcement.py

Documented meaning: lower <= f(x, y, z) <= upper. The scenario is a sparse dome model (3 contacts, 2 orientations) with 24 one-sided bounds on the 0.5 horizon that the data alone get wrong 4 times.

test result
control: without bounds, some bounded points are on the wrong side passes
ADMM with nmajor=10, admm_weight=0.01, maxiter=50 (the existing builder test's arguments) enforces every bound passes (0 of 24 violated, and the error against the truth falls from 0.134 to 0.088)
ADMM with its default arguments enforces the bounds xfail: 4 of 24 violated, worst 0.056, essentially the unconstrained solution
with solver cg, supplying inequality data warns that they are not used xfail: they are dropped and nothing is logged

The cg test is an expectation, not a documented contract, and the reason string says so. For the inequality problem the cause is not diagnosed: I only know that ADMM's result depends on its arguments (using only some of the three above also leaves the bounds violated). I checked that caplog does capture this library's log records, so the cg test can XPASS once a warning exists.

Correction

My earlier note on #373 said inequality constraints "did not change the solution" with ADMM. That was only for default arguments; I had also used a different call order from the existing test. With the existing test's flow and arguments they do work, as the passing test above shows.

Checks

Each xfail was run with --runxfail and fails for the stated reason (numbers above). ruff clean.

🤖 Generated with Claude Code

Recipe: to_recipe_json / from_recipe_json do not store a feature's build parameters
(nelements, interpolatortype, regularisation), so a model built with anything but the defaults
rebuilds with the defaults and gives a different field (differences of 12 to 60 on a field of
range about 560). The default-parameter case passes; the three others are strict xfails.

Inequality: ADMM enforces the bounds with the arguments the existing builder test uses, but
with its default arguments returns essentially the unconstrained solution (4 of 24 bounds
violated, worst 0.056), and with solver cg the inequality data are dropped without a warning.
A control (the unconstrained solution does violate bounds) and the working ADMM configuration
pass; the default-ADMM case and the cg-warning expectation are strict xfails.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant