Repository navigation
Special, Stats, Integrate, Optimize: go-numerics and go-statistics as cgo-free scipy-style modules - #819
Merged
Conversation
Ruby gets scipy.special's function set with no SciPy and no cgo: 67 of the module's 67 exported functions, under scipy's own names. The names are taken from the correspondence the Go module MEASURES itself against -- bench/scipy_server.py and testdata/gen/fam_*.py pair each Go function with the scipy function it is compared to in ULPs -- rather than transliterated from the Go name. That is why Log1pExp is #softplus and not #log1pexp, which is what guessing would have produced, and why LogGamma is #gammaln. The two functions with no scipy counterpart keep the Go name: #log_factorial and #loggamma_sign. Each binding is a ufunc, as scipy's is. The module ships both a float64 form and an element-wise form over *ndarray.Array -- in two packages, since Go cannot hold two functions of one name -- and rbgo already binds go-ndarray at the same v0.11.0. So one Ruby name serves both: a Float takes the scalar path, and as soon as one argument is an NDArray every argument goes through go-ndarray, a Float becoming a 0-d array, which broadcasts against any shape. Special.gamma(5.0) answers a Float, Special.gamma(arr) an NDArray of the same shape, and Special.xlogy(2, arr) broadcasts. Nothing here reimplements the module: the element-wise path calls its own nd package, which parallelises and broadcasts already, and argument checking reuses NDArray's helpers (ndArgc, ndFloat, ndInt, mustArray, ndRaise). Verification is against values that are mathematical facts, not against the Go functions being bound -- comparing a binding to the code it calls proves only that the call happened. 56 closed forms (gamma(5)=24, zeta(2)=pi^2/6, digamma(1)=-euler_gamma, ellipk(0)=pi/2, xlogy(0,0)=0, ...), the five non-finite answers scipy gives outside a domain, jn_zeros against the tabulated zeros of J0 AND against j0 vanishing there, and the property that the scalar and element-wise paths agree -- the one part of this that is not the module's own code -- at every arity. 98 subtests. Three negative controls, each checked to have really changed the file: binding #softplus to Logit, pointing #gamma's array path at LogGamma, and dropping an arity check all turn the suite red. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tions
Ruby gets scipy.stats's univariate distributions with no SciPy and no cgo: all
32 the Go module has Continuous or Discrete constructors for, under scipy's own
names, with scipy's argument order and scipy's method set.
n = Stats.norm(0.0, 1.0)
n.ppf(0.975) # => 1.9599639845400538
Stats.poisson(3.0).pmf(2) # => 0.22404180765538775
Stats.t(10.0).cdf(arr) # element-wise over an NDArray
As with Special, the names are the ones the module MEASURES itself against:
each distribution's doc comment states its scipy.stats equivalent and the
fixtures are generated from it. So StudentT is Stats.t, ChiSquared is
Stats.chi2, DiscreteUniform is Stats.randint, Weibull is Stats.weibull_min,
Gumbel is Stats.gumbel_r and NoncentralChiSquared is Stats.ncx2 -- none of
which transliteration would have produced.
The method set is not invented either. The module's Continuous and Discrete
interfaces name their scipy counterpart method by method, and differ only in
the density, so #pdf/#logpdf and #pmf/#logpmf are defined per class and the
other seventeen methods once, for both, through a Go interface holding exactly
what the two share. Every per-point method is a ufunc, over a Float or an
NDArray, through the module's own stats.Map.
#rvs answers a Float with no size and an NDArray with one, as scipy's does.
Without seed: it draws from one generator per VM that advances across calls --
scipy's module-level random state -- and seed: asks for a fresh reproducible
stream.
A parameter outside its domain is refused at CONSTRUCTION, with the module's
own message, as an ArgumentError where SciPy raises ValueError.
Verification is against mathematical facts, not the code being bound: 57 closed
forms covering all 32 distributions -- each constructor is written out
separately in scipy's order, so a table of known moments is what catches a
swapped parameter. norm.ppf(0.975)=1.96, norm.entropy=(1/2)log(2 pi e),
poisson.var=mean, binom(10,.5).pmf(5)=252/1024, zipf(2).pmf(1)=1/zeta(2),
randint(0,10).var=8.25, gumbel_r.mean=euler_gamma, rayleigh.mean=sqrt(pi/2),
f(5,10).mean=1.25, ncx2(2,3).mean=5, ... Plus 42 identities checked across four
continuous and two discrete laws (sf=1-cdf, ppf inverts cdf, std^2=var,
interval covers its confidence), loc/scale behaving as scipy says, the ufunc
property at every method, the method SET of each class, a 20000-draw check that
#rvs really draws from the law asked for (within 6 standard errors), and the
parameter and arity errors.
A README guard is added with them: internal/vm/{special,stats}.go register
their Ruby names in string literals and the README lists them one by one, so
the two sets must be equal in both directions. Writing it found two defects in
the README text committed minutes earlier -- the Special entry claimed all 67
functions while omitting log_factorial and loggamma_sign from its list, and it
backticked log1pexp and log_gamma as counter-examples, which reads as a
promise. Both fixed; the counter-examples are now italic, because a name that
deliberately does not exist is not code.
95 Stats subtests, 193 with Special. Seven negative controls, each checked to
have really changed the file: swapping loc and scale, pointing Stats.t at
ChiSquared, wiring #pdf to CDF, dropping a shared method, ignoring seed:,
drawing from the wrong law, and skipping parameter validation all turn the
suite red. The README guard was broken in all three of its directions too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ys checked Two gaps in the tests just added, both found by measuring rather than reading. A census over the live VM -- every method of every numeric module called with zero arguments, classified by its message -- says Special defines 66 methods, not the 67 the module exports. The two counts differ in one place and it is scipy's: the Riemann and Hurwitz zeta are ONE scipy name with an optional second argument, so Zeta and HurwitzZeta meet in #zeta. Nothing asserted that, so a function added to the module and forgotten here, or a name quietly dropped, would have gone unnoticed. TestSpecialCoversEveryFunction pins the 66 names and the reason there are 66. The same census found a defect in the modules NEXT DOOR, filed as #820: all 17 of FFT's methods and 14 of Image's index args[0] with no arity check, so FFT.fft() answers "ArgumentError: fft: runtime error: index out of range [0] with length 0" -- a Go fault converted into a Ruby error, where MRI says wrong number of arguments. NDArray is clean because it uses ndArgc, which is what Special and Stats use. So the census becomes the test, for these two modules. It is better than a list because a method added later is covered without being named, and it asserts the number of methods WALKED as well as the empty result, so a census that reads nothing fails rather than passing -- 66 for Special, 70 for Stats (32 distributions and 19 methods on each of the two classes). Verified to bite: removing the arity check from Special's one-argument helper makes it name the exact 33 methods that helper registers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ll four
The request named go-numerics, which is THREE modules. Special covered one;
these are the other two, and with them the four symbols the notebook census
ranks highest inside scipy are bound: integrate.quad, optimize.minimize,
minimize_scalar and brentq.
Integrate.quad(0.0, 1.0) { |x| x * x } # => [0.333..., 3.7e-15]
Integrate.quad(0.0, Float::INFINITY) { |x| Math.exp(-x) }
Optimize.brentq(0.0, 2.0) { |x| x * x - 2 } # => 1.4142135623730951
Optimize.minimize([-1.2, 1.0]) { |v| 100*(v[1]-v[0]**2)**2 + (1-v[0])**2 }
The function is a BLOCK, which is how Ruby passes a callable and which takes
&f for a Proc or lambda with no further surface -- so there is no second
spelling to document or test. Return shapes are scipy's under one rule: one
number is a Float, a tuple an Array (quad is (value, abserr), curve_fit is
(popt, pcov)), and an OptimizeResult or RootResults a Hash with symbol keys,
which full_output: true asks a root finder for.
Verified against integrals and roots known exactly, not against the Go code:
16 definite integrals including the Gaussian integral over the whole line, an
integrable singularity, and the Gauss-Legendre exactness property with the
two-point rule's +/-1/sqrt(3) nodes; five root finders on sqrt(2) and the
Dottie number, each checked against the known value AND by evaluating the
function at the answer; Rosenbrock through all five minimisers, whose minimum
is exactly (1,1) so descending somewhere is not enough; curve_fit recovering
its own model's parameters from exact data. Every keyword is tested in the
"given" direction with an OBSERVABLE effect -- points: naming a kink,
divmax: 1 producing the one reachable non-convergence, a callable inner limit
distinguishing the triangle's 1/2 from the square's 1, bounds: binding
minimize to 1 where the free minimum is 5.
The first push failed CI on the per-function coverage ratchet, not on a test.
Recording the five functions in BELOW100 was the wrong fix, and the right one
needed three different answers:
- untested: Truthy on both distribution classes (only a boolean context
calls it) and factorial2 on an array -> tests.
- unreachable, so DELETED: an ErrFit arm with no Fit* function bound;
cycle_ier, which is only non-nil for the QAWF weight that is not bound;
nd.LogSumExp's error, which reports a COMPLEX array where rbgo's NDArray
is float64 only. Each was a branch nothing could reach, and BELOW100 would
have preserved it as a promise. #logsumexp now calls the root function
with the slice it already holds: fewer layers and no dead arm.
- looked unreachable and was NOT: #rvs's error arm. I validate negative
sizes myself, so I was about to call it unreachable; measuring instead,
Stats.norm.rvs(2 ** 40, 2 ** 40) answers "shape is too big", refused by
go-ndarray at validation without allocating. It has a test.
Twenty-one inline `if err != nil { raiseNumerics(err) }` arms are now one
raiseIf, which the refusals that ARE reachable cover -- twenty-one
half-covered functions became one covered branch. The rewrite that did it
matched its own helper and briefly made raiseIf call raiseIf: infinite
recursion that compiled and that go build accepted without a word. Caught by
counting both sides of the replacement; the helper is now shaped so the
pattern cannot match it.
All four files are at 100% of statements, located with covblocks, which names
the blocks go tool cover -func rounds away.
The README guard now covers all four modules, with each entry's non-method
words named beside their reason (scipy's loc/scale, the method: values, the
block-pass idiom) rather than a shared list that would also excuse a real
omission. It was broken in three directions again and went red for each.
go-numerics/integrate and /optimize stay at v0.1.0 deliberately: v0.2.0 is
performance work whose own notes say results are unchanged to the last bit,
AND a bump to go-ndarray v0.12.0, which Go would then apply to the whole
numeric stack. Filed separately with the measurement.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… absorbed The rbgo_core lane failed on its size bound: the core wasm module is 27 393 739 bytes against a 27 000 000 bound. The four scipy-style modules cost 4.2 MB between them, and the bound had 3.8 MB of headroom. A concurrent session reached this first and did the better experiment -- each module added to origin/main ALONE rather than cumulatively down a branch, which attributes the cost instead of ordering it -- and raised the bound to 29 000 000 with the table in ci.yml. Its argument is why that is right rather than indulgent: the size bound exists to notice growth NOBODY INTENDED, while a gem coming back is caught by the module-path check two lines below, by name and exactly. Growth that is intended, measured and attributed is what should move a bound; growth that is none of those is what the bound is for. I had implemented the other answer -- Stats behind `!rbgo_core`, joining Image outside the core build, which lands at 24 987 852 and needs no new bound. It is dropped here. It cost the browser build the single most-used part of scipy (stats.norm leads the notebook census), and it was the conservative choice rather than the argued one. What I did add is the reconciliation, because two independent measurements of the same thing did not agree to the byte and a reader should not have to find that out. "Special added to main alone" came out at +590 200 for me against +631 646 there -- 7% apart -- from baselines that themselves differ by 411 bytes between two worktrees of the same commit. So ci.yml now says to read the SHARES as measured and the exact bytes as one run's: Stats over half, Special under a fifth, the four together ~4.2 MB. Verified independently, every figure in the README's core table, to the digit: 28 081 523 unstripped (28.1 MB), 27 393 739 stripped (27.4 MB), 6 418 318 gzip -9 (6.4 MB), 47 modules. Also checked, and NOT a defect: the core build links 28 go-ruby-* modules while the README says "no gem". All 28 are standard-library bindings -- the rows the capability table calls "preloaded by MRI" and "small standard libraries" -- except go-ruby-timecop, which vm.go imports for the VM's own clock rather than as a registered gem; gems.go, which registers the Timecop module, is tagged out as it should be. Builds, vets and tests green under both tags. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…is attributed
The core build grew past its bound: 27_393_387 bytes against 27_000_000. The
gate was right to stop it, and raising the number without saying what bought
the growth would have turned a budget into a formality.
So the cost is measured, each module added to origin/main alone (js/wasm,
-s -w, 2026-10-17):
Special +631_646 15.0% go-numerics/special
Stats +2_416_448 57.3% go-statistics/stats
Integrate +238_387 5.6% go-numerics/integrate
Optimize +933_406 22.1% go-numerics/optimize
---------
+4_219_887 23_173_500 -> 27_393_387
Stats is more than half of it, which is what 36 distributions and 27_599 lines
of Go cost; the four bindings themselves are ~1_500 lines. The attribution sits
in ci.yml beside the bound it moved, so the next person reads why rather than
guessing.
The new bound is 29_000_000 -- about 1.6 MB above the measurement. It does not
restore the original proportional headroom (3.8 MB on 23.1 MB) on purpose: this
number only has to notice growth nobody intended, and a gem coming back is
caught by the module-path check in the same step, by name and exactly. That
check was run against this branch and passes: 47 modules linked, none on the
list, with the six numeric ones (go-ndarray, go-fft, go-numerics/{special,
integrate,optimize}, go-statistics/stats) present as intended.
The first attribution attempt was wrong and said so plausibly: a perl
`s/.../ if !$d++` applies to the first LINE of a file, not the first match, so
no register call was inserted and four builds reported deltas of 37 KB, 6 KB
and 60 bytes while the total sat 4.2 MB below the branch. Redone with the
insertion verified at each step, the figures close to within 344 bytes of the
branch's own build.
README's wasm table is re-measured in full rather than patched in one cell --
both columns moved, and the default build grew by the same 4.2 MB:
unstripped 102.6 -> 107.3 MB | 23.7 -> 28.1 MB
stripped 99.9 -> 104.5 MB | 23.1 -> 27.4 MB
stripped, gzip -9 22.3 -> 22.2 MB | 5.7 -> 6.4 MB
Go modules 253 -> 256 | 44 -> 47, no gem
and the table is now dated, with a line pointing at the attribution.
core.go's statement of what core IS names the four modules too: it listed the
numeric stack as "NDArray and FFT", which was the whole of it when written.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… is the shared code a20114f's comment, which I wrote, said the per-module figures were "each module added to origin/main alone". They were not: the loop that produced them kept the previous module in place, so each delta is the MARGINAL cost in that order. The peer who reviewed it measured Special in isolation, got +590_200 against my +631_646, and could only report the two as 7% apart from an unstable baseline. They were not noisy. They are two different quantities, and measuring both settles it -- isolated, one worktree, one baseline, 2026-10-17: one module added in turn alone down the branch Special +590_255 +631_646 Stats +2_685_070 +2_416_448 Integrate +258_839 +238_387 Optimize +965_130 +933_406 ---------- ---------- +4_499_294 +4_219_887 +590_255 reproduces the peer's +590_200 to 55 bytes, so the 7% was the method, not the measurement. And neither column is "the" per-module cost, which is the part worth writing down: - ISOLATED overcounts. The four share go-ndarray, the special/nd layer and the argument helpers, so each pays for the same code. The sum exceeds the real +4_220_231 by 279_063 -- and that residual IS the shared part, which is a figure neither method reports on its own. - CUMULATIVE undercounts all but the first: whoever goes first pays for the shared code and the rest ride free. It depends on the order. What both agree on is the ranking and the total, so the comment now says to read those: Stats ~60%, Optimize ~21%, Special ~13%, Integrate ~6%, together ~4.2 MB. One more thing the comparison turned up: the build is NOT byte-reproducible across worktrees. The same commit gave 23_173_500 in one and 23_172_890 in another, 610 bytes apart, which is why the peer's baseline and mine differed and why every figure above comes from a single worktree. A cross-worktree delta of a few hundred bytes means nothing. The bound stays at 29_000_000 and the module-path check is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…asure everything #821 landed the upgrade #822 asked for: go-ndarray v0.12.0, whose Repr prints what NumPy prints, with rbgo's `array([])` expectation corrected to `array([], dtype=float64)`. That removes the only reason this branch pinned go-numerics/{integrate,optimize} at v0.1.0 -- their v0.2.0 required v0.12.0 and would have dragged the whole numeric stack with them. They are now at v0.2.0, two releases of performance work whose own notes claim results unchanged to the last bit. That claim is checked here rather than taken: the 113 closed forms, 42 identities and every keyword test pass unchanged on the new pair. Nothing in the four bindings' tests depends on an array repr, so v0.12.0's printing rework touches none of them. Every size figure is re-measured, because all of them were taken on v0.11.0 and v0.1.0: was (v0.11.0) now (v0.12.0 + v0.2.0) baseline (main) 23_172_890 23_195_192 this branch 27_393_739 27_479_576 delta +4_220_231 +4_284_384 headroom to bound 1_606_261 1_520_424 The attribution too, both columns, same worktree: one module ALONE added IN TURN Special +591_000 +591_000 Stats +2_685_365 +2_438_092 Integrate +267_470 +266_797 Optimize +1_020_511 +987_989 ---------- ---------- +4_564_346 +4_283_878 Two things in that table are the model confirming itself. Special is IDENTICAL in both columns, which is what "whoever goes first pays for the shared code" predicts for whoever goes first. And the isolated sum overcounts the real +4_284_384 by 279_962, against 279_063 for the same pair of measurements on v0.11.0 -- 899 bytes apart across a dependency upgrade, so the shared-code residual is a property of the code and not of one run. The bound stays at 29_000_000: the growth moved by 64_153 bytes and the headroom is 1.5 MB. README's wasm table is re-measured in full and re-dated (107.4 / 28.2 unstripped, 104.5 / 27.5 stripped, 22.2 / 6.4 gzipped, 256 / 47 modules). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tannevaled
force-pushed
the
wave70-special
branch
from
October 18, 2026 06:52
7f8b731 to
109fb95
Compare
The module comment and the README both said the Ruby names "come from the
correspondence the Go module measures itself against". Auditing that sentence
rather than repeating it: 63 of special's 67 functions carry such a pairing in
bench/scipy_server.py or testdata/gen/fam_*.py, and FOUR do not. For those the
name is my reading of scipy, which is a weaker claim and is now made as one.
GammaSign -> gammasgn scipy has it; the module pairs nothing
Kn -> kn scipy has it; unpaired here
In -> in scipy has NO `in`
LogGammaSign -> loggamma_sign no scipy counterpart at all
LogFactorial is a fifth of the kind, and the module's own fixture says so in
words: "log n! (SciPy: gammaln(n+1))".
So Special.in is a name I INVENTED that merely looks like it came from a spec.
That is the failure worth naming: a plausible name sitting inside an otherwise
measured table inherits the table's authority. scipy cannot have an `in` --
it is a Python keyword -- and iv(0, x) is the faithful spelling.
The three relationships those names stand on are asserted now instead of
asserted in prose: in(n, x) == iv(n.to_f, x) EXACTLY, the same for kn against
kv, and gammasgn against the sign of gamma itself on the intervals where it
alternates. Exact rather than approximate, because they are the same Go call
and a tolerance would hide a wrong conversion.
The audit's first answer was wrong, by my own instrument, and it is worth
recording how: I extracted the paired fixture names with a pattern anchored on
exactly four spaces of indentation. fam_bessel.py indents its table by eight,
so fourteen Bessel functions looked unpaired and the count came out 49/18
instead of 63/4. The Go program doing the join was correct and faithfully
reported my filter's view. The answer was implausible against a grep I had
already read, and `^[[:space:]]+` would have cost nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ruby gets scipy's special functions, distributions, quadrature and optimisers with no SciPy and no cgo — four modules of two organisations, under scipy's own names:
Specialgo-numerics/specialStatsgo-statistics/statsIntegratego-numerics/integrateOptimizego-numerics/optimize66Ruby names for67Go functions, because scipy spells the Riemann and Hurwitz zeta as onezeta(x, q = nil).The names are measured, not transliterated
Every module states its own specification: it compares itself against scipy in ULPs, and the pairing is a file in its repository —
bench/scipy_server.pyandtestdata/gen/fam_*.pyforspecial, each distribution's doc comment forstats. Those files are the bibliography, so the Ruby names come from them rather than from the Go name:Log1pExpsoftplusLogGammagammalnGammaP/GammaQgammainc/gammainccStudentTtChiSquared/NoncentralChiSquaredchi2/ncx2DiscreteUniformrandintWeibull/Gumbelweibull_min/gumbel_rTwo functions have no scipy counterpart and keep the Go name:
log_factorial,loggamma_sign.One Ruby name, two Go paths — which is what a ufunc is
specialships each function twice: afloat64form ingo-numerics/specialand an element-wise form over*ndarray.Arrayingo-numerics/special/nd. They cannot share a package, because Go cannot hold two functions of one name — which is why there are two import paths here. rbgo already binds go-ndarray, so the array type is shared rather than introduced (see Dependency versions below for which version, and why it moved).float64function, answering a Float;NDArray→ every argument goes through go-ndarray, a Float becoming a 0-d array, which broadcasts against any shape.Measured before being written:
Scalar(2) × shape{3},shape{3} × Scalar(2), ternary, 2-D shape preservation.Stats's per-point methods are ufuncs the same way, through the module's ownstats.Map.Nothing is reimplemented. The element-wise path calls the module's own
ndpackage, which already parallelises and broadcasts; argument checking reusesNDArray's helpers (ndArgc,ndFloat,ndInt,mustArray,ndRaise,ndKw); and the seventeen methodsStats::ContinuousandStats::Discreteshare are defined once, against a Go interface holding exactly the part of the module's two interfaces that is the same. The two Ruby classes differ only in the density —pdf/logpdfagainstpmf/logpmf— which is the only place the Go interfaces differ.#rvsanswers a Float with no size and anNDArraywith one, as scipy's does; withoutseed:it draws from one generator per VM that advances across calls (scipy's module-level random state), andseed:asks for a fresh reproducible stream. A parameter outside its domain is refused at construction, as anArgumentErrorwhere SciPy raisesValueError.Verification: facts, not the code being bound
Comparing a binding against the Go function it calls proves only that the call happened. So every expected value is a mathematical fact:
gamma(5)=24,gamma(0.5)=sqrt(pi),zeta(2)=pi^2/6,digamma(1)=-euler_gamma,polygamma(1,1)=pi^2/6,ellipk(0)=ellipe(0)=pi/2,beta(2,3)=1/12,xlogy(0,0)=0,betainc(1,1,x)=x, inverse pairs round-tripping,j0(0)=1…loc-then-scaleorder, so a table of known moments is exactly what catches a swapped or dropped parameter:norm.entropy=(1/2)log(2 pi e),poisson.var=mean,binom(10,.5).pmf(5)=252/1024,zipf(2).pmf(1)=1/zeta(2),randint(0,10).var=8.25,gumbel_r.mean=euler_gamma,rayleigh.mean=sqrt(pi/2),f(5,10).mean=1.25,ncx2(2,3).mean=5,hypergeom(10,5,3).mean=1.5…sf=1-cdf, the logs are logs,ppfinvertscdf,isf(q)=ppf(1-q),std^2=var,median=ppf(0.5),interval(0.95)covering 0.95 of the mass. These hold for any distribution, so a method wired to the wrong Go function breaks one without breaking the others.gamma(0)=Infinity,logit(2)=NaN…), which a binding easily turns into a Ruby error or a zero;jn_zerosagainst the tabulated zeros of J0 and againstj0vanishing at the first one — a check on the pair, not on either alone;#rvsdraws from the law asked for — 20000 seeded draws fromnorm(10, 2)with a sample mean and sd within 6 standard errors, which is a check and not a flake;Specialdefines exactly 66 names andStatsexactly the 32 the module has univariate constructors for, so a function added upstream and forgotten here fails rather than going unnoticed;Ten negative controls on the bindings, each verified to have really changed the file: binding
softplustoLogit, pointinggamma's array path atLogGamma, dropping an arity check (which makes the census name the exact 33 methods affected), swappinglocandscale, pointingStats.tatChiSquared, wiring#pdftoCDF, dropping a shared method, ignoringseed:, drawing from the wrong law, and skipping parameter validation — all turn the suite red.A README that cannot drift
The README lists these names one by one, which is the claim that goes stale silently.
readme_numeric_surface_test.goreads the names the four bindings register and the names the README lists and requires the sets to be equal in both directions, with each entry's non-method words named beside their reason (scipy'sloc/scale, themethod:values, the block-pass idiom) rather than one shared list that would also excuse a real omission.Writing it found two defects in the README text committed minutes earlier: the Special entry claimed all 67 functions while omitting
log_factorialandloggamma_signfrom its list, and it backticked log1pexp and log_gamma as counter-examples — which reads as a promise. Both fixed; the counter-examples are italic now, because a name that deliberately does not exist is not code. The guard was then broken in all three of its directions (a name dropped from the list, askellampromised that does not exist,#entropyremoved from the binding) and went red for each, and again for the two new modules.A side finding, filed separately
A census over rbgo itself found 31 methods — all 17 of
FFT's and 14 ofImage's — that indexargs[0]with no arity check, soFFT.fft()answersArgumentError: fft: runtime error: index out of range [0] with length 0: a Go runtime panic converted into a Ruby error, where MRI says wrong number of arguments.NDArrayis clean because it usesndArgc, which is what all four modules here use. Filed as #820 with the full table;TestSpecialArityIsAlwaysChecked,TestStatsArityIsAlwaysCheckedandTestNumericsArityIsAlwaysCheckedkeep these four modules at 0, as a census rather than a list, so a method added later is covered without being named.Second half:
IntegrateandOptimizeThe request names
go-numerics, which is three modules.Specialabove is one; these are the other two, and together they are the four symbols the notebook census ranks highest inside scipy —integrate.quad,optimize.minimize,minimize_scalar,brentq.The function is a block. That is how Ruby passes a callable, and it takes
&ffor a Proc or lambda with no further surface — so there is no second spelling to document or test. A function of several variables receives one Array;dblquad's inner limits may be numbers or callables, as scipy allows.Return shapes are scipy's, under one rule: one number → a Float; a tuple → an Array (
quadis(value, abserr)andcurve_fitis(popt, pcov), both destructuring as scipy's tuples do); anOptimizeResultorRootResults→ a Hash with symbol keys, whichfull_output: trueasks a root finder for.Verified against integrals and roots that are known exactly
x²over [0,1] = 1/3,sinover [0,π] = 2,1/xover [1,e] = 1,e^-xover [0,∞) = 1, the Gaussian integral √π over (−∞,∞),1/√xover [0,1] = 2 — an integrable singularity — the Gauss-Legendre exactness property (an n-point rule is exact to degree 2n−1), the two-point rule's nodes ±1/√3 with weight 1, the sampled rules on data where they are exact, and the unit square and a 1×2 rectangle.newtonwithfprime:answers1.4142135623730951exactly.curve_fiton data lying exactly on its model recovers the model's own parameters;approx_fprimeon a linear objective recovers its constant gradient;bracketis checked by the property that its three points bracket a minimum, not against a fixed triple.points:naming a kink,divmax: 1producing the one reachable non-convergence (aRuntimeErrornaming DivMax, where SciPy warns), a callable inner limit distinguishing the triangle's 1/2 from the square's 1,bounds:bindingminimizeto 1 where the free minimum is 5,fprime2:(Halley) andx1:(the secant's second point), andmethod:as a Symbol.What the coverage gate asked for, and what it found
The first push failed CI — not the tests, which were green, but the per-function coverage ratchet: five functions newly below 100%. Recording them in
BELOW100was the wrong fix, and working out why took three different answers:Truthyon both distribution classes (only a boolean context calls it) andfactorial2on an array: tests.ErrFitarm when noFit*function is bound;cycle_ierinquad's full record, which is only non-nil for the QAWF weight that is not bound;nd.LogSumExp's error, which reports a complex array when rbgo'sNDArrayis float64 only. Each was a branch nothing could reach, whichBELOW100would have preserved as a promise.#logsumexpnow calls the root function with the slice it already holds — fewer layers and no dead arm.#rvs's error arm. I validate negative sizes myself, so I was about to call it unreachable; measuring instead,Stats.norm.rvs(2 ** 40, 2 ** 40)answers "shape … is too big", refused by go-ndarray at validation without allocating. It has a test.Separately, twenty-one inline
if err != nil { raiseNumerics(err) }arms are now oneraiseIf, which the refusals that are reachable cover — twenty-one half-covered functions became one covered branch. The rewrite that did it matched its own helper and briefly maderaiseIfcallraiseIf: infinite recursion that compiled and thatgo buildaccepted without a word. Caught by counting both sides of the replacement, and the helper is now shaped so the pattern cannot match it.All four files are at 100% of statements with
covblocks, which names the blocksgo tool cover -funcrounds away. That figure is from a plaingo test -coverprofilerun: the CI-equivalent measurement (-race,-coverpkg ./internal/...) was started and killed — this machine was carrying 30 concurrent agent sessions and a load average of 70, and both the test binary and the ratchet died withsignal: terminated. So the-raceratchet here is CI's word, not mine; the lane result on this PR is the measurement.Dependency versions, and what resolved them
This branch originally pinned
go-numerics/{integrate,optimize}at v0.1.0 on purpose: their v0.2.0 requiredgo-ndarrayv0.12.0, and Go resolves to the maximum of all requirements, so taking it would have moved go-ndarray underNDArray,FFT,SpecialandStatsat once. That is a dependency decision with its own witness, not a side effect of a binding, so it was filed as #822 rather than smuggled in — with the evidence that rbgo'sNDArray.zeros(0)expectation ofarray([])is wrong on either version, since go-ndarray v0.12.0 ports NumPy 2.5.3'sarrayprint.pyand its expectations are copied verbatim from NumPy's own test suite, wherenp.zeros(0)isarray([], dtype=float64).#821 then landed exactly that, and #822 is closed. So the reason to hold back is gone, and this branch is rebased onto it:
go-ndarray/ndarraygo-numerics/{integrate,optimize}go-numerics/special,go-statistics/statsThe two v0.2.0 releases claim results unchanged to the last bit. That claim is checked here, not taken: the 113 closed forms, the 42 identities and every keyword test pass unchanged on the new pair, and nothing in the four bindings' tests depends on an array repr, so v0.12.0's printing rework touches none of them.
credscan0 over 37 directories walked;govulncheck0 reachable across./.... The gitsafe and agentsync binaries and both global hooks were checked at v0.10.0/v0.5.0, matching their repositories.What the
rbgo_corelane said, and the measurement it forcedThe core wasm build grew past its bound — 27,479,576 bytes against 27,000,000. The gate was right to stop it, and raising the number without saying what bought the growth would have turned a budget into a formality. So the cost is attributed, and the two obvious ways to attribute it disagree, which turned out to be the useful part:
SpecialStatsIntegrateOptimizeTwo rows of that table are the model confirming itself:
Specialis identical in both columns — which is what "whoever goes first pays for the shared code" predicts for whoever goes first.special/ndlayer, the argument helpers) — a figure neither method reports on its own. The same pair of measurements on go-ndarray v0.11.0 gave 279,063: 899 bytes apart across a dependency upgrade, so the residual is a property of the code and not of one run.Neither is "the" per-module cost. What both agree on is the ranking and the total, which is what the comment in
ci.ymlnow says to read:Stats~59% (36 distributions, 27,599 lines of Go),Optimize~22%,Special~13%,Integrate~6% — against ~1,500 lines of binding.An earlier revision of that comment, which I wrote, presented the cumulative column as "each module added to origin/main alone". It was not, and a reviewing session that measured
Specialin isolation could only report the two as 7% apart on an unstable baseline. They were not noisy — they were two different quantities.One more thing that comparison turned up: the build is not byte-reproducible across worktrees. The same commit gave 23,173,500 in one and 23,172,890 in another, 610 bytes apart. Only same-worktree comparisons mean anything, so every figure above comes from one.
The bound is 29,000,000, ~1.5 MB above the measurement. It deliberately does not restore the original proportional headroom (3.8 MB on 23.1 MB): this number only has to notice growth nobody intended, and a gem coming back is caught by the module-path check in the same step, by name and exactly. That check was run against this branch and passes — 47 modules linked, none on the list, with the six numeric ones present as intended. CI's
rbgo_corelane is green on the rebased head.A concurrent session had implemented the other answer —
Statsbehind!rbgo_core, joiningImageoutside the core build, landing at 24,987,852 with no new bound — and dropped it, because it would cost the browser build the single most-used part of scipy (stats.normleads the notebook census). That trade is recorded in the branch history rather than left implicit.README's wasm table was re-measured in full rather than patched in one cell, because both columns moved, and it is dated:cmd/wasm(js/wasm)rbgo_coregzip -9core.go's statement of what core is names the four modules too; it listed the numeric stack as "NDArray and FFT", which was the whole of it when written.Documentation
docs#27 adds a section per module to the site's scientific-stack page — and a
tools/docexampleguard that runs every Ruby example on the site and checks it against the answer written beside it, which nothing did before. It reports 11 examples / 25 claimed answers / 0 disagreements against an rbgo built from this branch, and correctly refuses the four new sections against today'smain, naming each missing module. It merges after this PR, which is why.