Descriptives, tests and the classical toolbox — the 0.23.0 conventions

Part of the numerical-validation suite. The definitions the convention inventory found implicit — chosen in the engine, matched by a reference, but never written down as a choice — written down once. Each section names the definition ChakataStat prints, the reference packages that agree with it, and the alternative a reader may meet elsewhere.

Every section here was written on 2026-08-28 for 0.23.0 item 9 from the engine source, not from memory: the file and function each cites is where the number comes from. Where a pin in the validation suite exercises the choice, it is named.

Descriptives

Skewness and kurtosis

ChakataStat prints the bias-adjusted sample skewness and excess kurtosis — the G₁ and G₂ of Joanes & Gill (1998):

G₁ = √(n(n−1)) / (n−2) · g₁          g₁ = m₃ / m₂^{3/2}
G₂ = (n−1) / ((n−2)(n−3)) · ((n+1)·g₂ + 6)    g₂ = m₄ / m₂² − 3

with m_k the central moments about the mean with divisor n (explore_row in descriptive.rs). These are the definitions SPSS, SAS, Excel and R's psych::skew(type = 2) print; SciPy's skew(bias = False) / kurtosis(bias = False) are the same (pins explore/bp/skewness, explore/bp/kurtosis, both oracles). The alternatives — the moment estimators g₁, g₂ (R's e1071 type 1, SciPy's default) and the b₁, b₂ forms (type 3, Minitab) — differ by factors of order 1/n.

Standard deviation, variance and the standard error of the mean

The sample versions with divisor n − 1; the standard error of the mean is s/√n. Under a frequency weight, n is the sum of the weights (the case-weighting convention). The population (divisor n) forms are not printed anywhere.

Quantiles

Recorded in tolerances.md: the Hazen definition, plotting position (k − ½)/n — R type 5, NumPy hazen. It is used for every percentile the application prints, the median included, the box plot's quartiles and fences, and the quantile method of Bin variable. Weighted, the same rule runs on cumulative weight, averaging the two adjacent values on an exact boundary. Readers of SPSS should note that its default (HAVERAGE, R type 6) is a different rule.

Missing-data policy

Every procedure is listwise on the variables it uses — a case with a missing value in any analysis variable is dropped from that analysis — with one exception: bivariate correlations default to pairwise (each pair uses the cases valid on both), with missing: listwise as the option (correlation.rs). The dataset fingerprint's footer names the filter and weight; it does not name the listwise sample, which is the procedure's, and each table's N is that sample's.

Tables

Pearson chi-square: no continuity correction

The crosstabs χ² is the plain Pearson statistic on every table, 2 × 2 included — no Yates correction. The likelihood-ratio χ² is 2 Σ O ln(O/E). Pins crosstabs/pearson_chisq and lr_chisq (SciPy correction = False, R chisq.test(correct = FALSE)). SPSS prints a "Continuity Correction" row beside the Pearson row for 2 × 2 tables; ChakataStat does not, and the exact alternative for a small 2 × 2 is Fisher's test, below.

Fisher's exact test: the two-sided rule

Two-sided p is the sum of the probabilities of every table with the same margins whose probability is no more likely than the observed table — the rule R's fisher.test and SciPy use (fisher_exact in descriptive.rs). The alternative, doubling the smaller one-sided tail, can exceed 1 and is not used. The test is not defined for fractional weighted counts and is omitted then.

McNemar's test

For a matched 2 × 2 table with discordant counts b, c: the exact binomial two-tailed p when b + c ≤ 25 (integer counts), the continuity-corrected χ² (|b − c| − 1)²/(b + c) on one df otherwise — SPSS's rule (mcnemar, same file). The uncorrected form is what Cochran's Q reduces to for two treatments (recorded in validation.md).

Cramér's V and φ

φ = √(χ²/N) and V = √(χ²/(N·(k − 1))) with k the smaller dimension — the classical, uncorrected forms. Bergsma's bias-corrected V is not printed.

Mantel–Haenszel and Cochran's tests

The stratified 2 × 2 tests print both: the Mantel–Haenszel statistic with its continuity correction and Cochran's without, as the table note says.

Column-proportion z tests (custom tables)

The pairwise column comparisons use the pooled two-proportion z with no continuity correction and no multiple-comparison adjustment; a cell with an empty column or zero pooled variance prints no test.

Comparing means

The independent-samples t test: both variances rows

Both rows are always printed — equal variances assumed (pooled s², df n₁ + n₂ − 2) and not assumed (Welch, with the Welch–Satterthwaite df (a + b)² / (a²/(n₁−1) + b²/(n₂−1)), a = s₁²/n₁, b = s₂²/n₂) — with Levene's test beside them to guide the choice. No automatic switch is made. Pins ttest_ind/* and ttest_ind/welch_*.

Levene's test: mean-centred

The original Levene statistic — a one-way ANOVA on |x − group mean| — not the Brown–Forsythe median-centred variant (levene_table in compare_means.rs; pins levene/W, levene/p against SciPy center = "mean" and car::leveneTest(center = mean)). R's leveneTest and SciPy default to the median form, so a reader reproducing the number there must set the centre.

Cohen's d and Hedges' g

Independent samples: pooled SD with n − 1 variances, s_p = √(((n₁−1)s₁² + (n₂−1)s₂²)/(n₁+n₂−2)); d = (x̄₁ − x̄₂)/s_p.
Paired samples: d_z, the mean difference over the SD of the differences (not Cohen's d_av over the average SD).
One sample: (x̄ − μ₀)/s.
Hedges' g applies the small-sample factor J = 1 − 3/(4·df − 1) (the usual approximation to the exact gamma ratio).
The confidence interval on d is from the noncentral t (not the large-sample normal approximation).

Pins ttest_ind/cohen_d, ttest_one/cohen_d (formulas on both oracles).

One-way ANOVA effect sizes

η² = SS_between / SS_total; ε² = (SS_between − df_between·MS_within) / SS_total; ω² = (SS_between − df_between·MS_within) / (SS_total + MS_within) — the Olejnik–Algina forms. Pin anova/eta_squared.

Post hoc tests

After a one-way ANOVA the pairwise table carries three adjusted significances on the pooled within-groups MS: Bonferroni, Scheffé and Tukey HSD (studentized range on df_within), with the Bonferroni-adjusted confidence interval of each mean difference. After a Kruskal–Wallis test, Dunn's pairwise z is adjusted by Bonferroni (default), Holm or none (adjust). Games–Howell, Dunnett and LSD are not offered.

Planned contrasts

Reported both ways — pooled (the ANOVA's within MS) and Welch–Satterthwaite — like the t test; the user's coefficients are not normalized.

Correlation

Spearman's ρ

The Pearson correlation of the mid-ranks (tied values share the mean of the ranks they span); significance by the t approximation on n − 2 df, t = ρ√((n−2)/(1−ρ²)) — SciPy's rule and SPSS's. R's cor.test uses the exact / AS 89 distribution for n < 1290, which differs in the third digit at n = 60 (the second oracle recomputes the t form on R's ρ; pin correlate/spearman_age_bp_p).

Kendall's τ

τ-b, the tie-corrected form; significance by the normal approximation with the no-tie variance 2(2n + 5)/(9n(n − 1)). Under a weight the concordances are weighted by the product of the two cases' weights and the effective N is the sum of the weights. τ-a and τ-c are not printed. Pin correlate/kendall_age_bp.

Partial correlation

The first-order partial from the inverse of the correlation matrix over the listwise sample, tested on n − 2 − k df, k the number of controls (pin partial/age_bp_given_chol, pingouin and psych::partial.r).

No confidence interval on r

The classical correlation tables print r and its p only; no Fisher-z interval. The Bayesian correlation prints a credible interval (recorded in validation.md).

Regression and modelling

Confidence intervals on GLM coefficients: Wald, z

Logistic, probit, Poisson, negative-binomial and gamma coefficients carry Wald intervals B ± z·SE, and Exp(B) the interval exp(B ± z·SE) — statsmodels' and SPSS's convention. R's confint.glm defaults to the profile-likelihood interval, which differs for small samples and for coefficients far from zero; confint.default gives the Wald one. Linear regression uses t on the residual df. Under a survey design every model's inference is t on (design df) − p + 1, the summary.svyglm convention (recorded in clinical-and-survey.md).

Stepwise selection

Forward, backward and stepwise linear regression enter a predictor at p ≤ .05 and remove at p ≥ .10 (p_enter, p_remove) — the SPSS defaults, chosen as significance levels rather than F-to-enter values (regression.rs). Stepwise discriminant analysis uses Wilks' Λ with F = 3.84 to enter and remove.

Multinomial and ordinal regression: the reference and the sign

Multinomial logistic fits one logit per non-reference category against the lowest category (by value) as the baseline — R nnet::multinom and statsmodels MNLogit. SPSS defaults to the last category.
Ordinal regression is the proportional-odds cumulative logit logit P(Y ≤ c) = θ_c − β·x: the thresholds θ_c carry the intercepts and the slopes β enter with a minus sign, so a positive β means higher categories — the parameterization of SPSS PLUM, MASS::polr and statsmodels OrderedModel (pins ordinal/*, both oracles). Some references (Stata ologit reports the same sign; SAS proc logistic models P(Y ≥ c) by default) flip the sign of β.

Robust regression: the M-estimator constants

Huber's ψ with c = 1.345 and Tukey's bisquare with c = 4.685 (each ~95 % efficient under normality); the scale is the MAD / 0.6745; the covariance is the M-estimator sandwich — the statsmodels RLM conventions (robust.rs). R's MASS::rlm uses the same constants.

Mixed models: REML by default, ML on request

The linear mixed model fits by restricted maximum likelihood unless reml: false; the fit block names which. REML corrects the downward bias ML puts on the variance components (the degrees of freedom the fixed effects absorb — the n − 1 idea), and it is what lme4::lmer, nlme, SAS proc mixed, Stata mixed, SPSS MIXED and statsmodels MixedLM all report by default; ML remains the option because a likelihood-ratio test between models with different fixed effects is only valid under ML. Until 0.23.0 the default was ML — switched on 2026-08-28 when this inventory surfaced it. The GLMM (random intercept) integrates the random effect by fixed Gauss–Hermite quadrature (glmm.rs); lme4::glmer defaults to the Laplace approximation (one quadrature point), which differs for binary outcomes with small clusters.

Nonlinear regression

Levenberg–Marquardt from the user's starting values; standard errors from the Gauss–Newton covariance σ̂²(JᵀJ)⁻¹ at the solution (nonlinear.rs). The pinned SSE and R² are optimizer-invariant; the parameters on a flat objective are not (the second oracle's nls/* agree to ~3 digits; recorded in validation.md).

Regularized regression

Predictors are standardized (and the response centred) before the penalty is applied, on each training fold separately; the λ path is 100 log-spaced values from λ_max down; 10-fold cross-validation reports both the CV-minimum λ and the 1-SE λ, the glmnet conventions (regularized.rs). The coefficients are reported on the original scale.

Loglinear models

Fitted as a Poisson GLM on the cell counts (Newton/IRLS), not by iterative proportional fitting; the two agree at convergence for hierarchical models, and the Poisson route gives standard errors directly (regression_ext.rs).

Nonparametric tests

Wilcoxon signed-rank

Zero differences are dropped before ranking (n counts the non-zero pairs); the asymptotic z carries the tie correction and no continuity correction — the SPSS form (pin wilcoxon/z; R wilcox.test(correct = FALSE)). R's default applies the ½ continuity correction.

The sign test and the runs test

Both use the normal approximation with the continuity correction (SPSS); the runs test cuts at the median (values below vs at-or-above) and offers an exact significance from the enumerated runs distribution. Pins sign/z, runs/*.

The binomial test: which tail

Exact by default; two-tailed when the test proportion is ½, one-tailed otherwise — SPSS's rule, kept because it is the one users of the classical toolbox expect (binomial in nonparametric.rs; pin binomial/exact_p at ½). R's binom.test is two-sided at any proportion (summing outcomes no more likely than the observed), so for p₀ ≠ ½ the two differ by construction; method = "asymptotic" gives the continuity-corrected normal.

Normality tests

Shapiro–Wilk by Royston's (1992) AS R94 algorithm, and the one-sample Kolmogorov–Smirnov against the fitted normal with Lilliefors significance — i.e. the SPSS "Tests of Normality" table. SciPy's kstest(z, "norm") gives the same D (pin normality/ks_D) but its p is the plain KS p, which is too conservative when μ and σ are estimated; ChakataStat's p is the Lilliefors one.

Survival

Kaplan–Meier

The survival table prints S(t) with its Greenwood standard error and no confidence band — a reader who wants one should know that R's survfit draws a log-transformed band by default and SPSS none, so bands from different tools are not comparable without naming the transform. The median survival is the first time at which S(t) ≤ 0.5 (SPSS's rule; R's survfit reports the same first crossing, interpolating only when the curve sits exactly at 0.5). The log-rank test is Mantel–Cox with the hypergeometric variance; Peto and Gehan–Wilcoxon weightings are not offered.

Cox regression: tie handling and intervals

Tied event times use Efron's approximation by default, Breslow's on ties: "breslow"; the fit block names the method. Efron is the more accurate of the two whenever ties exist (and real event times nearly always tie) and is what R's coxph, Stata and lifelines use; SPSS and SAS default to Breslow, which is why it stays an option. Until 0.23.0 the default was Breslow — the maintainer switched it on 2026-08-28 when this inventory surfaced the mismatch with the validation pins, which fit Efron on both sides (survival_cox/*). Hazard-ratio intervals are Wald, exp(B ± z·SE); the baseline hazard is Breslow's. The proportional-hazards test's form is recorded in clinical-and-survey.md.

Time series

Autocorrelation

The ACF uses the divisor n at every lag (the biased, positive-definite estimator; statsmodels adjusted = False, R acf), Bartlett's cumulative standard error √((1 + 2Σ_{j<k} r_j²)/n), and the Box–Ljung Q on k df at lag k (no model-order adjustment — the residual ACF after an ARIMA fit subtracts p + q). The PACF is computed by Durbin–Levinson from the ACF, not by successive OLS regressions; the two agree to ~5 digits at n = 120 (pin pacf/lag1).

ARIMA

Conditional sum of squares by default (method: "css"), exact Gaussian maximum likelihood by Kalman filter on "ml" (warm-started from CSS); a constant is fitted by default for an undifferenced series. Recorded for the AR(1) pin in tolerances.md.

Stationarity tests

ADF and KPSS both default their lag / bandwidth to Schwert's rule ⌊12·(n/100)^¼⌋; ADF's p is MacKinnon's (1994) polynomial approximation as embedded in statsmodels — his 1996 response-surface tables, which R's urca::punitroot uses, agree to 2–3 digits (pin adf/p, inside 1e-3). KPSS uses the Bartlett kernel. R's tseries::adf.test defaults to ⌊(n−1)^⅓⌋ lags, so a default-to-default comparison differs.

Exponential smoothing and decomposition

Simple exponential smoothing initializes the level at the first observation and chooses α by minimizing the one-step SSE (statsmodels' legacy-heuristic, R HoltWinters); the trended and seasonal forms are unit-tested at fixed parameters, not pinned (their optima are flat — validation.md). Seasonal decomposition is the classical centred moving-average decomposition, additive or multiplicative — not STL. Cross-correlations carry the ±2/√N bands.

Scale reduction and classification

Factor analysis: extraction, retention, rotation

Principal components by default, iterated principal-axis factoring (started from the squared multiple correlations) and maximum likelihood (Jöreskog's concentrated criterion) on request; components retained by Kaiser's rule (eigenvalue > 1) unless a count is given; Varimax with Kaiser normalization, or Promax (κ = 4 by convention) on request, no rotation by default. SPSS defaults agree; R's psych::fa defaults to minres extraction and oblimin rotation, factanal to ML — none of which match a default-to-default comparison.

Clustering

k-means (Lloyd's) on squared Euclidean distance over the variables as given (no automatic standardization — standardize first if the scales differ); the initial centres are the deterministic maximin (farthest- point) rule unless a seed asks for seeded random starts, so an unseeded run is reproducible. Hierarchical clustering defaults to average linkage (UPGMA); Ward's method works on squared Euclidean distance (the ward.D2 convention), single and complete on Euclidean; a case weight counts as that many coincident objects. Two-step selects the cluster count by a BIC-style criterion. Cluster labels and centres are not pinned across packages (recorded in validation.md).

Discriminant analysis: prior probabilities

Classification functions use priors proportional to the group sizes (ln(n_g/n) in the constant), not equal priors. SPSS defaults to equal priors; R's lda to proportional.

Reliability

Both Cronbach's α on the raw covariances and the standardized α on the correlation matrix are printed (pin reliability/alpha is the raw one). The six ICC forms follow Shrout & Fleiss (recorded in validation.md).

Charts

Box plot

Tukey's rule: fences at Q₁ − 1.5·IQR and Q₃ + 1.5·IQR, whiskers to the last observation inside the fences, points past them listed as outliers; the quartiles are the Hazen ones above.

Histogram

Equal-width bins over [min, max], 10 bins by default (bins); no automatic Sturges / Freedman–Diaconis rule.

Q–Q plot

Blom plotting positions (r − ⅜)/(n + ¼) against the theoretical quantiles, the reference line through the first and third quartile pairs (R's qqline). SPSS uses Blom by default as well; R's qqnorm uses ppoints, which is Blom for n > 10.

Power analysis

Two-tailed by default (tails: 1 for one-sided); every power is a tail of the relevant noncentral distribution at the central critical value (recorded in validation.md).

Degenerate inputs — the 0.23.0 conventions

Decided by the adversarial fixture battery (0.23.0 item 6): 84 rows of degenerate data, degenerate designs, numeric extremes and case-semantics edges, each driven through the tool layer and held to one of three outcome classes — a number with a reference, a named error, or an undefined cell whose reason is recorded. The battery's list lives beside its fixtures in the repository (test/fixtures/battery/README.md); these are the choices it forced, stated once.

Three outcome classes, and what is never one

A statistic that exists is printed and pinned; one that cannot be computed for the input is either refused with a named reason (when the whole procedure has no answer — a t test on one case, a regression with no residual degrees of freedom) or left blank (when one cell is undefined while its neighbours are not — skewness on two cases, a correlation with a constant). What is never acceptable: a crash, a hang, a silent drop of cases, or a plausible-looking number that is not one (a negative F, a fitted coefficient of 12,911 with no standard error).

Fewer than two cases, or a constant variable

A t test (one-sample, independent, paired) on fewer than two valid cases or pairs, or on a variable with zero variance, is refused — R's not enough 'x' observations and data are essentially constant. With one case in one of two groups the pooled t still exists on n₁ + n₂ − 2 df and is printed (R's var.equal = TRUE); the Welch row is blank. A one-way ANOVA on a constant prints its sums of squares and a blank F; on a one-level factor it is refused, as the t test and the Kruskal–Wallis test already were. A correlation involving a constant is blank; on two points it is ±1 with a blank significance. Normality tests need n ≥ 3 and variance, and are blank otherwise. Cronbach's α with a constant item is refused (its standardized form is undefined; the raw form printed 0.12 beside a standardized 0.95 before this was decided).

Zero cases is an answer

A filter that selects no case, a split group with no valid case, or a variable that is entirely user-missing all report N = 0 rows, not an error: an empty selection is a legitimate description of the data. A weighted count that comes to nothing prints 0, never −0.

Weights

Case weights are frequency weights: a weight of 10¹² is 10¹² replicates, and the inference is at that N. A case whose weight is zero or negative is excluded — it contributes nothing, and a group left empty by exclusion is reported as absent (the t test then refuses for want of a second group). R refuses negative weights outright; this engine follows the exclusion rule its Weight Cases documentation already states. A survey-design weight is different: every case needs a positive one, and a non-positive value is refused naming the case.

Regression: collinearity, degrees of freedom, separation

Exact collinearity is refused with the verdict text the predictors are collinear (singular matrix) — a constant predictor falls under it (it is collinear with the intercept). R drops the aliased column and prints NA; Stata omits it with a note; this engine names the condition and leaves the choice of column to the analyst.
No residual degrees of freedom — weighted N at or below the parameter count — is refused, stating both numbers. R fits such a model and prints NaN standard errors; a weighted N below p printed a negative df and a negative F here before the battery.
Complete or quasi-complete separation in logistic regression is refused: the maximum-likelihood estimate does not exist (Albert & Anderson 1984). The test is R's own — a fitted probability within ten machine epsilons of 0 or 1 — but where glm.fit warns and prints the diverged fit, this engine stops and names Firth's penalised likelihood (Heinze & Schemper 2002) as the remedy. The propensity-score tool already refused separation; the two now agree.
A binary outcome with one observed level, or a count outcome that is all zero, is refused: the intercept's MLE is infinite.
A negative survival time is a data error and is refused naming the case, in Kaplan–Meier as in Cox; it was dropped silently before.

Extremes

Statistics are scale-invariant: a variable at 10¹⁵ or 10⁻¹⁵ gives the same t, p and R² as at unit scale (the 0.22.0 precision review's precedent), and a slope of 10³⁰ is exact. At the limits of a double the order statistics (min, max, median) are exact and the moments that overflow (mean, SD, sum) are blank. A power analysis with an effect of 0 reports that no sample size reaches the target; an n below 2 is refused in every power tool.