Clinical and survey conventions

Part of the numerical-validation suite. The recorded conventions behind equivalence testing, method comparison and complex survey designs.

Complex survey designs — the v1 conventions

Decided 2026-08-18, before any code, as sub-task 1a of docs/releases/release_0.19.0.md item 1 (kept private). ChakataStat's Weight Cases weight is a frequency weight; a survey dataset carries a design — strata, PSUs (clusters), sampling weights — and needs design-based standard errors. Every convention below was verified against R survey (4.5, R 4.5.3) on a hand-computed toy design before being recorded; the reference scripts and pinned values live in tool/validation/generate_survey_references.R and test/validation/survey_reference_values.json (sub-task 1b).

The declaration

A dataset-level design, declared once (Data ▸ Survey Design… / the survey_design tool) and carried in the request JSON as a design field beside filter/weight/split:

weight (required): the sampling-weight variable. Every case must have a positive, finite weight — a missing, zero or negative value is a declaration error, not an exclusion (a sampling weight is part of the design; silently dropping a case would misestimate every total).
psu (optional): the primary-sampling-unit (cluster) variable. Absent means element sampling — each case is its own PSU (R ids = ~1).
strata (optional): the stratum variable. Absent means one stratum.
Missing values in psu/strata are declaration errors.
PSU identifiers are read nested within strata (R nest = TRUE): the same PSU value in two strata is two distinct PSUs.
Weight Cases and a survey design are mutually exclusive. A request carrying both errors ("choose one"): layering a frequency weight over a sampling weight would compound two case multiplicities (the propensity precedent). The Survey Design dialog clears Weight Cases on apply and vice versa.

The covered procedures (the v1 list, quoted verbatim by later sub-tasks)

Design-aware behaviour ships for exactly these engine kinds: explore, frequencies, crosstabs (two-way), means, regression_linear, regression_logistic. With a design declared, every analysis kind not on this list appends a plain note that it ran ignoring the design (the whole point of the item: nothing is wrong silently); the descriptives tool (the light describe endpoint) gets the same note on the Dart side. Power analysis takes no dataset and carries no note.

The five estimation conventions

Variance estimation is Taylor linearization over stratum/PSU totals (with replacement): for per-case linearized scores uᵢ, with PSU totals z_hi = Σ_{i∈PSU} uᵢ in stratum h holding n_h PSUs, V̂ = Σ_h n_h/(n_h−1) Σ_i (z_hi − z̄_h)² — verified to reproduce svymean/svytotal/svyby standard errors exactly. Scores: total uᵢ = wᵢyᵢ; mean/proportion uᵢ = wᵢ(yᵢ−μ̂)/N̂; ratio uᵢ = wᵢ(yᵢ−R̂xᵢ)/X̂. Design df = (number of PSUs) − (number of strata) (degf.svydesign), and all confidence intervals use t on the design df — the SPSS Complex Samples / Stata convention, deliberately not R's confint(svymean(…)) normal default; references are therefore pinned as SEs from R plus intervals recomputed in this convention (an agreement about definitions, recorded here). Deff uses the with-replacement SRS baseline (R deff = "replace", consistent with no-FPC v1): deff = V̂_design / (s²/n) for a mean (for a total, V̂_design / (N̂²·s²/n)) with s² = (n/(n−1))·Σwᵢ(yᵢ−ȳ_w)²/N̂ — the svyvar estimate, verified numerically.
Lonely PSUs error. A stratum with a single PSU makes the variance estimator undefined; the R default (survey.lonely.psu = "fail") is kept: declaring (or analyzing under) a design whose analyzed structure has a single-PSU stratum returns the errors-as-data message stratum "<value>" has only one PSU; design-based variance needs at least two PSUs per stratum (lonely PSU). Because subsetting is domain estimation (below), the PSU structure never shrinks after declaration, so the check at declaration is authoritative (the estimator re-checks defensively).
Subpopulations are domains, never row subsets. Select Cases, Split File groups, and missing values on analysis variables all become domain estimation: out-of-domain cases keep their stratum/PSU membership with their score zeroed, and the variance runs over the full design structure (the classic subset-vs-domain SE trap). Verified to reproduce svyby/domain-svymean exactly; this is also what svymean(na.rm = TRUE) does with missing values.
Proportion CIs are logit-scale (the svyciprop(method = "logit") default, named "logit" in the output note): expit( logit(p̂) ± t_df · SE(p̂)/(p̂(1−p̂)) ) with SE(p̂) the Taylor SE of the mean of the indicator — verified to reproduce svyciprop end-to-end. Degenerate p̂ ∈ {0, 1} leaves the CI blank (the logit is undefined), never fabricated.
The crosstab test is the Rao–Scott second-order (Satterthwaite) correction with the F reference — svychisq's default (statistic = "F"), implemented exactly as survey does: Pearson X² on the estimated proportions scaled to the unweighted n; Δ = (CᵀD̂⁻¹C/n)⁻¹ (CᵀD̂⁻¹V̂D̂⁻¹C) with C the interaction contrasts residualized on the main effects and V̂ the full Taylor covariance of the cell proportions; F = X²/tr(Δ) on (d₀, d₀·ν) df, where d₀ = tr(Δ)²/tr(Δ²) and ν is the design df. The first-order correction is deliberately not offered in v1.

Regression under a design

svyglm's conventions, verified: point estimates are the sampling-weighted fit (WLS for linear; weighted IRLS for the binary links), and the coefficient covariance is the Taylor sandwich A⁻¹BA⁻¹ — A the weighted information (XᵀWX), B the stratified PSU-total variance of the per-case score uᵢ = wᵢ(yᵢ−μᵢ)xᵢ·(dμ/dη)/V(μ) (for identity and logit links this is wᵢeᵢxᵢ). Inference is t on (design df) − p + 1 (p = number of coefficients, intercept included) — summary.svyglm's df, verified — for the coefficient tests and the CIs (confint.svyglm's Wald form). The logistic table keeps Exp(B) with the CI as exp(B ± t·SE). Under a design the linear model's classic ANOVA table, Durbin-Watson, standardized betas, partial/part correlations and tolerance/VIF are not produced (they are SRS constructs); the model summary reports the weighted R² and names the design. Options that would silently mean something else under a design error as data in v1: stepwise method selection, robust (the design SEs are already a sandwich), casewise, diagnostics, save, and the logistic hosmer_lemeshow.

Out of scope for v1 (recorded cuts)

No finite population correction — NHANES-class with-replacement designs do not use it; the design JSON leaves room for an fpc field.
No BRR/jackknife replicate weights — Taylor linearization only; replicate methods are a recorded follow-up (the bucket pointer).
No multistage variance — first-stage (ultimate-cluster) PSUs only, which is exactly what the with-replacement approximation uses.
No design-based quantiles (svyquantile is its own methodological area): the design-aware Explore table reports estimate/SE/CI/deff for means and totals only, and is labelled design-based.
The layered (three-way) crosstab and its CMH block are not design-aware: under a design the layered path runs as today and carries the ignores-design note.

Equivalence tests (TOST) — the v1 conventions

Decided 2026-08-18, before any code, as sub-task 2a of docs/releases/release_0.19.0.md item 2 (kept private). "Not significantly different" is not "equivalent"; the two-one-sided-tests procedure tests whether a mean difference lies inside declared equivalence bounds. Every convention below was verified against R TOSTER 0.8.6 (the reference package, R 4.5.3) by running its t_TOST on hand-checkable data and reproducing its output from the hand formulas — not recorded from memory. The pinned values live in tool/validation/generate_tost_references.R and test/validation/tost_reference_values.json (sub-task 2f).

The shape (the 2a decision)

A dedicated equivalence_ttest tool — not a tost mode on the three t-test tools — modeled on the bayesian_ttest precedent: one tool, a test argument (one_sample / paired / independent), sharing the t-test machinery engine-side (compare_means.rs) under one new equivalence_ttest analyze kind. Reasons, recorded: the presentation (bounds, two one-sided tests, TOST p, 1−2α CI, decision) shares almost no columns with the classic t tables, so a mode would triple-touch three dialogs and three table layouts for no reuse; the in-catalog precedent for "alternative inference over the t-test machinery" (bayesian_ttest) is a dedicated tool; and a named tool is what an assistant/MCP caller can find ("equivalence test" → equivalence_ttest). Catalog moves 103 → 104, which settles the 0.19.0 expected final count at 108.

The test

TOST is two one-sided t tests of the observed difference against the declared bounds, on the same standard error and df as the corresponding classical t test — verified to reproduce t_TOST exactly from the hand formulas: t_lower = (diff − L)/SE with the upper-tail p (H₀: diff ≤ L), and t_upper = (diff − U)/SE with the lower-tail p (H₀: diff ≥ U). The designs: one-sample (diff = mean − test_value, df = n−1), paired (diff = mean of case differences, df = n−1), and independent with both rows, pooled and Welch (df = n₁+n₂−2 and Welch–Satterthwaite respectively — the shipped Independent-Samples Test's two-row presentation; t_TOST's default is the Welch row, var.equal = FALSE).
The TOST p is the larger of the two one-sided p's, and the decision is "equivalent" when TOST p < α (α from the alpha option, default 0.05) — TOSTER's decision rule verbatim, printed as a per-row decision cell, never only implied by the p.
The interval shown is the 1−2α t confidence interval of the raw difference (90% at the default α, the classical TOST presentation; t_TOST reports conf.level = 1 − 2·alpha, verified at α = 0.05 and 0.01). Equivalence at level α coincides with this CI lying inside the bounds.
Bounds are declared as an explicit pair (bound_lower < bound_upper), in raw units by default or Cohen's-d units (bound_unit = "cohen_d"). d-unit bounds convert to raw bounds by multiplying by the uncorrected standardizer of that design — verified against eqbound_type = "SMD": the sample SD (one-sample), the SD of the case differences (paired, d_z), the pooled SD (independent, pooled row), and √((s₁²+s₂²)/2) (Welch row, d_av). With d-unit bounds the independent table's two rows therefore carry two (slightly) different raw bounds, each printed. TOSTER itself warns that SMD bounds "produce biased results" (the bound becomes sample-dependent); the output note carries the same caution.
One-sample bounds are centered on the test value: the TOST is on mean − test_value with raw bounds meaning test_value + L … test_value + U. Recorded divergence: TOSTER 0.8.6's one-sample t_TOST does not center its bounds on mu (its TOST rows test the raw mean against ±bound about zero while its t-test row subtracts mu — internally inconsistent, verified by comparing mu = 100 against the shifted x − 100, mu = 0 run). The pins therefore use the shifted form, where TOSTER is self-consistent and agrees with our convention exactly.
Degenerate bounds error as data: bound_lower ≥ bound_upper is an error (TOSTER silently sorts reversed bounds and accepts zero-width bounds; deliberately not copied — a silently reordered hypothesis is the kind of quiet wrongness the suite exists to prevent). Bounds whose interval does not include zero are allowed with a plain note (the t_TOST behaviour) — they state an unusual but coherent hypothesis.

Out of scope for v1 (recorded cuts)

No minimal-effects test (TOSTER's hypothesis = "MET") and no single-bound non-inferiority presentation — asymmetric bounds cover the practical need; both are recorded follow-ups in the feature bucket only if asked for.
No TOST effect-size table — the classic t-test tools already report Cohen's d / Hedges' g with noncentral-t CIs; duplicating them under a 1−2α level would invite misreading (and TOSTER's SMD CIs are themselves approximations, per its own vignette).
Frequency weights flow through the shared moments machinery like the classic t tests; a declared survey design is not covered — the tool carries the standard ignores-design note (the item-1 list is unchanged).

Bland–Altman method comparison — the v1 conventions

Decided 2026-08-18, before any code, as sub-task 3a of docs/releases/release_0.19.0.md item 3 (kept private). The Bland–Altman analysis quantifies agreement between two measurement methods — bias, limits of agreement and the mean-vs-difference plot — which correlation deliberately cannot (two methods can correlate near 1 while disagreeing by a constant or proportional amount). Every convention below was verified against both reference packages, R blandr 0.6.0 and R BlandAltmanLeh 0.3.1 (R 4.5.3), by reading their sources and reproducing their output from the hand formulas on the original Bland & Altman (1986) PEFR data — not recorded from memory. The pinned values live in tool/validation/generate_bland_altman_references.R and test/validation/bland_altman_reference_values.json (sub-task 3f).

The shape

A dedicated bland_altman tool — two numeric method variables, the statistics tables plus the mean-vs-difference plot (plot argument, default true: the figure is what the method exists for) — one bland_altman analyze kind with the numeric core in the descriptive family (rust/src/analyze/descriptive.rs). Catalog moves 104 → 105. The plot is built Dart-side from the result tables (the ROC-curve precedent), not as a separate graphs.rs chart kind: one engine call produces tables and figure together, so the journaled command reproduces both.

The statistics

Differences are method 1 − method 2 (the first variable minus the second) and the abscissa is the pair mean, (m₁ + m₂)/2 — both packages' default (blandr.statistics(method1, method2); bland.altman.stats(group1, group2, mode = 1)). The table title and the plot's y-axis name the direction (A − B), so the sign is never ambiguous. Only complete pairs enter (both packages na.omit); n is the number of complete pairs, and fewer than 2 errors as data.
The limits of agreement are bias ± 1.96 × SD of the differences, with the multiplier fixed at the literal 1.96 in v1 — the universal reporting convention and both packages' default (BlandAltmanLeh's two = 1.96; blandr's LoA.mode = 1). SD uses the n−1 divisor (both). Verified on the 1986 PEFR first-measurement pair (n = 17): bias −2.117647, SD 38.76513, LoA −78.0973 / 73.86201 — both packages agree to machine precision. No multiplier option (recorded cut).
Every CI is t-based on n − 1 df at the confidence level (percent, default 95 — the ICC precedent, ci_level engine option). The bias CI uses SE = s/√n — both packages agree exactly (verified: t(16) CI −22.04884 / 17.81354 on the PEFR pair). The LoA CIs use the Bland & Altman (1999) exact variance, Var(LoA) = s²·(1/n + z²/(2(n−1))) with z the same literal 1.96 as the multiplier — self-consistency: z enters the variance only because the limit is bias ± z·s.
Two recorded reference divergences on the LoA CI, one per package: blandr uses the same 1999 form but with z = Φ⁻¹(0.975) ≈ 1.959964 while drawing its lines at the literal 1.96 (internally inconsistent — the TOSTER-mu precedent; ~1.2e-5 relative in the SE, so the blandr LoA CI pins carry a loosened tolerance naming this); BlandAltmanLeh uses the Bland & Altman (1986) approximation √(3s²/n) (≈ 0.7% larger SE — 1986's own "about" rounding of 1 + z²/2 = 2.92 up to 3; its values are pinned in the generator as the recorded alternative, and are not what we print). The confidence option moves only the t quantile, never the 1.96 (verified: exactly BlandAltmanLeh's conf.int behaviour; blandr's sig.level also moves its variance z — deliberately not copied).
Proportional bias is reported as a table: the OLS regression of the differences on the means — intercept and slope with SE, t and p on n − 2 df. blandr fits exactly this regression (it reports only a rounded equation string; the pins are against R lm on the same pairs). Undefined (n < 3, or no variance in the means) omits the table and says so in the statistics-table note rather than printing fabricated cells. The fitted line is not drawn on the plot in v1 (the plot keeps the three canonical reference lines); drawing it is a recorded follow-up.
s = 0 (identical measurements) is not an error: the limits coincide with the bias and every interval is zero-width — the correct degenerate values, printed as-is.
Frequency weights flow through the shared moments machinery (weighted bias and SD, n = Σw in the CIs and df; the proportional-bias regression is the correspondingly weighted least squares), keeping the engine-wide weight ≡ replication identity, asserted by a Rust unit test. The plotted points do not move with weights (the scatterplot precedent). A declared survey design is not covered — the tool carries the standard ignores-design note (the item-1 list is unchanged).

The tables and the plot

"Bland-Altman Statistics: A − B" — Measure / Value / Lower / Upper; rows Bias (Mean Difference) with CI, Std. Deviation of Differences, Lower Limit of Agreement with CI, Upper Limit of Agreement with CI. The note carries N (the complete-pair count — a count does not belong in a real-formatted Value column), the 1.96 multiplier, the t(n−1) CI forms and the direction of the differences.
"Proportional Bias: A − B on Mean" — Term / Coefficient / Std. Error / t / Sig.; rows Intercept and Slope (Mean).
"Bland-Altman Points: A vs B" — Mean / Difference, one row per complete pair: the plot's own geometry, printed and exportable like the ROC coordinates and the GLM residual panels.
The plot draws the points, a solid reference line at the bias and dashed lines at both limits, each labelled with its value at the right edge; the CI bands are not shaded in v1 (recorded follow-up). It is a single-color chart: colors (one slot) and title are its journaled style arguments, which is what lights up the chart editor.

Out of scope for v1 (recorded cuts)

No repeated-measures Bland–Altman (replicate measurements per subject, within-subject variance components) — the release plan records this cut up front; it stays in the feature bucket.
No percentage-difference / ratio variant (Giavarina-style percent plots, blandr's proportion) and no non-default multiplier — raw differences at 1.96 are the 1986 presentation the audience cites.
No CI shading on the plot and no fitted proportional-bias line (the table carries the test) — both recorded follow-ups if asked for.