ChakataStat
Download

Complex Samples (Survey Designs)

Data → Survey Design… Real survey datasets — DHS, ESS, NHANES and their kin — are not simple random samples: cases are drawn in clusters (primary sampling units, PSUs) within strata, with unequal sampling weights. Analyzing such data as if every case were an independent draw produces wrong standard errors, silently — usually too small, because cases in the same cluster resemble each other. Declaring the survey design fixes that: the covered procedures then report design-based estimates with Taylor-linearized standard errors, the way R's survey package and the major packages' complex-samples modules do.

Declaring a design

Open Data → Survey Design…, choose Declare a design, and pick:

  • Sampling weight (required) — the numeric variable holding each case's sampling weight. Every case must have a positive weight; a missing, zero or negative value stops the declaration with a message naming the case, because a sampling weight is part of the design, not a filter.
  • PSU / cluster (optional) — the primary-sampling-unit identifier. PSU values are read within strata, so reusing PSU numbers 1–n in every stratum works (the common survey-file convention). Leave it out for a design without clustering (each case its own PSU).
  • Strata (optional) — the stratum identifier. Leave it out for an unstratified design.

On OK the declaration is validated and a Survey Design summary is written to the output: the number of strata and PSUs, the design degrees of freedom (PSUs − strata — the df every design-based confidence interval uses), the case count and the sum of weights. The status bar shows Survey Design On while a design is active, and the declaration is journaled as a survey_design command, so a saved syntax file reproduces it:

survey_design weight=wtmec2yr, psu=sdmvpsu, strata=sdmvstra
survey_design clear=true

Two rules to know up front:

  • Every stratum needs at least two PSUs. With a single PSU the variance estimator is undefined (the "lonely PSU" problem); the declaration fails with a message naming the stratum, matching R's default behaviour.
  • Survey Design and Weight Cases are mutually exclusive. The design's sampling weight already weights every estimate; layering a frequency weight on top would count cases twice. Declaring one clears the other.

What is design-aware (and what is not)

With a design declared, exactly these procedures estimate design-based — Explore, Frequencies, Crosstabs (two-way), Means, Linear Regression and Logistic Regression:

Procedure What you get
Explore Design-based mean, its standard error, the confidence interval on the design df, the design effect (deff), and the estimated population total with its SE. The full descriptive battery (quantiles, skewness) is not design-based and is not shown here.
Frequencies Per category: the unweighted count, the estimated population count, the design-based proportion with its SE, a logit-scale confidence interval (never outside 0–1), and the deff.
Crosstabs Estimated population counts, and — with the chi-square option — the Rao–Scott second-order corrected test with an F reference distribution (the svychisq default) instead of the classic Pearson block.
Means Each group's design-based mean, SE and CI, estimated as a domain (see below).
Linear / Logistic Regression The sampling-weighted fit with Taylor-linearized (sandwich) standard errors; t tests and CIs on (design df) − p + 1. The logistic table keeps Exp(B) with its interval. The classic ANOVA table, standardized betas and collinearity diagnostics are simple-random-sample constructs and are not produced; stepwise selection, robust SEs, casewise diagnostics and the Hosmer–Lemeshow test are unavailable under a design.

Every other analysis states, in its output note, that it ran ignoring the design. Nothing is wrong silently: a procedure either estimates design-based or says plainly that it did not. (Descriptives is in this second group — use Explore for design-based summaries.)

Domains: subgroup analysis done right

Estimating a subgroup of a survey — one sex, one region, the cases passing a filter — is not the same as deleting the other cases and re-analyzing. The subgroup's sample size is itself random, and dropping cases can drop whole PSUs, which changes the variance structure. The correct method, domain estimation, keeps every case's stratum/PSU membership and zeroes the out-of-domain contributions.

ChakataStat applies domain estimation automatically wherever a subgroup arises under a declared design:

  • Select Cases — the filter defines a domain, never a row subset;
  • Split File — each group is a domain;
  • the Means factor — each level is a domain;
  • missing values on an analysis variable — the complete cases form a domain.

You do not have to do anything; the point of this section is that you can trust Select Cases and Split File under a design, where naive subsetting would misstate the standard errors.

Reading the output

  • Std. Error — Taylor-linearized over the stratum/PSU structure (with replacement; no finite-population correction in this version).
  • Confidence intervals — t on the design df (PSUs − strata), the SPSS/Stata convention. Proportions use a logit-scale interval (svyciprop's "logit" method), so bounds stay inside 0–1; a proportion of exactly 0 or 1 has no interval rather than a fabricated one.
  • Deff — the design effect: the ratio of the design-based variance to the variance a simple random sample of the same size would have had (with-replacement baseline, R's deff = "replace"). Deff ≈ 2 means the clustering/weighting doubled the variance — the effective sample size is about half the nominal one.
  • Population Size / Weighted N — the sum of the sampling weights: the population the sample stands for, which is what a survey estimate is about.

Every design-based table names the design (variables, strata, PSUs, df) in its note, so output stays self-describing when exported.

Scope, precisely

This version estimates by Taylor linearization only — no BRR/jackknife/bootstrap replicate weights, no finite population correction, no multistage variance (first-stage PSUs carry the variance, the standard with-replacement approximation), and no design-based quantiles. Every number the design-aware procedures print is validated end to end against R's survey package — see the project's validation methodology.

Related: Case Semantics & Transforms (Select / Weight / Split), Descriptive Statistics, Correlation & Regression.