Scale, Dimension Reduction & Classify

Three menu groups for multivariate work: assessing a set of items as a scale, reducing many variables to a few, and grouping or classifying cases.

Scale → Reliability Analysis

When to use it. To check whether a set of items (e.g. questionnaire questions) measure one underlying construct consistently.

Dialog. Move the item variables into the list.

Output. Cronbach's alpha for the scale, plus item-total statistics: how each item correlates with the rest and what alpha would become if the item were dropped. Low item-total correlation flags an item that does not belong.

Scale → Intraclass Correlation

When to use it. To measure the reliability or agreement of ratings when several raters score the same subjects (inter-rater reliability), or one measure is repeated (test-retest). The data are a wide layout: one column per rater, one row per subject.

Dialog. Move the rater columns into the Raters list and pick a confidence interval level (95% by default).

Output. A table of the six Shrout & Fleiss ICC forms, each with its confidence interval, F test and significance, plus the ANOVA mean squares they are built from. Pick the form from your design:

One-way vs two-way: one-way if each subject is rated by a different random set of raters; two-way if the same raters score every subject.
Absolute agreement vs consistency: agreement if the raters' actual values must match; consistency if a constant offset between raters is acceptable.
Single vs average measures: single if one rater's score will be used; average if you will use the mean of all raters.

ICC reuses the two-way variance decomposition of the repeated-measures ANOVA. Values near 1 mean highly reliable ratings; near 0, no better than chance.

Comparing two measurement methods, where the question is how far apart they get in the measurement's own units, not a relative index, is the Bland-Altman analysis's job (Scale → Bland-Altman).

Dimension Reduction → Factor

When to use it. To find a small number of latent factors behind many correlated variables. (Testing a structure you already hypothesize is the complementary procedure; see Confirmatory Factor Analysis.)

Dialog. Move the variables in. In Options choose:

Extraction: principal components (default), principal-axis factoring, or exact-discrepancy maximum likelihood;
Rotation: none, Varimax (orthogonal) or Promax (oblique);
Scree plot: on by default, the eigenvalues as a marked line with the Kaiser criterion line at 1 (the classic pick-the-elbow view);
save factor scores back as new variables.

Output. Communalities, total variance explained, the component/factor matrix (loadings), the scree plot, and, for ML, a goodness-of-fit χ² test. Oblique rotation adds Pattern, Structure and Factor-Correlation matrices.

Example. Reduce the body-measurement and blood-pressure variables to a couple of interpretable factors.

Dimension Reduction → Canonical Correlation

When to use it. To relate two sets of numeric variables to each other (the canonical_correlation tool): it finds pairs of weighted combinations, one from each set, that are as strongly correlated as possible.

Dialog. Tick variables in the Set 1 and Set 2 checklists. Only numeric variables are offered, and the sets cannot overlap: Set 2 lists only variables not already in Set 1. Each set needs at least one variable.

Output. A Canonical Correlations table with one row per function (as many functions as the smaller set has variables): Canonical R, R Square, Wilks' Lambda, Chi-Square, df and Sig. Each row tests whether the canonical correlations from that function onward are all zero (Bartlett's χ²); the note gives N. Two Standardized Canonical Coefficients tables (Set 1, Set 2) give each variable's weight in each Function, with signs chosen so every canonical correlation is positive.

Cases missing any variable in either set are left out (listwise). The analysis honours the Weight Cases setting: the correlations are weighted, and N is the sum of the weights. A variable with no variance, or a set whose correlation matrix is singular, is refused with a message saying so.

Dimension Reduction → Correspondence Analysis

When to use it. To see how the categories of two categorical variables go together (the correspondence_analysis tool). It breaks down how far their two-way table departs from independence into a few dimensions and places every row and column category on them.

Dialog. Pick a Row variable and a Column variable (two different variables, text or numeric codes) and the number of Dimensions to report (default 2).

Output. A Correspondence Summary with one row per dimension (Singular Value, Inertia, % of Inertia, Cumulative %) and a Total row; the note gives the χ² test of independence (total inertia = χ²/N). Row Points and Column Points tables list each category's Mass (its share of the total) and its principal coordinate on each Dimension.

The categories are the distinct values present, in ascending order; a case missing either variable is left out, and each variable needs at least two categories. There are at most (smaller category count − 1) dimensions, so a 2×2 table gives one whatever Dimensions says. The analysis honours the Weight Cases setting: each case adds its weight to its cell of the table.

Dimension Reduction → Multidimensional Scaling

When to use it. To picture the cases as points in a few dimensions, laid out from their Euclidean distances over a set of numeric variables: classical (metric) scaling (the multidimensional_scaling tool).

Dialog. Tick at least two numeric Variables, then set Dimensions (default 2) and Max cases (0 = no cap) (default 200).

Output. An MDS Model Summary giving each dimension's Eigenvalue, % of Variance (its share of the positive eigenvalues) and Cumulative %, and an MDS Coordinates table with each case's position on each Dimension (Case is its row number). A dimension whose eigenvalue is numerically zero is dropped, so you can get fewer than you asked for.

Scope limits. Distances are computed on the variables as given (not standardized) over the cases complete on every variable (at least three). By default only the first 200 complete cases are placed, and the coordinates table says so; raise Max cases, or set it to 0 to lift the default. No setting goes past 3,000 cases: the method holds two n × n matrices and its eigen-decomposition grows with the cube of n, so a larger request is refused. The coordinates are computed unweighted under Weight Cases (a case whose weight is not positive is left out; every other case counts once), and the output says so.

Classify

Analyze → Classify. Five ways to group or classify cases.

K-Means Cluster

Partition cases into a fixed number K of clusters. Reports the final cluster centers, an ANOVA of how each variable separates the clusters, the cluster sizes, and (optionally saved) each case's cluster membership and distance. A random seed in Options makes a random start reproducible.

Hierarchical Cluster

Build a nested clustering by agglomeration (single, complete, average or Ward linkage). Reports the agglomeration schedule, a dendrogram of the merge tree (on by default; a very wide tree is rendered as its last merges, with collapsed clusters sized in parentheses and a note saying so), and, optionally, each case's membership at a chosen number of clusters.

Scope limits. Capped at 200 cases by default, because the method's cost grows steeply with case count (cubic in time, quadratic in memory). Raise Maximum cases (or set it to 0 to lift it) to cluster more; the result then carries a note that the cap was lifted. No setting takes it past 10,000 cases: the distance matrix alone would need 800 MB there, and a larger request is refused rather than risking a crash (use k-means for larger data). The agglomeration honours the Weight Cases setting: each case enters as a cluster of its own weight, so the size-weighted linkages (between-groups average, Ward) are weighted. Single and complete linkage use nearest/furthest distances, which do not depend on how many times a case is counted, so they are unaffected.

Two-Step Cluster

Find the number of clusters automatically (minimising a BIC criterion) or fix it. Reports the auto-clustering BIC table, the centers, the counts and the saved membership. Handles larger datasets via a pre-clustering pass.

Discriminant

When. Predict a known group membership from scale predictors (the supervised counterpart to clustering).

Dialog. A grouping variable and the predictor variables. Options add forward/backward stepwise selection (by Wilks' lambda) and classification / cross-validation (resubstitution and leave-one-out confusion matrices).

Output. Canonical discriminant functions, eigenvalues and canonical correlations, the overall Wilks' lambda, Fisher classification functions, and the classification results table with percent correctly classified. A Plot… option draws the combined-groups canonical scatter (optionally with a territorial map).

Scope limits. Leave-one-out cross-validation is capped at 500 cases and the canonical-scores table behind the plot at 2000 cases, since both are expensive on large datasets. Each cap is configurable in the dialog (0 removes it); when a cap applies, the output says the cross-validation was skipped or the table truncated, and how to lift it. The analysis honours the Weight Cases setting: the case weight is a frequency weight, folded into the group means, the within- and between-groups scatter, and the classification tallies.

Example. Predict Diabetic (Yes/No) from BMI, FastingGlucose, Age and SystolicBP, with leave-one-out cross-validation to gauge real accuracy.

Decision Tree

When. Predict a class (a classification tree) or a numeric value (a regression tree) from numeric predictors, as readable if-then rules (the decision_tree tool).

Dialog. A Target (any variable) and the numeric Predictors. Method is CART (the default: the split with the largest impurity decrease) or CHAID (the most significant χ² or F split, kept only when p ≤ 0.05 after a Bonferroni correction). Target type is Auto-detect (categorical if the target is text or has at most 10 distinct values) or forced to Categorical or Continuous. Max depth defaults to 4. Entropy impurity and the minimum node and leaf sizes (2 and 1) are set through the command's arguments.

Output. Tree Structure: one row per node, each split shown as predictor ≤ threshold, with n, Impurity, Predicted and each leaf's class Distribution. Variable Importance: each predictor's share of the total impurity decrease. Then a Classification Table (actual × predicted, with the resubstitution accuracy and the leaf count in the note) or, for a regression tree, a Model Summary (R², RMSE, N).

Splits are always binary, at the midpoint between two adjacent predictor values; a coded categorical predictor is split as if ordered, and CHAID does not merge categories. Growth stops at the depth and size limits or at a pure node. There is no pruning or cross-validation, so the accuracy is optimistic. Cases missing the target or any predictor are dropped. Weight Cases is honoured.

ROC Curve

When. Judge how well a continuous score (a measurement, or a fitted probability from logistic regression) separates two known classes, the standard way to assess a diagnostic test or a binary classifier. Analyze → ROC Curve… (a top-level item, next to Classify).

Dialog. A Test variable (the numeric score) and a State variable (the true class). Set the Value of state (positive) to whichever state value marks the positive / condition-present case (default 1); every other value is taken as negative. Positive if test value is chooses whether a larger score (the default) or a smaller one points to a positive case. Plot ROC curve is on by default.

Output. The Area Under the Curve (AUC) with its standard error, confidence interval and the asymptotic significance of the test against chance (AUC = 0.5); the Coordinates of the Curve (sensitivity and 1 − specificity at every cut-point); and the Optimal Cut-Point maximising Youden's J (sensitivity + specificity − 1). The plot draws the curve on the unit square against the chance diagonal, annotates the AUC, and marks the optimal cut-point.

The AUC equals the Mann-Whitney U statistic divided by the number of positive×negative pairs (ties counted as half): the probability that a random positive scores above a random negative. A natural workflow is to fit a binary Logistic regression, save its predicted probabilities, and run ROC on that saved column.

Example. Assess FastingGlucose as a screen for Diabetic (positive value 1): read the AUC, and take the cut-point that best balances sensitivity and specificity.

How this is computed

This section names which method each procedure prints: the estimator, the statistic, its degrees of freedom, and the tie, missing-value and weight rules. Each line links the recorded convention that explains the choice. The full index is the convention inventory; every number here is validated against independent packages per the validation suite.

Reliability: Cronbach's α on the raw covariances and the standardized α on the correlations, both printed; item-total statistics and α-if-deleted (reliability). Intraclass correlation: the six Shrout–Fleiss forms with F-based intervals (validation).
Factor: principal components by default, principal-axis (from the squared multiple correlations) or maximum likelihood on request; Kaiser's eigenvalue > 1 retention unless a count is given; Varimax with Kaiser normalization or Promax (κ = 4); no rotation by default (factor analysis).
K-Means: Lloyd's algorithm on squared Euclidean distance over the variables as given (standardize first if scales differ); deterministic maximin seeding unless a seed asks for random starts. Hierarchical: average linkage (UPGMA) by default; Ward on squared Euclidean; a weight counts as coincident objects. Two-Step: the cluster count by a BIC-style criterion (clustering). Partitions are not pinned across packages (validation).
Discriminant: Fisher's canonical functions from W⁻¹B; classification functions with priors proportional to group size; stepwise by Wilks' Λ with F = 3.84 (discriminant).
ROC Curve: a larger score is more positive; tied scores get half credit (the Mann–Whitney identity); the Hanley–McNeil SE for the interval, the null SE for the test; Youden's J for the cut-point (validation).
Decision Tree: CART with Gini (or entropy) impurity, or CHAID. Correspondence analysis / Multidimensional scaling / Canonical correlation: the classical eigen-decompositions (validation).
IRT, Latent Class, CFA, Propensity, Bland–Altman have their own pages with their own sections.

Choosing among them

Are the groups known? → Discriminant (supervised).
Groups unknown, want a fixed number → K-Means.
Groups unknown and the variables are categorical items (want fit indices and membership probabilities) → Latent Class Analysis.
Groups unknown, want the structure / a dendrogram-style schedule → Hierarchical.
Groups unknown, want the count chosen for you → Two-Step.
Checking that items form one scale → Reliability.
Asking how binary items measure (difficulty, discrimination, where the test is precise) → Item Response Theory.
Measuring rater / repeated-measurement agreement → Intraclass Correlation.
Comparing two measurement methods in their own units → Bland-Altman.
Reducing many variables to a few → Factor.