Working with Data

Everything in ChakataStat starts with a dataset. This page covers getting data in and out, the two ways you look at a dataset (Data View and Variable View), the native .ckd document, and the tools for tidying a dataset up.

The shape of a dataset

A dataset is a rectangle: rows are cases, columns are variables. Behind the scenes a variable holds one of just two storage types:

• Numeric — numbers; a blank or non-number is missing.
• String — text (and dates are entered as text in ISO YYYY-MM-DD form).

On top of storage sits each variable's definition (label, value labels, measurement level, missing rules) — that lives in the Variable View and travels with the data inside an .ckd file.

Importing data

File → Import Data… reads the formats below. The Carries metadata column is the one to notice: those formats bring variable labels, value labels and more across with the data, so you do not re-type in the Variable View what the file already knew.

• CSV / TSV — the delimiter is sniffed from the content: comma, semicolon, tab or pipe, with quoted fields respected (so a ;-file full of quoted commas still reads correctly; .tsv always reads as tab). The header row is detected, and each column's type is inferred (all-numeric → numeric, otherwise string). An import-options dialog lets you confirm before loading. A gzipped text file (.csv.gz, .tsv.gz, .json.gz, …) is decompressed and read the same way.
• Fixed-width text (.fwf, .prn) — column-aligned text with no delimiter (mainframe extracts, printed reports). The one thing no sniffer can know is where the columns break, so a small dialog asks you: the first lines of the file are shown in a monospaced preview under a clickable ruler — click to place or remove a column break (▼), with initial breaks suggested from the runs of spaces every sampled line shares. The same spec is editable as text beside the preview (start-end ranges, 1-based and inclusive, e.g. 1-8,9-12,13-20), and the two stay in step; ranges may skip over junk columns. Once split, columns type-infer exactly as CSV's do. Headless (batch or MCP) passes the same spec with --fwf-spec.
• Excel (.xlsx) — the first worksheet, read in pure Dart. Legacy .xls and OpenDocument .ods read the first sheet through the native engine, with the same header prompt and type inference.
• Parquet / Arrow — typed columns read directly by the native engine.
• JSON — either an array of records ([{"age": 30, …}, …], one object per case) or a columnar object ({"age": [30, 31, …], …}); newline-delimited .ndjson/.jsonl has one object per line. The column names come from the JSON, so there is no header prompt; each column's type is inferred like CSV.
• SPSS (.sav, .zsav) — typed columns plus the file's metadata: variable labels, value labels (1 → Male arrives as a ChakataStat value label), measurement levels, and discrete user-missing values. SPSS ranges of missing values have no equivalent here; the import says so in its summary rather than dropping them silently. Legacy portable .por files are not read — save them as .sav from SPSS first.
• Stata (.dta) — typed columns plus variable labels and value labels (every .dta release is read). Daily dates (%td) and datetimes (%tc) arrive as readable dates; the rarer week/month/quarter display formats arrive as their stored numbers, and the import summary notes it. Stata's long strings (strL) are carried like any other text — including files written by third-party tools whose strL layout differs from Stata's own. A rare binary strL payload is decoded as text and the import summary names the variable.
• SAS transport (.xpt) — the open format SAS writes for exchange (PROC CPORT/COPY), typed columns plus variable labels. SAS date, datetime and time formats arrive as readable dates and times. Two limits, both noted in the import summary: value labels are not in the file at all — SAS keeps them in a separate .sas7bcat catalog — and if the file holds more than one dataset, the first is imported.
• SAS data (.sas7bdat) — SAS's own data files open directly, typed columns plus variable labels and the file label, with dates, datetimes and times rendered exactly as from .xpt (fractional seconds are truncated to the second; the month-year / quarter / year display formats arrive as their stored numbers, noted in the summary). Value labels live in a separate SAS format catalog, so put the .sas7bcat next to the data file — named like it (study.sas7bcat beside study.sas7bdat) or as formats.sas7bcat, SAS's default catalog name — and every variable whose format the catalog defines gets its value labels. With no catalog beside the file, a variable with a user-defined format is named in the import summary, with that advice. SAS's missing values (. and the special .A–.Z) arrive blank, and a .sas7bcat that cannot be read is reported, not fatal. Both compressed (COMPRESS=CHAR/BINARY) and uncompressed files read, in any of SAS's session encodings.

Text encodings

Every text import (delimited, fixed-width, JSON — gzipped included) detects its encoding rather than assuming UTF-8: a byte-order mark wins when present (UTF-8, UTF-16LE, UTF-16BE); without one, UTF-16 is recognised heuristically, then strict UTF-8 is tried, and Windows-1252 — what "Latin-1" files in the wild almost always are — is the final fallback, so the decode itself never fails. UTF-8 files open silently (that is the norm, not news); anything else imports and says so: the import summary carries a note such as decoded as UTF-16LE, telling you a conversion happened and which encoding was assumed. If text still looks wrong despite the note, the file is in some other single-byte code page — re-save it as UTF-8 and it opens exactly. Exported files are always UTF-8.

A progress dialog shows while large files load, and the import runs without freezing the window.

Tip. After importing a CSV, visit the Variable View and set measurement levels and labels. Type inference gets storage right but cannot know that a numeric 1/2 column means Male/Female — that is a value-label job (below). An SPSS or Stata import does this for you: the labels arrive with the data.

Exporting data

File → Export Data… writes the current dataset (honouring any active case filter) to CSV/TSV, Excel, Parquet or JSON (an array of records) — the format is chosen from the file extension you give.

CSV/TSV exports are protected against spreadsheet formula injection by default: a text cell that starts with =, +, -, @, a tab or a carriage return and is not a number is written with a leading apostrophe ('), which Excel and Calc read as "this is text" and hide — so a value that arrived in your data can never run as a formula on someone else's machine. Numbers such as -5 are never touched. If you need a byte-faithful file (for a tool that would show the apostrophe), turn the guard off under Settings → Files.

To export results rather than data, see the output log export.

The Data View

The Data View (Ctrl+1) is the spreadsheet you type into:

• Click a cell and type to edit; Tab/Enter move the active cell.
• Paste (Ctrl+V) a block from another spreadsheet; rows and columns grow to fit. Typing into the empty grid creates the first variables automatically.
• Find & Replace (Ctrl+F) opens a search bar with match case, entire cell and wrap-around options; Replace All is a single undoable edit.
• Hover any column header to see that variable's label, type and measurement level as a tooltip — the editor documents itself.
• When the dataset is empty, the view shows an empty state with Open and Import buttons instead of a blank grid.
• Hide a column you don't need right now: right-click its header and choose Hide column (selecting several headers first hides them all together). A thin double line marks where columns are hidden — click it, or right-click the header on either side and choose Unhide, to bring them back. View → Unhide All Columns is the escape hatch if you lose track. Hiding is a display preference, not a data change: the variable still exists, still appears in every dialog's variable list, and stays valid in .cks syntax — it just isn't shown in the grid.

Edits are undoable (Ctrl+Z / Ctrl+Y).

The Variable View

The Variable View (Ctrl+2) shows one row per variable. The columns are the variable's metadata; click a cell to edit it:

Why the measurement level matters

The Measure tells procedures how to treat a variable:

• Scale — a true number (age, blood pressure): means, correlations, the numeric axis of a histogram.
• Ordinal — ordered categories (education level): ranks and ordered tests.
• Nominal — unordered categories (region, sex): counts, crosstabs, grouping.

Charts and dialogs use the level to offer sensible defaults and to label output, so setting it correctly is worth the minute it takes.

Value labels and missing values

Both are edited in a small dialog from the relevant Variable View cell:

• Value labels make output readable — a crosstab shows Male/Female instead of 1/2, and the underlying codes stay numeric for analysis.
• User-missing values mark specific codes as "not a real value". They reach the engine as nulls and are dropped from every statistic, while still being visible (and editable) in the Data View.

Saving: the .ckd document

File → Save (Ctrl+S) writes an .ckd file — ChakataStat's native document. Unlike a CSV, it stores both the data and every variable definition (labels, value labels, missing rules, measurement levels), plus document provenance. Reopening an .ckd restores the dataset exactly.

Internally .ckd is a versioned ZIP of a meta.json (all metadata) and a typed data.parquet (the columns). You never need to look inside, but the format is documented for the curious in docs/file_format.md.

Your wider session — the open document, the results log, the AI conversation and the syntax — is also remembered between launches, and the unsaved working dataset is autosaved for crash recovery.

Tidying a dataset

A few tools under the Data menu help with messy or wide datasets:

• Define Variable Properties… — scan the data and set value labels, measurement levels and missing rules for several variables at once, guided by the values actually present.
• Bulk Rename Variables… — rename many variables with a regular expression (with $1 group references), a live preview and collision-safe de-duplication. Ideal for stripping a common prefix or renaming q1, q2, ….
• Define Variable Sets… — group related variables under a name, then pick the set in the Variable View's filter bar to focus on just that group. Since 0.14.0 the active set reaches everywhere you pick variables: the Data View shows only the set's columns, and analysis dialogs offer only the set's variables in their source lists — the way to keep a 400-column survey workable. Sets are a view convenience — they never change the data, a hidden column stays in every analysis, and a variable you already selected in a dialog stays listed even if the set excludes it. Sets persist with the session.

The data dictionary (.ckdict)

File → Export Data Dictionary… writes all variable metadata to a standalone, versioned JSON .ckdict file — names, labels, types, value labels and missing rules, without the data. Import Data Dictionary… applies one to the current dataset: it matches by variable name, writes only the fields each (even hand-edited, partial) entry carries, leaves your data untouched, and reports the fit (applied / unmatched / not in dictionary).

This is how you reuse one carefully-defined codebook across many data files.

Where to go next

• Case Semantics & Transforms — filter, weight and split cases; compute, recode and rank.
• Analyzing Data — run statistics on your dataset.