Skip to content

ADR-0007 — Versioned TOML case format with a named upgrade path

Status: Accepted

Context

aeolus1d has two persisted formats. The HDF5 resume checkpoint has carried a schema_version root attribute and a step-by-step upgrade framework since the resume-checkpoint refactor (see src/aeolus1d/io/schema_upgrades.py and the discussion in ADR-adjacent docs). The human-authored TOML case file — the primary way a user defines a simulation — had no version marker at all.

That asymmetry is a maturity gap. The TOML format has changed across releases (the v0.9 strict [case] form, the minimal default-fill loader, the compact diff-against-default writer). Each change happened to stay backward-compatible by luck and care, but nothing in the format declared which version a given file was, and nothing gave a future breaking change (a renamed key, a restructured section, a units correction) a defined migration path. A v(N+1) file opened by a v(N) aeolus1d would either silently mis-parse or fail with an opaque unknown-key/missing-key error far from the real cause.

Decision

Version the TOML case format, mirroring the HDF5 schema_upgrades pattern.

  • A schema_version integer marker is the first key of the [case] block. CURRENT_CASE_VERSION = 1 today; the format is at v1 and the upgrade chain is empty. The framework lives in src/aeolus1d/io/case_upgrades.py.
  • load_case reads the marker before constructing any dataclass. It refuses a forward-version file with a clear "upgrade aeolus1d" message, identity-loads a current-version file, and walks any registered CaseUpgradeSteps to migrate an older parsed dict to the current shape. The marker is popped from the parsed [case] dict so it never reaches CaseHeader — it is file metadata, not a solver knob, and is absent from the runtime model.
  • The marker lives inside [case], not in a separate table. aeolus1d expand rewrites only the [case] block (every other section is preserved verbatim), so the one shared [case]-block formatter (_format_minimal_case_block / _format_full_case_block in expand_case.py) is the single source of truth for emitting the marker. The GUI save_case, the CLI import/optimize writers, and expand all route through it. A top-level key or a [meta] table would sit outside the only block expand manages and could drift.
  • An unversioned file is treated as v1. Files written before this framework lack the key; v1 is the current shape, so the load is bit-identical with or without it — no warning. The bundled examples are intentionally not rewritten to add the key; they migrate the first time they pass through aeolus1d expand or a GUI save.

CaseUpgradeStep carries an apply(dict) -> dict callable, unlike the HDF5 UpgradeStep (which is descriptive-only, because its resume reader bridges gaps via .get(default) fallbacks). A TOML upgrade genuinely has to transform the parsed dict — rename a key, move a field between sections, convert units — so the step owns that transform and the framework applies the chain in order.

Alternatives considered

  • Make schema_version a real CaseHeader field. Rejected: it would pollute the runtime solver model with file metadata, and the version must be read from the raw parsed dict before CaseHeader construction anyway (that is the whole point — an old file may not be constructible by the current dataclass until it is upgraded).
  • A separate [meta] table or a top-level schema_version key. Rejected: it falls outside the [case] block that aeolus1d expand rewrites, so the marker could silently drift from the format the file actually uses.
  • Defer emitting the marker until the first breaking change (CURRENT > 1). Rejected: a versioning scheme whose version is invisible until v2 can't distinguish a deliberate v1 from an ancient unversioned file, and can't be tested end-to-end today. Emitting schema_version = 1 now makes the scheme real and pinned immediately.

Consequences

  • The next backward-incompatible case-format change has a named, tested migration path: bump CURRENT_CASE_VERSION, append a CaseUpgradeStep, add a pin in tests/unit/test_case_upgrades.py. The bump procedure is documented in the case_upgrades module docstring.
  • Forward-version files fail fast with an actionable message instead of a confusing deep parse error.
  • New writes (GUI save, aeolus1d expand, import, optimize) emit schema_version = 1; existing files keep working unversioned and migrate on their next rewrite.
  • aeolus1d expand only migrates the [case] block; a future format upgrade that restructures other sections will require a load_case → re-save round-trip (or an extension to expand), since expand preserves those sections verbatim. This is noted for the first real upgrade step; it does not arise at v1.

References

  • src/aeolus1d/io/case_upgrades.py — the versioning framework.
  • src/aeolus1d/io/parsers.py (load_case) — the read-side hook.
  • src/aeolus1d/io/expand_case.py — the shared [case]-block formatter (emit) + forward-version guard.
  • tests/unit/test_case_upgrades.py — chain machinery + load/expand pins.
  • src/aeolus1d/io/schema_upgrades.py — the HDF5 checkpoint sibling this mirrors.