aeolus1d — validation report, v0.3.0¶

Status: DRAFT, frozen for the v0.3.0 (pre-release) line. Frozen: 2026-06-07. Scope of this document: a citable, non-moving snapshot of every validation workflow, its reference type, its current verdict, the mesh it was measured on, and its uncertainty class. The validation dossier is the living companion — it drifts with the code; this report is the one to cite. When v0.3.0 is tagged, cite this file at that git tag (and the Zenodo DOI minted on the tag, once available).

⚠️ ERRATUM 2026-08-30 — W1's reference type is overstated¶

The frozen body below is deliberately not edited; this note is the correction. The reference-type legend in §3.1 says "experiment … only W1 is here", and the W1 row cites ricardo_ktm among its sources.

ricardo_ktm compares against the OEM column of Tables 2 + 3 of IMECE2017-70638, which is Ricardo WAVE's own simulated baseline — the figure the paper's tuning study computes a percentage gain over. It is code-to-code, not experiment. The paper's chassis-dyno curves exist but are not what this project scores against, and its own Figs. 3 + 4 show the WAVE model running well above them, with drivetrain and roller losses named as the reason.

This was wrong when the document was frozen, not superseded by later work — the underlying facts have not moved since 2026-06-07. A frozen, citable report carrying a wrong classification is worse than a living one carrying it, which is why this erratum sits above the body rather than in the changelog. mendeley_ethanol_water remains genuinely experiment- referenced, so W1 is mixed, not reclassified wholesale.

Evidence and the corrected text: examples/validation/ricardo_ktm/README.md and docs/validation-dossier.md, both updated 2026-08-30.

This report exists so that an outside engineer — or a reviewer — can read a number, see exactly what kind of evidence stands behind it, and judge whether to rely on it. It deliberately separates verification (are the equations solved correctly?) from validation (does the answer match reality?), and it labels every validation row by reference type, because a self-consistency check and an experimental match are not the same evidence even when both say "pass".

1. Provenance¶

Field	Value
Report version	v0.3.0 (pre-release draft)
Frozen date	2026-06-07
Solver	aeolus1d 1D compressible-Euler pipe-network solver
Reference baseline host	Python 3.10, 16-core WSL2, NumPy 2.2.6, Numba 0.64.0 (see `benchmarks-baseline.md`)
Key physics change since v0.2	ADR-0008 — compressible-orifice cylinder↔valve coupling is now the default (`cylinder_orifice_valve_flow = True`). This conserves mass across the valve and re-baselines every engine-breathing number below.

Absolute kernel timings are machine-relative (the dev host shows ~2.3× timing noise; numba fastmath is cache-sensitive at 1e-14). The portable results are the ratios, the step-count collapse, and the physical agreements — not absolute seconds.

2. Verification — are the equations solved correctly?¶

Dataset-independent; these gate every MR (or run nightly) and do not depend on any external data.

Check	What it pins	Gate	Test
Spatial order (flux)	HLLC + MUSCL/MP5 vs analytic Sod; observed order on a smooth wave	MUSCL ≥ 1.8 (lower bound away from extrema)	`tests/validation/test_sod_against_analytical.py`, `tests/integration/test_smooth_advection.py`
Temporal order (per integrator)	Formal order of all seven time integrators	IE≈1, SSP-RK2/BDF2≈2, SSP-RK3≈3, RK4≈4, RK5/Radau-IIA(3)≈5	`tests/integration/test_temporal_order.py`
Source-term discretisation (MMS)	One manufactured-source proof: an energy-source closure converges at formal order, and does not converge without the source	methodology proof (not MMS over all closures)	`tests/integration/test_mms_source.py`
Conservation (discrete balance)	flux-in − flux-out − ∫source ≈ 0 for mass/momentum/energy/species under Strang splitting	discrete balance; machine-precision only for the closed source-free config	`tests/integration/` balance tests
Valve mass conservation	cylinder↔runner orifice coupling conserves total mass	`< 1e-10` (orifice); legacy coupling demonstrably leaks `> 1e-4`	`tests/integration/test_cylinder_valve_conservation.py`
Reproducibility (golden snapshots)	final state of ~6 canonical cases, ≥1 an independently-validated anchor	bit-identity on the pinned arch (fastmath OFF); measured cross-arch tolerance	`tests/regression/`

Verification verdict: the numerics converge at their design rates as scheme-specific lower bounds; conservation holds as a discrete balance; the valve coupling is conservation-pinned. This is the gate that the numbers below are produced by trustworthy numerics before any tolerance was tuned to data.

3. Validation — does it agree with reality?¶

Every row carries its reference type. Read the type before the number.

#	Workflow	Reference type	Verdict (v0.3.0)	Mesh / point	Uncertainty class
W1a	KTM 510 cc SI breathing	experiment	breathing physics verified (conservation-pinned, < 1e-10); quantitative 6/10 at the 25 % gate (recommended `--throttle --wall-heat`; 3/10 bare), down from 8/10 legacy same-invocation — re-validated 2026-06-09: with mass-conservative breathing the idealised cam over-admits air across RPM (η_v ≤ 1.18; every point over-predicts torque), not a solver regression; per-RPM re-cal owed (OEM cam)	10 ops, 1000–10000 RPM	calibrated; GCI-not-applicable
W1b	Mendeley ethanol/water SI	experiment	44/60 under both legacy and the shipped-default orifice coupling — re-validated 2026-06-09; fixed-fuel suite so the orifice is ~1 % (near-neutral), no re-tuning. 0–30 % reads 40/40 under orifice (gate-edge realignment, not a model gain); residual at 40–50 % water is structural (charge-cooling / const-γ)	60 ops (5 RPM × 6 blend × 2 CR)	calibrated; GCI-not-applicable
W2	Turbo matching	self-consistency	✅ 3/3 — power balance to ~1e-7 rel; matched power monotone in wastegate; surge/PR on-map	3 wastegate points	self-consistency (no external data)
W3	AVL C02006 DPF/CSF regen (aftertreatment drive cycle)	code-to-code	✅ runs end-to-end in ~6.5 min / 500 s; Non-Cat T/soot/Δp within calibrated bands; `co_conv_cat` structurally blocked (no CSF oxidation kinetics — ~100 % gap, not a tolerance issue)	n_cells = 20 (matches AVL grid)	calibrated; GCI-not-applicable
W4	Acoustics / NVH	analytic + code-to-code	✅ TL vs analytic resonance; GT-Suite TMM cross-check < 0.5 dB across 10–200 Hz (5.45 dB @ 990 Hz where GT adds viscothermal/higher-mode)	plane-wave regime	analytic reference
W5	Electrified drive cycle	self-consistency	✅ coherent SoC trajectory + energy bookkeeping; battery clamps `[0,1]`	full cycle	self-consistency (no external data)
EPW	AVL EPW HD diesel (steady)	code-to-code	breathing fixed (vol-eff ~6 %→74 %); IMEP_High/PFP within a few % at anchors; blind 2nd-point T_EXH −5.6 % / −1.8 %; residual BMEP ~30 % low + NOx over-prediction	3 anchor ops + 2 blind ops	partly blind (T_EXH); residual gaps tracked

3.1 Reference-type legend¶

experiment — compared against published experimental engine data. The strongest evidence type; only W1 is here.
code-to-code — compared against another simulator's output (AVL Cruise M digitized traces, GT-Suite TMM). Useful, not equivalent to experiment.
analytic — compared against a closed-form solution (acoustic resonance, Sod). Exact where it applies.
self-consistency — the solver is shown to agree with itself (conservation, energy bookkeeping). This is not external validation; W2 and W5 are self-consistency checks and are labelled as such throughout.

We do not claim un-tuned predictive closure of any gate whose physics is structurally missing. Classification (full detail on the scope page):

Blind-predictable (verified, no tuning): W1 breathing physics (mass conservation), W4 acoustics (analytic), EPW breathing / IMEP_High / PFP + the blind 2nd-point T_EXH check.
Calibrated (not blind): W1 KTM/Mendeley quantitative match, W3 C02006 Non-Cat channels, EPW wall-heat coefficient.
Structurally blocked (cannot pass without new physics): W3 C02006 co_conv_cat — aeolus1d's passive DPF brick has no CSF CO/HC oxidation kinetics.
Self-consistency (not a prediction): W2 turbo, W5 electrified.
Not yet closed (named v0.3.0 gate): the coupled DOC + DPF + SCR aftertreatment train.

5. Quoted-number uncertainty (GCI / V&V-20)¶

GCI is computed only on the handful of numbers actually quoted in the docs / marketing where the discretisation term u_num is the meaningful one. For the calibrated lumped-closure cases (DOC Voltz, SCR Temkin, C02006 regen) GCI is not applicable: per ASME V&V-20, u_val = √(u_num² + u_input² + u_D²) is dominated by the fitted model-form term u_D, so a GCI u_num is misleadingly small. Those pass_criteria.toml files carry an explicit GCI-not-applicable — calibrated banner.

The SCR With-NO NO-conversion is the canonical not-mesh-converged example: 45 % @ 25 cells → 30 % @ 50 → 19 % @ 100 (the agreement at 50 cells is a coarse-mesh numerical-diffusion artefact). It is quoted only with its mesh.

6. How to reproduce¶

Every row is runnable from a fresh checkout (see the dossier for the full command list). Engine suites that depend on local-only datasets auto-skip with exit 0 when the data is absent — CI-safe on fresh checkouts.

python examples/validation/ricardo_ktm/run_validation.py          # W1a (local data)
python examples/validation/mendeley_ethanol_water/run_validation.py  # W1b (local data)
python examples/validation/turbo_match/run_validation.py          # W2
aeolus1d run examples/avl_cruise_m/c02006_dpf_csf_regen_trans.toml  # W3
python examples/validation/gt_tmm_pipe/run_validation.py          # W4
aeolus1d drive-cycle examples/86_electrified_drive_cycle.toml       # W5
python examples/validation/avl_epw_diesel_steady/run_validation.py --anchor-bcs  # EPW

7. Change log for this report¶

Governance rule: this report is frozen — after the freeze date its content changes only via an erratum entry recorded in this section (date, commit, what changed, why); any edit without a matching entry here violates the report's "citable, non-moving" contract.

ERRATUM 2026-08-08 (this cycle). The EPW row's residual-gap clause is stale on both counts and is corrected here rather than in the frozen table. It reads "residual BMEP ~30 % low + NOx over-prediction". Re-measured on current main, 3 anchor ops, both modes:
BMEP is 6.6-8.9 % low, not ~30 %. The ~30 % figure predates the cylinder-breathing and scheme fixes.
NOx: the frozen row's SIGN is wrong, and this erratum's own first correction was wrong too — amended 2026-08-08. The historic over-prediction (14 117 ppm) was a residual runaway, fixed by !396/!397. This erratum then recorded 0.000 ppm as structural, "the well-mixed burnt zone peaks ~1960 K". It was the integration window. Narrowed to [IVC, EVO), the same unchanged trajectory yields 306 / 312 / 318 ppm against AVL 293 / 240 / 509 — which a too-cold zone could not produce. NOX remains report_only, now because the value is set by the T_active_min gate rather than by kinetics (sweep it 1500→2000 K and the answer walks 117→1115 ppm). The live reason is in pass_criteria.toml. Also newly available and absent from the frozen row: CV-A and CV-B both close (CV-B at 1.302 / 0.808 / 0.701 % of fuel against a 3 % threshold), and T_EXH carries u_num = +/-0.035 degC from a three-level mesh study (p = 1.286, in the asymptotic range) - so the frozen report's "GCI-not-applicable" labelling no longer holds for this suite. The living numbers are in the validation dossier; this entry exists so a reader of the frozen table is not misled about the SIGN of the NOx gap.
2026-06-07 (v0.3.0 draft). First frozen report. Re-baselined W1 + EPW for the ADR-0008 orifice valve coupling (now default); relabelled W2/W4/W5 by reference type (self-consistency / analytic); added the blind-vs-blocked classification and the GCI-not-applicable labelling.
ERRATUM 2026-06-09 (commit feda0bd). W1a (KTM 510) row updated after re-running the suite on current main under the shipped-default orifice coupling: verdict revised from "3/10 at the 25 % gate (was 7–8/10 legacy)" to "6/10 recommended invocation (--throttle --wall-heat) / 3/10 bare, down from 8/10 legacy same-invocation" — the original row mixed invocations; the re-measurement separates them. Recorded here retroactively (the edit predates this governance rule).
ERRATUM 2026-06-09 (commit 0bbe6a2). W1b (Mendeley ethanol/water) row updated after re-validating on the shipped-default orifice coupling: the frozen row's "44/60 pre-orifice; re-validation owed; not re-runnable in CI" framing rested on false premises (the suite had been running on the legacy scaffold default all along) — re-measured 44/60 under both couplings (fixed-fuel, orifice ~1 % / near-neutral). Recorded here retroactively (the edit predates this governance rule).
ERRATUM 2026-08-15. Two W1b claims aged out of truth. (1) The identical no-flag invocation scores 39/60 on the 2026-08-15 tree — and the same-day bisect attributed the 5-point difference to 643fc3fb (2026-07-01): until that fix the Livengood-Wu knock integral silently ratcheted a 10° timing retard on the suite's scaffold, deflating predicted IMEP and flattering five gate-edge points, so the 44/60 above was measured with a bug cancelling part of the model's over-prediction and 39/60 is the honest score of the same physics (closure with the measured mechanism in BACKLOG_HISTORY.md, "Mendeley baseline drifted 44→39"). (2) The row's "residual at 40–50 % water is structural (charge-cooling / const-γ)" diagnosis was implemented and measured over all 60 points on 2026-08-15 and is retracted: charge cooling moves the prediction away from the data (39/60 → 36/60; predicted BMEP 9.27 → 9.55 bar vs a measured 5.73 at 50 % water). The measured gap is combustion deterioration under water dilution; see the suite README's retraction block and BACKLOG.md for the replacement physics item. The frozen row's text is left as frozen per this section's rule.