# ORCHID ESPEI bundle: schema

## Layout

- `experimental/<SYS>/<source>.<file>.json`, `model/<SYS>/...`, `invariants/<SYS>/...`: corpus datasets.
  `<SYS>`: elements sorted, uppercase, joined by `-` (as in `literature/`); `<source>`: the corpus folder the dataset
  was written in (the assessment / TDB it was mapped through); `<file>`: the corpus file name.
- `dft/<literature|mp|oqmd>/<SYS>/<TDB stem>.<PHASE>.json`: DFT formation enthalpies of endmembers of one phase of one
  TDB file (`solver` datasets, output `HM_FORM`, T = 298.15 K, P = 101325 Pa).
- `dft/CALIBRATION.json|md`: DFT sigma calibration.
- Optional phase constraints (ZPF; tagged; never mixed with measured data): `derived/invariant-brackets/<SYS>/<TDB>.<file>.json`
  (one per source invariant dataset, same phase names), `derived/liquid-stability/<SYS>/<TDB>.liquid-stability.json`,
  `assessed/<SYS>/<TDB>.<DIAGRAM>.json` (assessed-phase-assemblages in that TDB's phase names).
- `UPSTREAM_CORRECTIONS.md`: literature records the export had to differ from (reference states, rows read twice).

## Dataset files (ESPEI 0.9.1 format)

ESPEI-native keys: `components`, `phases`, `conditions` (`P` Pa, `T` K, `X_<EL>` mole fractions), `output`, `values`
(J/mol-atom, J/mol-atom/K, activities; ZPF: `[phase, [components], [mole fractions]]` per vertex, `null` where not
printed), `reference_states` (or `reference_state` for activities), `solver` (DFT: `mode`, `sublattice_site_ratios`
and `sublattice_configurations` exactly as in the TDB phase model), `tags`, `weight` (scalar for equilibrium and
activity datasets; one per value for DFT `solver` datasets; absent = 1). Values removed by the inclusion gate are cut
from `values` and the matching `conditions` entries.

| key | meaning |
|---|---|
| `reference` | source paper(s) of the values (where measured / calculated): `Author Year, doi:...`; DFT databases by name and release |
| `bibtex` | BibTeX key(s) in `references.bib` (space-separated when several) |
| `comment` | literature: `ORCHID literature values, Version 1, machine-extracted. Evidence class: <class>. ORCHID observation ids (...): ORC-...`; DFT: `0 K DFT used as 298.15 K enthalpy. ...` with TDB, SHA-256, phase, ids and the eV conversion |
| `orchid.evidence_class` | `experimental`, `model` or `dft` |
| `orchid.observations[]` | per value, in order: `id` (ORCHID literature id), `reader_agreement` (`multi_reader`, `single_reader`, `single_reader_text_layer`; absent when not classified), `paper`, `measured_in`, `caveats` |
| `orchid.papers`, `orchid.dois` | paper ids (`literature/papers.json`) and DOIs involved: source, containing paper, printed cited works |
| `orchid.source_dataset`, `orchid.values_dropped`, `orchid.espei_evaluation` | corpus file; values removed by the gate; invariant ZPF driving forces against the assessment |
| `orchid.mapping[].also_read_as`, `orchid.mapping[].evidence_identity` | source-verified duplicate IDs retained with the canonical fitting observation; decisions in `DFT_EVIDENCE_IDENTITIES.json` |
| `orchid.dft_source`, `orchid.tdb`, `orchid.registry_phase`, `orchid.mapping[]` | DFT: source; TDB file (name, path, SHA-256, phase); phase-registry id; per value: id (MP / OQMD entry or ORCHID id), compound, mapping basis, space group, match quality |
| `orchid_value_bases` | ZPF from ranges / central values: how a temperature was derived, keyed by ORCHID id |
| `orchid_model_evidence` | model datasets: `tdb` (name, path, SHA-256, catalog entry), `link_basis`, `model_citation`, `model_reproduction` |
| `orchid_uncertainty` | see below |
| `orchid_phase_assignment` | literature property datasets: `phases`, `basis` (counts: `printed_labels`, `caption_names_liquid`), `rule`, `outside_calculated_field[]` {id, phase, x, T_K} (flag only), `endmember` {tdb, path, sha256, phase, reference} for compound formation datasets |
| `orchid_certification` | literature datasets: `basis` (`pycalphad_invariant_agreement`), `all_values_certified`, `n_certified`, `values[]` {id, espei_certified, calphad_check}; tag `calphad_certified` when all values are certified |
| `orchid_reference_state` | activities (E5): `element`, `phase` written, `basis` (`printed` / `not_printed_implied`), `quote`, `where`, `record_reference_state_agrees`, `written_before_audit` |
| `orchid_weight_floor` | thermochemical data (E6, **superseded** 2026-10-24): `superseded` (text), `suggested_weight` (the former opt-in suggestion), `weight_before_2026_10_24`, `weight_written` (from sigma_effective), `sigma_floor` (+ unit), `rule` |
| `orchid_weighting` | split datasets: `rule`, `parent_dataset`, `parent_value_index`, `parent_n_values`, `parent_weight`, `parent_values_dropped`, `source_dataset`, `measured_in`, `correlation_status`, `correlation_note`. Source linkage does not imply independence; covariance is not quantified. `UNCERTAINTY_SPLITS.json` gives the complete parent-to-child mapping. |
| `orchid_uncertainty_budget` | phase-constraint datasets (assessed/, derived/): per point, the temperature and composition budget (components as in `orchid_uncertainty.values[].budget`) |
| `orchid_zpf_encoding` | ZPF (E2): number of line-compound vertices written with composition null, hyperplanes added, rule |
| `orchid.observations[i].also_reported_as` | E3: the same ZPF region from other datasets (`id`, `paper`, `measured_in`, `dataset`) |
| `orchid.observations[i].also_read_as` | E4: ids of the same printed row read again (not written as separate values) |
| `orchid.observations[i].evidence_identity` | canonical identity and alternate unit representations of one observation; unresolved experimental conditions remain excluded from fitting. Source decisions: `MGPB_EVIDENCE_REVIEW.json` |
| `orchid.points` (derived brackets) | per point: `source_observation`, `side` (above/below), `T_invariant_K`, `offset_K`, `T_sigma_K`, `T_sigma_basis`, `x0`, `hyperplane_delta`, `reaction_printed`, `reaction_type`, `phases_above`, `phases_below` |
| `orchid.basis` (liquid stability) | `T_liq_max_K` and the temperatures it was taken from |
| `orchid.record_ids`, `orchid.confidence` (assessed) | assessed-phase-assemblages record id and confidence per point; `target_tdb`, `points_left_out` |

## Target TDB provenance

For a downstream fit, record the target TDB SHA-256 and match it to `phases/tdb_provenance.json`.
`generation_method` and `tags` distinguish `phaseforge` from `phaseforge_plus`; `potential` records the MLIP name
and checkpoint digests. Preserve these in the fitting run manifest together with the input-data version and
weight overrides. These model-provenance tags are not ESPEI evidence tags and must not be passed to `apply_tags`
as weight rules. An unmatched TDB digest has unknown provenance; never infer it from its filename.

## Uncertainties (`orchid_uncertainty`)

`{unit, default_sigma {value, name}, weight_rule, sigma_rule, values[]}`, one entry per value in value order:

| field | meaning |
|---|---|
| `sigma` | the printed standard uncertainty in the dataset's units (`unit`) (DFT: the calibrated sigma), or `null` when none was printed (never invented) |
| `sigma_basis` | `printed`; `not printed`; `dft_calibrated`; `zpf (...)` |
| `budget` | the uncertainty budget in the dataset's units: `sigma_printed`, `sigma_reading`, `sigma_method`, `sigma_composition`, `sigma_scatter`, `sigma_conversion`, `sigma_effective` (activities: also `*_mu_J_per_mol`), `method_class`, `method_basis` (calibrated / prior / calibrated vs experiment), `method_n_pairs`, `method_entry` (entry of experimental/CALIBRATION_EXPERIMENT.json; DFT: dft/CALIBRATION.json), `sigma_effective_reason` when none; ZPF: `{temperature_K, composition_mole_fraction}` budgets |
| `sigma_effective_used` | the sigma_effective the weight uses (ESPEI's default, with `sigma_effective_note`, only where the budget has none) |
| `printed` | `{value, unit, interval, coverage, lower?, upper?, note?}`: printed uncertainty as published (`coverage`: `1sigma_assumed` for a bare ±x, `range_half_width` for a printed range) |
| `sigma_mu_J_per_mol` | activities: R T sigma_a / a, the chemical-potential sigma ESPEI's activity residual uses |
| `calibration` | DFT: `{source, stratum, used (stratum or source_level), leave_system_out, system_left_out, n, sigma_robust_kJ, bias_median_kJ_not_applied, ci95_sigma_robust_kJ_full_stratum, n_full_stratum}` |
| `temperature_sigma_K`, `temperature_sigma_basis` | ZPF: printed temperature uncertainty |
| `weight_used` | the ESPEI weight that applies to this value; `weight_per_value_equivalent` = default sigma / this value's sigma_effective |
| `printed_uncertainty_not_used` | model-calculated values: a printed uncertainty kept as printed, not used |

Caveat codes (published flags of the value): `magnitude_suspect`, `system_mismatch`, `source_disputed`,
`dft_outlier_unexplained`, `basis_inferred_from_magnitude`, `basis_corrected_from_formula`, `sign_from_heading`,
`unsigned_in_print`, `judge_adjudicated`, `page_unreadable`, `plausibility_fail`, `energy_basis_unstated`.

## index.json / index.csv

One row per dataset: `path`, `folder`, `system`, `components`, `output`, `phases`, `n_values`, `n_values_dropped`,
`T_min_K`, `T_max_K`, `evidence_class`, `reference`, `bibtex`, `dois`, `papers`, `observation_ids`, `reader_agreement`
(counts by code; `not_classified` where the record builder did not classify), `reader_agreement_source`, `caveats`,
`tags`, `weight` (`per_value` for DFT), `sigma_unit`, `sigma_min`, `sigma_median`, `sigma_max` (over printed or
calibrated sigmas), `sigma_basis_counts`, `dft_calibration_stratum`, `tdb`, `tdb_sha256`, `model_link_basis`,
`espei_median_abs_driving_force_J`, `source_dataset`. CSV: lists joined with `;`, objects as compact JSON.

## validation.json

`espei_version`; `folders` (files and datasets loaded per folder); `invalid_final` (empty); `excluded` (rejected by
ESPEI, with reason code); `dropped_datasets`; `dropped_values`; `dft_excluded` (compound or compound x TDB model, with
reason); `fixes` (E1-E6 counts, E3 merges, E5 audit per activity dataset); `upstream_corrections`; `phase_constraints`;
`summary`. index.json / index.csv also list the phase-constraint datasets (`evidence_class` `derived` or
`assessed_phase_diagram`); `totals.datasets` / `totals.values` count the literature and DFT datasets only.