SPINE · real study corpus · spine-corpus-0.1.1
Analyze your studies with SPINE
This mode uses the responses and binary ratings already stored in each study. It compares filtering before the objective with soft preferences and task-only selection. No model API calls are made.
The criteria are textual measures. Judge agreement does not measure a person’s actual safety, autonomy, or a model’s internal mechanism. This is an experimental protocol.
Full protocol and interpretation limits
- version
- spine-corpus-0.1.1
- purpose
- Offline replay of filter-before-objective ordering on actual stored responses and binary ratings; experimental textual proxies, not human-state or robot validation.
- mapping
- C1 task utility, C3/C4 joint textual constraints, C5 release-to-own-resources proxy by default. IDs are configurable and the actual rubric wording is frozen per study.
- selection
- One latest valid rating per response/judge/criterion within the latest compatible rubric. Modes never mix. Failed/empty responses, invalid scores and incompatible rubrics are reported, not imputed as zero.
- panel
- V6.5 excluded judges are omitted by default. Hash/seed split of all eligible judges: one third held out, at least 2; policy and reference sets are disjoint. LLM judges are correlated and are not independent ground truth.
- belief
- Joint constraint pass requires the same policy judge to pass every configured constraint. Beta(1,1) working prior: p=(passes+1)/(n+2), var=(passes+1)(failures+1)/((n+2)^2(n+3)). This is a judge-consensus proxy, not calibrated human viability.
- gate
- Required p = 1 - epsilonMax * exp(-kappa * var). L2 filters then maximizes task score, with C5 breaking exact ties. Soft maximizes task + lambda*log(p); task-only maximizes task. Stable model/run ID resolves remaining ties.
- rounds
- Candidates are actual responses to the same study prompt grouped by stored run_index. These are offline replicate groups, not temporal trajectories. Groups with fewer than two eligible candidates are excluded explicitly.
- reference
- After selection only: compare joint pass fraction of held-out judges with referenceAgreement, if enough complete held-out ballots exist. Disagreement is not observed harm or a clinical risk label.
- withdrawal
- C5 is a text-level release proxy only. KL saturation, demonstrated resilience, AE/SE/SS, physical viability and causal intervention effects are not inferred from this corpus.
- uncertainty
- 95% percentile bootstrap, 500 resamples of response-level panel means per model. Conditional on this fixed prompt/panel; shared judge and prompt dependence prevents a population or clinical interpretation.
- comparison
- Study and model results retain their own denominators. Vulnerability comparisons are descriptive prompt comparisons; two L1 variants are not pooled or counted as separate vulnerability levels.