Decision
Fixture provenance belongs to each array, not merely to the file.
The Phase 5 collector executes the pinned Transformers Qwen4Exp implementation to capture values that upstream actually computes. An independent NumPy/scalar implementation checks the same math. That provides a strong oracle for exposed module outputs.
Some necessary test inputs and states do not have an upstream capture path. Token sequences are selected by the fixture author. Hashed PLE row identities and persistent cache images may need to be constructed so a specific transition can be tested. Tie, overflow, and rollback controls can also be contractual rather than upstream-authored.
Those arrays are useful—but they are not upstream captures.
The self-approval trap
Suppose a local implementation defines a cache layout, generates an expected cache image using the same assumption, and stores both beside genuine upstream outputs. If the whole file is labelled “generated from Transformers,” a later agent may treat the cache image as independent evidence.
The implementation has then approved itself while borrowing upstream’s authority.
Provenance classes
A practical fixture can distinguish at least:
- upstream capture — emitted by the exact pinned implementation;
- independent cross-check — recomputed through a separate algorithm or language;
- contract control — deliberately selected input, state image, tie, overflow, or failure case;
- derived metadata — dimensions, offsets, or digests computed from already identified sources.
Each class answers a different question. None should silently inherit the authority of another.
Regeneration rules
The networked collector may be needed to reproduce upstream arrays, but the normal repository test should stay offline. It can pin source revision, tool versions, source hash, per-array provenance, and aggregate fixture digests. Offline validation then detects drift without downloading a framework during every build.
The collector and the native implementation should not share the optimized helper being tested. Agreement is valuable only when likely defects are independent.
Agent checklist
Before trusting a golden fixture, ask:
- which exact function produced each expected array;
- whether that function is upstream, independent, or part of the implementation;
- which arrays are intentionally synthetic controls;
- whether regeneration pins immutable source identity;
- whether ordinary checks detect changes without network access;
- what the fixture cannot say about full-model behavior.
Failure boundary
Phase 5’s fixture is intentionally tiny and model-free. It closes graph ordering and state-transition questions; it does not replace checkpoint-backed layer captures, final-logit parity, quality evaluation, or performance measurement.