OpenGrad · Study 002 · in progress

Relabelling, on-policy distillation and speculative decoding.

Study 002 is everything after the Study 001 freeze. It has no results yet. Each experiment below is pre-registered before any GPU time is spent, and its results will be reported against the frozen Study 001 checkpoints, not in place of them.

SCOPE

Three lines of work, none yet run.

01
Dataset corrections, starting with refusal relabelling
A heuristic detector finds 18,114 of Canonical-v2’s 173,237 records are refusal-shaped targets labelled ANSWER. First measure the detector’s precision on a hand-labelled sample. Then relabel in a new corpus version (Canonical-v2 stays as published), retrain M0 with only the labels changed, and re-measure GSM8K zero-shot, IFEval, MMLU-Pro and the tool-policy metrics against pre-registered thresholds. A null result is publishable.
Blocked on preflight
02
On-policy distillation (M2)
Study 001 did not run it. The trainer seams exist (teacher and rollout providers), but the live training path is not implemented and the one recorded M2 attempt is mock-only. It needs a working live path, a teacher that fits the available hardware, and its own pre-registration and confirmatory evaluation.
Not started
03
Speculative decoding integration
Target-attached or native speculative decoding, to improve decode speed without a separate draft model. The benchmark suite has a speculative tier, but no runtime support exists and nothing has been measured.
Planned · GPU required

WHAT STUDY 002 STARTS FROM

Study 001’s checkpoints, corpora and evaluations, frozen at the OpenGrad tag study-001. None of them is modified, relabelled or replaced.
The promoted checkpoint refuses all 1,319 zero-shot GSM8K questions. The regression is associated with tool-policy post-training on When2Call-derived data; causation is not established.
Cross-model replication and joint capability-efficiency studies stay on the roadmap, not assigned to Study 002.

Read Study 001 Study scopes in the repository Roadmap