# decoderstack-d12

Training-state checkpoints for the **DecoderStack d12** Colab notebook line
(`stacks/decoder-medium`, single-GPU 40GB-A100 configuration of the nanochat-style
d12: 286,261,730 params, 12 layers x 768, FA3 varlen, Muon + AdamW with per-step
schedule tables, bf16-live + uint16-mantissa fp32 masters).

## Layout

```
checkpoints/<run_name>/state_stepNNNNNN.pt
```

Each `state_step*.pt` is the **entire** training state at that step, written by the
walkthrough notebook's `write_state` and consumed by its `load_state`:

- `params[name]` — per-Param `w` (live weights), `mantissa` (uint16 low bits of the
  fp32 master, where applicable), `first_mntm`, `scnd_mntm`
- `step`, `t_step` — schedule position
- `rng` — torch CPU + CUDA generator states
- `batch` — the next unconsumed training micro-batch (`inputs`, `targets`,
  `cu_seqlens`), so the walkthrough dissects exactly the batch training would have
  seen next
- `config` — the full `StackConfig` (asserted on load)
- `code` — the notebook source that produced the state

Sign convention (since 2026-09-01): the backward pass is seeded with
`logits_grad = (onehot - probs) * softcap' * loss_scale`, so `grad`/`first_mntm`
hold **update directions** (the negated loss gradient) and both optimizers apply
`w + update`. States saved before this date carry `first_mntm` under the old sign
and should not be loaded into the current notebook.

Produced by the `DecoderStack d12 Walkthrough` notebook: Part 1 trains under the
real 1680-step schedule, stops at `cfg.walkthrough_step`, saves + pushes here; the
walkthrough part reloads and spells the last layer's forward/backward and one Muon
step out flat.

## Current states (run `40GB-A100_d12_walkthrough`, 2026-09-01)

| file | where in the run | val bpb |
|---|---|---|
| `state_step000250.pt` | step 250 of 1680 (past all warmups, lr at peak) | 1.077130 |
| `state_step000015.pt` | step 15 (mid lr/scalar/lm-head warmup) | 1.905353 |

To walk through a different state, set `cfg.walkthrough_step` to its step number
and run the notebook's walkthrough part.