d12 SFT (1 epoch, ~612 steps)

SFT on d12 pretrain checkpoint

Pretrain source: d12 H100 SSSL final (val_bpb=0.854, step 2205)

Config:

• Default SFT mixture: SmolTalk 460K + MMLU×3 + GSM8K×4 + Identity×2 + Spelling 280K = ~1.07M rows
• num_iterations=700 (cap; epoch-based stopping at ~612 steps)
• device_batch_size=32, total_batch_size=524288 (inherited from pretrain)
• upload_checkpoint=1 for downstream RL

Expected: ~15 min on 1×H100, ~$3-5

Status: COMPLETED

Results:

• 88 optimizer steps (1 full epoch), 1.24 min training
• val_bpb: 1.94 → 0.5714 (pretrain was 0.854)
• loss: 3.30 → 1.76
• MFU: 42%, step_time: ~0.96s, peak_mem: 28.8GB
• Checkpoint uploaded to MLflow ✅
• Actual cost: $1.79 (estimated $13.94)

Observations:

• Full epoch = 88 optimizer steps = ~700 dataloader yields
• num_iterations=700 was exactly right as a cap
• LR warmdown completed normally (lrm 1.0 → ~0)