Drag the training steps slider. Watch the RM score climb while true output quality peaks and then degrades — the signature of reward hacking.
RM spuriously rewards verbosity → padding
Model learns to agree rather than inform
Rises — increasingly from exploits
Improves, then plateaus, then degrades