'Grokking' paper documents sudden generalisation long after overfitting
OpenAI researchers found small networks could suddenly jump from memorisation to perfect generalisation well after apparent overfitting, opening a mechanistic-interpretability research thread.
Ideas & essays