Person
Tri Dao
Tri Dao is chief scientist of Together AI, a US company providing cloud infrastructure for open AI models, a role he holds alongside an assistant professorship at Princeton University. He completed his PhD at Stanford and is known for the FlashAttention algorithms and, with Albert Gu, the Mamba state-space model architecture.
Appears alongside
Featured in threads
Tracks
- Ideas & essays 2
- Compute & infrastructure 1
Mamba paper proposes selective state-space models as a transformer alternative
Letting the model's internal state-update rules depend on the input let a 3-billion-parameter Mamba match transformers twice its size while running five times faster.
Ideas & essays
FlashAttention makes exact attention IO-aware
Reordering attention around GPU memory rather than approximating it cut training time and unlocked longer sequences — and became default infrastructure.
Ideas & essays · Compute & infrastructure