Transformer & MoE Architecture Visualizer
Standard Transformer
Mixture of Experts (MoE)
Side-by-Side Comparison
Component Information
Click on any component to learn more about it.
Try it yourself!
Process Token