Sakana AI's Continuous Thought Machine (CTM) is a novel neural architecture that treats time as a first-class citizen in AI. Unlike transformers, CTMs use an internal time dimension decoupled from input data, allowing the model to 'think' across multiple internal ticks. Key components include a synapse (LSTM-based cross-neuron interaction), per-neuron MLP models that track activation history for temporal dynamics, and a neural synchronization matrix capturing neuron co-activation over time. CTMs support adaptive compute at inference, stopping early on simple inputs and thinking longer on complex ones. Evaluated on tasks like 2D maze solving, CTMs outperform LSTM baselines and generalize to larger mazes than seen during training, though large-scale evaluation remains pending.
Sort: