Sakana: Continuous Thought Machines -- Neural Dynamics and Synchronization

The paper presents the **Continuous Thought Machine (CTM)**, a novel AI model that draws inspiration from biological brains, where the precise timing and interaction of neurons are crucial for processing information. Departing from traditional deep learning methods that largely ignore these temporal dynamics for computational simplicity, the CTM explicitly incorporates neural timing as a core element. Its key innovations include processing a history of incoming signals at the individual **neuron level** using unique parameters for each neuron, and utilizing **neural synchronization**—how neurons fire together over time—directly as the internal latent representation for computation and interaction with data. By allowing processing to unfold over an internal sequence of "thought steps" or "internal ticks" decoupled from input data, the CTM aims to capture essential temporal dynamics while remaining computationally feasible. The goal is not necessarily to achieve state-of-the-art results everywhere but to demonstrate a step towards more biologically plausible and powerful AI, showing versatility and strong performance across a range of tasks requiring complex sequential reasoning, such as ImageNet classification, 2D maze solving, sorting, parity computation, question answering, and reinforcement learning. The CTM can also adapt its computation, using fewer internal ticks for simpler tasks and more for complex ones. https://arxiv.org/pdf/2505.05522