Learning Algorithm Of Biological Networks

To try everything Brilliant has to offer—free—for a full 30 days, visit https://brilliant.org/ArtemKirsanov . You’ll also get 20% off an annual premium subscription Socials: X/Twitter: https://x.com/ArtemKRSV Patreon: https://patreon.com/artemkirsanov My name is Artem, I'm a graduate student at NYU Center for Neural Science and researcher at Flatiron Institute. In this video we explore Predictive Coding – a biologically plausible alternative to the backpropagation algorithm, deriving it from first principles. Backpropagation video: https://youtu.be/SmZmBKc7Lrs?si=qduaq9hRYIymFOiL Outline: 00:00 Introduction 01:15 Credit Assignment Problem 02:49 Problems with Backprop 06:05 Foundations of Predictive Coding 08:07 Energy Formalism 11:08 Activity Update Rule 15:12 Neural Connectivity 17:42 Weight Update Rule 20:58 Putting all together 25:15 Brilliant 26:27 Outro References: Bogacz, R., 2017. A tutorial on the free-energy framework for modelling perception and learning. Journal of Mathematical Psychology 76, 198–211. https://doi.org/10.1016/j.jmp.2015.11.003 Friston, K., 2018. Does predictive coding have a future? Nat Neurosci 21, 1019–1021. https://doi.org/10.1038/s41593-018-0200-7 Huang, Y., Rao, R.P.N., 2011. Predictive coding. WIRES Cognitive Science 2, 580–593. https://doi.org/10.1002/wcs.142 Keller, G.B., Mrsic-Flogel, T.D., 2018. Predictive Processing: A Canonical Cortical Computation. Neuron 100, 424–435. https://doi.org/10.1016/j.neuron.2018.10.003 Lillicrap, T.P., Santoro, A., Marris, L., Akerman, C.J., Hinton, G., 2020. Backpropagation and the brain. Nat Rev Neurosci 21, 335–346. https://doi.org/10.1038/s41583-020-0277-3 Marino, J., 2021. Predictive Coding, Variational Autoencoders, and Biological Connections. https://doi.org/10.48550/arXiv.2011.07464 Millidge, B., Salvatori, T., Song, Y., Bogacz, R., Lukasiewicz, T., 2022a. Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation? Millidge, B., Seth, A., Buckley, C.L., 2022b. Predictive Coding: a Theoretical and Experimental Review. https://doi.org/10.48550/arXiv.2107.12979 Millidge, B., Song, Y., Salvatori, T., Lukasiewicz, T., Bogacz, R., 2023. A THEORETICAL FRAMEWORK FOR INFERENCE AND LEARNING IN PREDICTIVE CODING NETWORKS. Millidge, B., Tschantz, A., Buckley, C.L., 2022c. Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs. Neural Computation 34, 1329–1368. https://doi.org/10.1162/neco_a_01497 Millidge, B., Tschantz, A., Seth, A., Buckley, C.L., 2020. Relaxing the Constraints on Predictive Coding Models. https://doi.org/10.48550/arXiv.2010.01047 Rao, R.P.N., Ballard, D.H., 1999. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2, 79–87. https://doi.org/10.1038/4580 Rosenbaum, R., 2022. On the relationship between predictive coding and backpropagation. PLoS ONE 17, e0266102. https://doi.org/10.1371/journal.pone.0266102 Salvatori, T., Mali, A., Buckley, C.L., Lukasiewicz, T., Rao, R.P.N., Friston, K., Ororbia, A., 2025. A Survey on Brain-Inspired Deep Learning via Predictive Coding. https://doi.org/10.48550/arXiv.2308.07870 Salvatori, T., Song, Y., Lukasiewicz, T., Bogacz, R., Xu, Z., 2023. Reverse Differentiation via Predictive Coding. https://doi.org/10.48550/arXiv.2103.04689 Salvatori, T., Song, Y., Yordanov, Y., Millidge, B., Xu, Z., Sha, L., Emde, C., Bogacz, R., Lukasiewicz, T., 2024. A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks. https://doi.org/10.48550/arXiv.2212.00720 Song, Y., Lukasiewicz, T., Xu, Z., Bogacz, R., n.d. Can the Brain Do Backpropagation? — Exact Implementation of Backpropagation in Predictive Coding Networks. Song, Y., Millidge, B., Salvatori, T., Lukasiewicz, T., Xu, Z., Bogacz, R., 2024. Inferring neural activity before plasticity as a foundation for learning beyond backpropagation. Nat Neurosci 27, 348–358. https://doi.org/10.1038/s41593-023-01514-1 Whittington, J.C.R., Bogacz, R., 2019. Theories of Error Back-Propagation in the Brain. Trends in Cognitive Sciences 23, 235–250. https://doi.org/10.1016/j.tics.2018.12.005 Whittington, J.C.R., Bogacz, R., 2017. An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity. Neural Computation 29, 1229–1262. https://doi.org/10.1162/NECO_a_00949 This video was sponsored by Brilliant