Geometric Clifford Algebra Networks and Clifford Neural Layers for PDE Modeling

Join the Learning on Graphs and Geometry Reading Group: https://hannes-stark.com/logag-reading-group Paper: "Geometric Clifford Algebra Networks" https://arxiv.org/abs/2302.06594 Abstract: We propose Geometric Clifford Algebra Networks (GCANs) that are based on symmetry group transformations using geometric (Clifford) algebras. GCANs are particularly well-suited for representing and manipulating geometric transformations, often found in dynamical systems. We first review the quintessence of modern (plane-based) geometric algebra, which builds on isometries encoded as elements of the Pin(p,q,r) group. We then propose the concept of group action layers, which linearly combine object transformations using pre-specified group actions. Together with a new activation and normalization scheme, these layers serve as adjustable geometric templates that can be refined via gradient descent. Theoretical advantages are strongly reflected in the modeling of three-dimensional rigid body transformations as well as large-scale fluid dynamics simulations, showing significantly improved performance over traditional methods. Authors: David Ruhe, Jayesh K. Gupta, Steven de Keninck, Max Welling, Johannes Brandstetter Paper: "Clifford Neural Layers for PDE Modeling" https://arxiv.org/abs/2209.04934 Abstract: Partial differential equations (PDEs) see widespread use in sciences and engineering to describe simulation of physical processes as scalar and vector fields interacting and coevolving over time. Due to the computationally expensive nature of their standard solution methods, neural PDE surrogates have become an active research topic to accelerate these simulations. However, current methods do not explicitly take into account the relationship between different fields and their internal components, which are often correlated. Viewing the time evolution of such correlated fields through the lens of multivector fields allows us to overcome these limitations. Multivector fields consist of scalar, vector, as well as higher-order components, such as bivectors and trivectors. Their algebraic properties, such as multiplication, addition and other arithmetic operations can be described by Clifford algebras. To our knowledge, this paper presents the first usage of such multivector representations together with Clifford convolutions and Clifford Fourier transforms in the context of deep learning. The resulting Clifford neural layers are universally applicable and will find direct use in the areas of fluid dynamics, weather forecasting, and the modeling of physical systems in general. We empirically evaluate the benefit of Clifford neural layers by replacing convolution and Fourier operations in common neural PDE surrogates by their Clifford counterparts on two-dimensional Navier-Stokes and weather modeling tasks, as well as three-dimensional Maxwell equations. Clifford neural layers consistently improve generalization capabilities of the tested neural PDE surrogates. Authors: Johannes Brandstetter, Rianne van den Berg, Max Welling, Jayesh K. Gupta Speakers: Johannes Brandstetter - https://twitter.com/jo_brandstetter Twitter Hannes: https://twitter.com/HannesStaerk Twitter Dominique: https://twitter.com/dom_beaini Twitter Valence Discovery: https://twitter.com/valence_ai Reading Group Slack: https://join.slack.com/t/logag/shared_invite/zt-u0mbo1ec-zElmvd1oSCXGjXvxLSokvg ~ Chapters 00:00 - Intro 03:35 - Why Clifford Algebras for Deep Learning 12:26 - Introduction to Clifford Algebras 21:23 - Clifford Convolution 24:44 - Clifford Fourier Transform & Results 39:52 - Follow Up Work 41:29 - Geometric Clifford Algebra Networks 44:16 - The Pin(n) Group and Transformations 54:39 - The Overall Picture 57:34 - Group Action Layers 01:10:16 - Fluid Dynamics 01:14:16 - Q+A