Many Model Forecasting on Databricks

Welcome to our in-depth discussion on the Many Models for Forecasting (MMF) framework, a powerful open-source solution for large-scale time series forecasting, now available on Databricks! Join Maria, Lara, and Ryuta as they explore MMF’s capabilities to handle thousands to millions of time series using over 40 diverse models. Discover how to integrate local, global, and foundation time series models and apply advanced techniques like hyperparameter tuning, multivariate forecasting, and automatic model selection. Key Moments: - [00:00] Introduction to Many Models for Forecasting (MMF) - [00:27] Overview of MMF: What it is and why it was created - [02:50] Project Origins and Team Involvement - [03:02] Explanation of the 40+ Forecasting Models Supported - [04:45] Introduction to Foundation Time Series Models - [05:46] Comparison Between MMF and AutoML - [07:30] Starting with the Code: Overview of GitHub Repo and Setup - [09:20] Exploring Model Types: Local, Global, and Foundation Models - [12:19] Understanding Compute Requirements: Clusters, CPUs, and GPUs - [13:20] How to Choose Between Local, Global, and Foundation Models - [15:03] Best Practices for Data Preparation - [17:13] Univariate vs. Multivariate Time Series Forecasting - [17:55] Integrating External Regressors for Multivariate Forecasting - [18:56] Configuration Options for Training and Tuning - [21:06] Automated Validation and Model Evaluation with Delta Tables - [26:21] Custom Metrics and Flexible Evaluation Strategies - [29:18] Scaling Your Forecasting with Clusters and Compute Resources - [32:21] Efficient Hyperparameter Tuning and Model Configuration - [34:16] Time Series Foundation Models and Fine-Grained Model Selection - [37:08] On-the-Fly Forecast Serving and Anomaly Detection Stay tuned to learn more about MMF's novel approach to time series forecasting, including its ability to handle both local and global models, fine-tune foundation models, and automatically log and register models for reproducibility. We also discuss how to effectively scale forecasting jobs using Databricks clusters and integrate external data for more accurate predictions. Don't forget to like, subscribe, and click the notification bell for more updates! Explore the MMF framework in detail by visiting our GitHub repository: https://github.com/databricks-industry-solutions/many-model-forecasting