Dask and Optuna for Hyper Parameter Optimization

Notebook: https://gist.github.com/mrocklin/683196204d9387d1a30a9d4c660e7be0 How to use Optuna and Dask together to perform parallel hyper-parameter optimization at scale. This notebook uses a toy example, for a more fully-featured example take a look at https://github.com/coiled/dask-xgboost-nyctaxi https://optuna.org https://dask.org XGBoost and Hyperparameter Optimization https://youtu.be/Hh_6X6VFm-E 00:00 Intro 00:35 Optuna Overview 02:52 Optuna with Threads 04:03 Optuna with Dask and Threads 05:32 Caveats 07:09 Optuna and Dask Integration 08:49 Dask within Optuna, sequentially 10:42 Dask within Optuna, in parallel 13:17 Example --- Scale Your Python Workloads with Dask and Coiled. Coiled is a Dask company. With Coiled's rock-solid infrastructure, you can quickly and securely create Dask clusters in your cloud account. Learn more about Coiled and get started for free https://coiled.io/start More content on our blog: https://coiled.io/blog