Many people use Dask and Coiled for geospatial analysis with Xarray. This example goes through two short notebooks that process a small dataset, and then a larger dataset (6TB) in a couple of minutes. We show how Xarray works with Dask to parallelize analyses and how Coiled scales up both to process large volumes of data on the cloud.
For further engagement we recommend people check out the Pangeo Discourse: https://discourse.pangeo.io/
Notebooks here:
- https://github.com/coiled/examples/blob/main/geospatial.ipynb
- https://github.com/coiled/examples/blob/main/geospatial-large.ipynb
conda install -c conda-forge xarray dask coiled s3fs geogif
Libraries and things to check out:
Xarray: https://docs.xarray.dev/en/stable/
Geopandas: https://geopandas.org/en/stable/
Pangeo: https://pangeo.io/
Dask Infrastructure with Coiled for Pangeo:
https://www.youtube.com/watch?v=DpA-pfBZfCk
For more info on getting started with Coiled for free:
https://coiled.io/start
Key Moments
00:00 Intro
01:50 Larger Example
03:33 Computation Starts and Scaling
06:00 Inspect results
06:50 Summary
---
Scale Your Python Workloads with Dask and Coiled.
Coiled is a Dask company. With Coiled's rock-solid infrastructure, you can quickly and securely create Dask clusters in your cloud account.
Learn more about Coiled and get started for free
https://coiled.io/start
More content on our blog:
https://coiled.io/blog