Getting Started with Prefect | Task Orchestration & Data Workflows

FREE Modern Data Checklist to give you clarity → https://bit.ly/kds-checklist Project-based training to help you level-up → https://bit.ly/simple-stack Consulting to help you implement → https://bit.ly/kds-consulting A big challenge for many data teams is orchestrating all of the tools within their data stack. Fortunately, there are tools designed to address this exact issue (task orchestration) and make our lives as data engineers easier. So in today's video, I want to show you how to use a great open-source task orchestration tool called Prefect that you can use to orchestrate and monitor your entire stack from a single location. It has a great UI, is designed with data tools in mind and only requires basic Python knowledge to get started. By the end of this video you'll understand what Prefect is all about and how you can start using it in your own data stack. Thank you for watching! Timestamps: 00:00 - Intro 00:44 - What is Prefect? 02:32 - Install Prefect 04:48 - Create a Python Script 06:44 - Add a flow 07:42 - Add a task 08:45 - Add a subflow 09:49 - Intro to Deployments 12:03 - Create a Deployment 15:17 - Start an Agent 16:48 - Use Prefect Cloud 20:25 - Using Blocks 23:09 - Version Control & Storage w/ GitHub 26:07 - Automations & Task Concurrency Title & Tags: Getting Started with Prefect | Task Orchestration & Data Workflows #kahandatasolutions #dataengineering #prefect