The NEW way to ingest data with Airflow in a single task!

The NEW way to ingest data with Airflow in a single task!

26.219 Lượt nghe
The NEW way to ingest data with Airflow in a single task!
The NEW way to ingest data with Airflow in a single task! Get started with Airbyte with the dedicated Udemy course 👇 (FREE until June 3th) https://www.udemy.com/course/the-complete-hands-on-introduction-to-airbyte/?couponCode=YOUTUBE_1 Or use the coupon YOUTUBE_1 Data ingestion is tricky, but it doesn't have to be! Stop wasting time finding the right operator to transfer data. Rule them all in a single task with Airflow and PyAirbyte! In this video, you will: ✅ Transfer data from S3 to BigQuery efficiently with PyAirbyte ✅ Create Python Virtual Environments to save running time ✅ Use the ExternalPythonOperator to avoid dependency conflicts ✅ Check data in BigQuery with the BigQueryHook And more! No need to juggle between different operators or use custom scripts anymore! 🤖 The code: https://robust-dinosaur-2ef.notion.site/The-new-way-to-ingest-data-with-Airflow-PUBLIC-bfe8f1359e4444279fac1a6b048c4edd?pvs=4 🏆 BECOME A PRO: https://www.udemy.com/course/the-complete-hands-on-course-to-master-apache-airflow/?couponCode=BIGPROMO1 👍 Smash the like button to become an Airflow Super Hero! ❤️ Subscribe to my channel to become a master of Airflow