Spark Tutorial in Microsoft Fabric (3.5 HOURS!)

10+ hours of FREE Fabric Training: https://www.skool.com/microsoft-fabric/classroom/d154aad4?md=3b108b0e216c46c88d891407ccd8647b Looking for Fabric consultancy? Fill in this form: https://forms.office.com/e/h5FaVBwQ7s In September, I recorded a series of video to help you Learn Apache Spark in Microsoft Fabric. I merged all videos in the series into this 3.5 hour Spark tutorial, so that it’s easier for you to watch and share will friends and colleagues who might also be interested in learning Spark in Fabric. New to Microsoft Fabric? Watch this: https://youtu.be/yRJ03n1U5-E Spark is the engine behind both the Data Engineering AND the Data Science experiences in Microsoft Fabric, so in this 3.5 HOUR tutorial, I walk you through Apache Spark: what it is, why you should learn it, how to use it, and how it integrates into Microsoft Fabric. Link to the GitHub for this series, including all links to reference material mentioned through this tutorial: https://github.com/LearnMicrosoftFabric/YouTube/tree/main/learn_spark_in_fabric No previous Spark knowledge is required, some basic Python would be useful! #pyspark #microsoftfabric #apachespark 00:00:00 Welcome 00:03:58 Why Spark? 00:13:47 Components of Spark 00:17:58 Spark DataFrame 00:23:54 Read Files into DataFrame 00:36:32 Read/Write to Lakehouse Table 00:43:07 Basic DataFrame Operations 00:52:20 DataFrame Filtering 01:03:43 GroupBy and Aggregate Functions 01:12:21 Handling missing values 01:24:44 Joining DataFrames 01:31:13 Time-series data 01:37:52 Spark SQL 01:46:03 Spark Machine Learning 02:07:30 Configuring Spark 02:24:19 Autotuning Spark Configuration 02:30:37 Library Management 02:35:30 High-concurrency mode 02:41:28 Spark Scala 02:47:55 Fabric MSSparkUtils 02:57:23 Monitoring Spark 03:03:45 Questions and Answers --BROWSE MY OTHER FABRIC PLAYLISTS-- DATA ENGINEERING https://www.youtube.com/playlist?list=PLug2zSFKZmV1NvKfnRzG9e3Fl-8QLD5MK END-TO-END FABRIC PROJECT https://www.youtube.com/playlist?list=PLug2zSFKZmV1BHk129X_2xi53ZRwQaTkb INTRO TO MICROSOFT FABRIC https://www.youtube.com/playlist?list=PLug2zSFKZmV0Yaya7NxRQfrrPtfF2vj0K DATA FACTORY https://www.youtube.com/playlist?list=PLug2zSFKZmV3FkUFDxlyrfSJMQ3CeqyEF --LINKEDIN-- Not following the LinkedIn page yet? Here's the link: https://www.linkedin.com/company/learnmicrosoftfabric/ --ABOUT WILL-- Hi, I'm Will! I'm hugely passionate about data and using it to create a better world. I currently work as a Consultant, focusing on Data Strategy, Data Engineering and Business Intelligence (within the Microsoft/Azure/Fabric environment). I have previously worked as a Data Scientist. I started Learn Microsoft Fabric to share my learnings on how Microsoft Fabric works and help you build your career and build meaningful things in Fabric. --SUBSCRIBE-- Not subscribed yet? You should! There are lots of new videos in the pipeline covering all aspects of Microsoft Fabric. https://youtube.com/@LearnMicrosoftFabric?sub_confirmation=1