Microsoft Fabric: Inspecting 28 MILLION row dataset in Bronze Lakehouse - Part 2

Microsoft Fabric End to End Demo - Part 2 - Planning and Architecting a Data Project. For a data platform, we need some #data! In this series we're going to be using Land Registry data provided by the UK government which registers the ownership of land and property in England and Wales. The dataset is almost 5GB in size, and provides different types of files for complete or incremental processing. This will allow us to benefit from UPSERT-like functionality enabled by #DeltaLake without having to load all the data every time we receive new information. In this video we'll take a quick look at the data, where it comes from and what format it's in, and we'll also frame up the insights we're aiming to achieve from the analysis taking place in this series. We'll finish by stepping through a sample architecture diagram - a powerful way to visualize involved data platforms at a high-level. 00:00 Introduction 00:17 Sample data introduction 00:58 Sample data inspection 03:54 Insight Discovery and defining goals 06:46 Fabric architecture walkthrough 11:32 Outro Useful links: 📖 UK Land Registry data: https://www.gov.uk/government/statistical-data-sets/price-paid-data-downloads Series contents: 📺 Part 1 - Lakehouse & Medallion Architecture - https://www.youtube.com/watch?v=x_CvCwSbRZI&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=1&ab_channel=endjin 📺 Part 2 - Planning and Architecting a Data Project - https://www.youtube.com/watch?v=miDa4FZY7GU&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=2&ab_channel=endjin 📺 Part 3 - Ingest Data - https://www.youtube.com/watch?v=yptYlWLsVYk&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=3&ab_channel=endjin 📺 Part 4 - Creating a shortcut to ADLS Gen2 in Fabric - https://www.youtube.com/watch?v=11urrDAW-bU&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=4&ab_channel=endjin 📺 Part 5 - Navigating OneLake data locally - https://youtu.be/DO779ZcAtTk 📺 Part 6 - Role of the Silver layer in the Medallion Architecture - https://www.youtube.com/watch?v=pCTl-nqDT_8&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=6&ab_channel=endjin 📺 Part 7 - Processing Bronze to Silver - https://www.youtube.com/watch?v=s_mHaLBlA94&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=7&ab_channel=endjin If you want to learn more about Fabric, take a look at some of our other content: 📺 Part 8 - Good Notebook Development Practices - https://www.youtube.com/watch?v=UyS6ZUgh-Wc&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=7&ab_channel=endjin 🤖 [Course] Microsoft Fabric: from Descriptive to Predictive Analytics: https://www.youtube.com/watch?v=uaRePHeqvQU&list=PLJt9xcgQpM63xH0x9K-oQMFiE4D9pdl4W 👉 Perspectives on #MicrosoftFabric: https://endjin.com/what-we-think/talks/perspectives-on-microsoft-fabric 👉 A Tour Around #MicrosoftFabric: https://endjin.com/what-we-think/talks/a-tour-around-microsoft-fabric 👉 Introduction to #MicrosoftFabric: https://endjin.com/blog/2023/05/intro-to-microsoft-fabric and find all the rest of our content here: https://endjin.com/blog/2023/05/microsoft-fabric-announced #Microsoft #PowerBI #MicrosoftFabric #lakehouse #datalake #onelake #data #ai #analytics #medallion #bronze #silver #gold #datafactory #projectplanning