Microsoft Fabric: Ingesting 5GB into a Bronze Lakehouse using Data Factory - Part 3

Microsoft Fabric End to End Demo - Part 3 - Ingesting 5GB into a Bronze Lakehouse using Data Factory. In this video we'll see how we can quickly ingest ~5GB of data from an unauthenticated HTTP data source into #OneLake using #DataFactory in #MicrosoftFabric. We'll see the distinction between Tables and Files in a Fabric #Lakehouse and look at how we can preview data in the Lakehouse explorer. 00:00 Intro 00:15 Dataset recap 01:25 Workspace and pipeline artifacts 01:57 Pipeline UI layout 02:21 Copy data activity options 03:07 Configure copy data activity source 05:00 Configure copy data activity destination 06:21 Add dynamic content for destination Lakehouse filepath 08:21 Copy Data activity additional settings 09:00 Manually trigger pipeline 09:21 Alternative parameterized pipeline 11:38 Reviewing pipeline run details 12:10 Default workspace artifacts 13:04 Viewing Lakehouse Files 13:46 Roundup and outro Useful links: 📂 Price Paid Data (PPD) in text or CSV format: https://www.gov.uk/government/statistical-data-sets/price-paid-data-downloads 📖 Data Factory in Microsoft Fabric: https://learn.microsoft.com/en-us/fabric/data-factory/ 📖 Copy Data activity in Microsoft Fabric: https://learn.microsoft.com/en-us/fabric/data-factory/copy-data-activity 📖 Lakehouse in Microsoft Fabric: https://learn.microsoft.com/en-us/fabric/data-engineering/lakehouse-overview Series contents: 📺 Part 1 - Lakehouse & Medallion Architecture - https://www.youtube.com/watch?v=x_CvCwSbRZI&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=1&ab_channel=endjin 📺 Part 2 - Planning and Architecting a Data Project - https://www.youtube.com/watch?v=miDa4FZY7GU&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=2&ab_channel=endjin 📺 Part 3 - Ingest Data - https://www.youtube.com/watch?v=yptYlWLsVYk&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=3&ab_channel=endjin 📺 Part 4 - Creating a shortcut to ADLS Gen2 in Fabric - https://www.youtube.com/watch?v=11urrDAW-bU&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=4&ab_channel=endjin 📺 Part 5 - Navigating OneLake data locally - https://youtu.be/DO779ZcAtTk 📺 Part 6 - Role of the Silver layer in the Medallion Architecture - https://www.youtube.com/watch?v=pCTl-nqDT_8&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=6&ab_channel=endjin 📺 Part 7 - Processing Bronze to Silver - https://www.youtube.com/watch?v=s_mHaLBlA94&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=7&ab_channel=endjin If you want to learn more about Fabric, take a look at some of our other content: 📺 Part 8 - Good Notebook Development Practices - https://www.youtube.com/watch?v=UyS6ZUgh-Wc&list=PLJt9xcgQpM61fxyB1aWzWCAEsHZHEZD6w&index=7&ab_channel=endjin 🤖 [Course] Microsoft Fabric: from Descriptive to Predictive Analytics: https://www.youtube.com/watch?v=uaRePHeqvQU&list=PLJt9xcgQpM63xH0x9K-oQMFiE4D9pdl4W 👉 Perspectives on #MicrosoftFabric: https://endjin.com/what-we-think/talks/perspectives-on-microsoft-fabric 👉 A Tour Around #MicrosoftFabric: https://endjin.com/what-we-think/talks/a-tour-around-microsoft-fabric 👉 Introduction to #MicrosoftFabric: https://endjin.com/blog/2023/05/intro-to-microsoft-fabric and find all the rest of our content here: https://endjin.com/blog/2023/05/microsoft-fabric-announced #Microsoft #PowerBI #MicrosoftFabric #lakehouse #datalake #onelake #data #datafactory #datapipline #ai #analytics #medallion #bronze #silver #gold #datafactory #projectplanning #hmrc #dataingestion