In this video, we explore different ways to import large CSV files in Laravel, benchmarking each approach and discussing their pros and cons. From simple collection-based imports to high-performance concurrent processing - let's try it out together.
📜 Repository & Additional Improvements:
https://github.com/christophrumpel/laravel-import-million-rows
🔢 Tools in this video:
- Laravel 11
- MySQL 8.3
🛠️ We weren't using any external tools to help us, but there are few to check out:
- CSV League https://csv.thephpleague.com
- Spatie Simple Excel https://github.com/spatie/simple-excel
- Flow PHP https://flow-php.com
⏳ Timeline:
00:00 Introduction
00:45 Basic Collection Insert
08:50 The Basic Collection with Big Insert
10:40 Improving with Chunk Processing
13:06 Exploring Lazy Collections
14:55 Streaming Files with Chunks
19:40 Single Row PDO Statements
23:13 Optimized PDO Chunk Processing
29:15 Parallel Processing with Concurrency
41:15 MySQL Native Import Solution
46:50 Recommendations & Wrap-up