Principal Component Analysis (PCA) | Dimensionality Reduction Techniques (2/5)

▬▬ Papers / Resources ▬▬▬ Colab Notebook: https://colab.research.google.com/drive/1n_kdyXsA60djl-nTSUxLQTZuKcxkMA83?usp=sharing Peter Bloem PCA Blog: https://peterbloem.nl/blog/pca PCA for DS book: https://pca4ds.github.io/basic.html PCA Book: http://cda.psych.uiuc.edu/statistical_learning_course/Jolliffe%20I.%20Principal%20Component%20Analysis%20(2ed.,%20Springer,%202002)(518s)_MVsa_.pdf Lagrange Multipliers: https://ekamperi.github.io/mathematics/2020/11/01/principal-component-analysis-lagrange-multiplier.html PCA Mathematical derivation #1: https://www.quora.com/Why-does-PCA-choose-covariance-matrix-to-get-the-principal-components-of-features-X PCA Mathematical derivation #2: https://towardsdatascience.com/principal-component-analysis-part-1-the-different-formulations-6508f63a5553 PCA Mathematical derivation #3: https://rich-d-wilkinson.github.io/MATH3030/4.2-pca-a-formal-description-with-proofs.html PCA Mathematical derivation #4: https://stats.stackexchange.com/questions/32174/pca-objective-function-what-is-the-connection-between-maximizing-variance-and-m/136072#136072 PCA Mathematical derivation #5: https://medium.com/@bishikh90/geometrical-and-mathematical-interpretation-principal-component-analysis-52f39a924b40 Eigenvectors and Eigenvalues: https://sebastianraschka.com/Articles/2015_pca_in_3_steps.html Image Sources: - Eigenfaces: https://towardsdatascience.com/eigenfaces-recovering-humans-from-ghosts-17606c328184 - Hyperplane: https://www.analyticsvidhya.com/blog/2021/07/svm-and-pca-tutorial-for-beginners/ ▬▬ Support me if you like 🌟 ►Link to this channel: https://bit.ly/3zEqL1W ►Support me on Patreon: https://bit.ly/2Wed242 ►Buy me a coffee on Ko-Fi: https://bit.ly/3kJYEdl ►E-Mail: [email protected] ▬▬ Used Music ▬▬▬▬▬▬▬▬▬▬▬ Music from #Uppbeat (free for Creators!): https://uppbeat.io/t/sulyya/weather-compass License code: ZRGIWRHMLMZMAHQI ▬▬ Used Icons ▬▬▬▬▬▬▬▬▬▬ All Icons are from flaticon: https://www.flaticon.com/authors/freepik ▬▬ Timestamps ▬▬▬▬▬▬▬▬▬▬▬ 00:00 Introduction 00:26 Used Literature 00:41 Example dataset 02:47 Variance 03:52 Projecting data 04:14 Variance as measure of information 05:15 Scree Plot 05:53 Principal Components 06:14 PCA on images 07:00 Reconstruction based on eigenfaces 07:35 Orthogonal Basis 08:12 Kernel PCA 08:45 Finding principal components 09:28 Distance minimization vs. Variance maximization 10:45 Covariance Matrix 11:35 Correlation vs. Covariance 11:50 Covariance examples 12:50 Linear Algebra Basics 14:22 Eigenvectors and Eigenvalues 15:30 Eigenvector Equation 16:10 Spectral Theorem 16:40 Connection between Eigenvectors and Principal Components 17:23 [STEP 1]: Centering the Data 17:54 [STEP 2]: Calculate Covariance Matrix 18:25 [STEP 3]: Eigenvalue Decomposition 19:05 How to find eigenvectors? 19:17 The truth :O 19:27 Singular value decomposition 20:21 Why eigendecomposition at all? 20:45 [STEP 4]: Projection onto PCs 21:12 Orthogonal Eigenvectors 21:49 Dimensionality Reduction Projection 22:11 [CODE] 24:52 Summary Table ▬▬ My equipment 💻 - Microphone: https://amzn.to/3DVqB8H - Microphone mount: https://amzn.to/3BWUcOJ - Monitors: https://amzn.to/3G2Jjgr - Monitor mount: https://amzn.to/3AWGIAY - Height-adjustable table: https://amzn.to/3aUysXC - Ergonomic chair: https://amzn.to/3phQg7r - PC case: https://amzn.to/3jdlI2Y - GPU: https://amzn.to/3AWyzwy - Keyboard: https://amzn.to/2XskWHP - Bluelight filter glasses: https://amzn.to/3pj0fK2