Lecture 23 | Descent, Backtracking & Unconstrained Minimization | Convex Optimization by Ahmad Bazzi

☕️ Buy me a coffee: https://paypal.me/donationlink240 🙏🏻 Support me on Patreon: https://www.patreon.com/c/ahmadbazzi In Lecture 23 of this course on Convex Optimization, we focus on algorithms that solve unconstrained minimization type problems. The lecture evolves around unconstrained minimization problems that might or might not enjoy closed form solutions. Descent methods are discussed along with exact line search and backtracking. MATLAB implementations are given along the way. This lecture is outlined as follows: 00:00:00 Introduction 00:01:06 Unconstrained Minimization 00:01:36 Iterative Algorithm Assumptions 00:04:28 Gradient Equivalence 00:09:04 Unconstrained Least Squares 00:20:13 Unconstrained Geometric Program 00:28:10 Initial Subset Assumption 00:35:16 Intuitive Solution of Logarithmic Barrier Minimization 00:40:42 Generalization of Logarithmic Barriers 00:42:57 Descent Methods 00:50:42 Gradient Descent 00:52:59 Exact Line Search 00:56:23 Backtracking 01:00:25 MATLAB: Gradient Descent with Exact Line Search 01:17:35 MATLAB: Gradient Descent with Backtracking 01:20:12 MATLAB: Gradient Descent with Explicit Step Size Update 01:28:07 Summary 01:30:59 Outro --------------------------------------------------------------------------------------------------------- Lecture 1 | Introduction to Convex Optimization: https://youtu.be/SHJuGASZwlE Lecture 2 | Convex Sets: https://youtu.be/QV5qtTq1Tro Lecture 3 | Convex Functions: https://youtu.be/XzW-B1CJ_ao Lecture 4 | Convex Optimization Principles : https://youtu.be/aMC3WPGMLes Lecture 5 | Linear Programming & SIMPLEX algorithm w MATLAB: https://youtu.be/dAyeNmz6p-c Lecture 6 | Quadratic Programs: https://youtu.be/kM52hSvBY4k Lecture 7 | Quadratically Constrained Quadratic Programs: https://youtu.be/SP2-7AG2wfk Lecture 8 | Second Order Cone Programming: https://youtu.be/sVbcJx4g-LQ Lecture 9 | Geometric Programs: https://youtu.be/PoLKyYsmVD0 Lecture 10 | Generalized Geometric Programs: https://youtu.be/Lio9kxMYbRk Lecture 11 | SemiDefinite Programming https://youtu.be/XwyE8vIszo4 Lecture 12 | Vector and Multicriterion Optimization | Pareto Optimal points and the Pareto Frontier https://youtu.be/iUAxxsz1Axw Lecture 13 | Optimal Trade-off Analysis https://youtu.be/YtGFNsZ0RVc Lecture 14 | Lagrange Dual Function https://youtu.be/Qneah_lyQ0o Lecture 15 | Lagrange Dual Problem https://youtu.be/0WpYucMfaHM Lecture 16 | Certificate of Suboptimality https://youtu.be/_muH67CqTME Lecture 17 | Complementary Slackness https://youtu.be/tFWjzEQMq2g Lecture 18 | KKT Conditions https://youtu.be/JFbC2uoN7e4 Lecture 19 | Perturbation and Sensitivity Analysis https://youtu.be/COQtsv1SohQ Lecture 20 | Equivalent Reformulations https://youtu.be/EjtIlfa5DSM Lecture 21 | Weak Alternatives https://youtu.be/CRAaGh2wVZ8 Lecture 22 | Strong Alternatives https://youtu.be/sB11BV1gyXc --------------------------------------------------------------------------------------------------------- References: [1] Boyd, Stephen, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004. [2] Nesterov, Yurii. Introductory lectures on convex optimization: A basic course. Vol. 87. Springer Science & Business Media, 2013. Reference no. 3: [3] Ben-Tal, Ahron, and Arkadi Nemirovski. Lectures on modern convex optimization: analysis, algorithms, and engineering applications. Vol. 2. Siam, 2001. --------------------------------------------------------------------------------------------------------- Instructor: Dr. Ahmad Bazzi IG: https://www.instagram.com/drahmadbazzi/ FB: https://www.facebook.com/profile.php?... RG: https://www.researchgate.net/profile/... MSE: https://math.stackexchange.com/users/... YT: https://www.youtube.com/c/AhmadBazzi --------------------------------------------------------------------------------------------------------- Credits : Microsoft OneNote: https://products.office.com/en-gb/one... #ConvexOptimization #UnconstrainedMinimization #Algorithms