Contrastive Learning in PyTorch - Part 1: Introduction

▬▬ Notes ▬▬▬▬▬▬▬▬▬▬▬ Two small things I realized when editing this video - SimCLR uses two separate augmented views as positive samples - Many frameworks have separate projection heads on the learned representations which transforms them additionally for the contrastive loss ▬▬ Papers/Sources ▬▬▬▬▬▬▬ - Intro: https://sthalles.github.io/a-few-words-on-representation-learning/ - Survey: https://arxiv.org/ftp/arxiv/papers/2010/2010.05113.pdf - Supervised Contrastive Learning: https://arxiv.org/abs/2004.11362 - Contrastive Loss:https://medium.com/@maksym.bekuzarov/losses-explained-contrastive-loss-f8f57fe32246 - Triplet Loss: https://towardsdatascience.com/triplet-loss-advanced-intro-49a07b7d8905 - NT-Xent Loss: https://medium.datadriveninvestor.com/simclr-part-2-the-encoder-projection-head-and-loss-function-809a64f30d4a - SimCLR ▬▬ Used Icons ▬▬▬▬▬▬▬▬▬▬ All Icons are from flaticon: https://www.flaticon.com/authors/freepik ▬▬ Used Music ▬▬▬▬▬▬▬▬▬▬▬ Music from Uppbeat (free for Creators!): https://uppbeat.io/t/t-check/lemon-limes License code: KJ7PFP0HB9BWHJOF ▬▬ Used Images ▬▬▬▬▬▬▬▬▬▬▬ All Images are from pixabay.com (Cats, Dogs, ...) and royalty-free. ▬▬ Timestamps ▬▬▬▬▬▬▬▬▬▬▬ 00:00 Introduction 00:22 Overview 01:35 Supervised vs. Self-Supervised CL 03:25 Applications 04:24 Popular Papers 06:17 Metric Learning 07:21 Loss 1 09:39 Loss 2 10:54 Loss 3 13:22 Variations between Losses 13:42 Part 2 Outlook ▬▬ Support me if you like 🌟 ►Website: https://deepfindr.github.io/ ►Support me on Patreon: https://bit.ly/2Wed242 ►Buy me a coffee on Ko-Fi: https://bit.ly/3kJYEdl ►Coursera: https://imp.i384100.net/b31QyP ►Link to this channel: https://bit.ly/3zEqL1W ►E-Mail: [email protected] ▬▬ My equipment 💻 - Microphone: https://amzn.to/3DVqB8H - Microphone mount: https://amzn.to/3BWUcOJ - Monitors: https://amzn.to/3G2Jjgr - Monitor mount: https://amzn.to/3AWGIAY - Height-adjustable table: https://amzn.to/3aUysXC - Ergonomic chair: https://amzn.to/3phQg7r - PC case: https://amzn.to/3jdlI2Y - GPU: https://amzn.to/3AWyzwy - Keyboard: https://amzn.to/2XskWHP - Bluelight filter glasses: https://amzn.to/3pj0fK2