In this video, we are going to go through the history of CNNs specifically for Image Classification tasks – starting from those early research years, to the golden era of the mid 2010s when many of the most genius Deep Learning architectures ever were conceived, and finally discuss the latest trends in CNN research now as they compete with attention and vision-transformers.
Members can now access the full write-up, all the animations, powerpoint slides, and more side-content from this video!
Visit the Patreon link to see what is available:
https://www.patreon.com/NeuralBreakdownwithAVB
Medium article: https://medium.com/@neural.avb/the-history-of-convolutional-neural-networks-for-image-classification-1989-today-5ea8a5c5fe20
Buy me a coffee at https://ko-fi.com/neuralavb !
#ai #machinelearning #deeplearning
Checkout the entire NLP History video here:
https://youtu.be/uocYQH0cWTs
Related videos:
Attention Series:
https://www.youtube.com/watch?v=frosrL1CEhw&list=PLGXWtN1HUjPfK_n9j5tPZ_a6Rx3yceZ_B
Latent Space:
https://youtu.be/FslFZx08beM
CNNs:
https://youtu.be/kebSR2Ph7zg
Paper links:
CNN with Backprop (1989): http://yann.lecun.com/exdb/publis/pdf/lecun-89e.pdf
LeNet-5: http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf
AlexNet: https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
GoogleNet: https://arxiv.org/abs/1409.4842
VGG: https://arxiv.org/abs/1409.1556
Batch Norm: https://arxiv.org/pdf/1502.03167
ResNet: https://arxiv.org/abs/1512.03385
DenseNet: https://arxiv.org/abs/1608.06993
MobileNet: https://arxiv.org/abs/1704.04861
MobileNet-V2: https://arxiv.org/abs/1801.04381
Vision Transformers: https://arxiv.org/abs/2010.11929
ConvNext: https://arxiv.org/abs/2201.03545
Other interesting papers not covered in the video:
EfficientNet: https://arxiv.org/abs/1905.11946
Squeeze-and-Excitation Network: https://arxiv.org/abs/1709.01507
Swin Transformers: https://arxiv.org/abs/2103.14030
Timestamps:
0:00 - Intro
1:17 - Visualizing CNNs
3:12 - 1989
5:00 - 1998 - LeNet 5
6:07 - The 2000s
7:37 - 2012 - AlexNet
9:45 - 2014 - GoogLeNet and Inception Module
11:49 - 2014 - VGG
12:30 - 2015 - Batch Normalization
13:04 - 2015 - Residual Network
14:46 - 2016 - DenseNet
15:22 - 2017 - MobileNet
16:19 - 2018 - MobileNet V2
17:31 - 2020 - Vision Transformer
19:15 - 2022 - ConvNext
20:50 - Outro