Rico贾若童的博客

Deep Learning - RNN Part 3 LSTM, Bi-Directional RNN, Deep RNN

LSTM LSTM came out in 1997 and GRU is a simplification of it. In LSTM, we have the “forget gate”, $\Gamma_r$, the output gate $\Gamma_o$, and the update gate $\Gamma_u$. We do NOT have $\Gamma_r$ ...

Posted by Rico's Nerd Cluster on March 15, 2022

Deep Learning - RNN Part 2 GRU

Vanishing Gradients of RNN, GRU

The Vanishing Gradient Problem of RNN RNN can doesn’t handle long range dependencies well. One example is in speech recognition, “The cat which ate, slept, played and had a good day … , was full” ...

Posted by Rico's Nerd Cluster on March 12, 2022

Deep Learning - RNN

Sequence Models, RNN Architectures

Sequence Models Some common sequence models include: DNA sequencing, audio clips, sentiment classification, etc. Another example is name indexing, where names in news for a past period of time wil...

Posted by Rico's Nerd Cluster on March 9, 2022

Deep Learning - PyTorch Model Training

Checkpointing, Op Determinisim, 🤗 HuggingFace Trainer

Checkpointing Checkpointing is a technique to trade compute for memory during training. Instead of storing all intermediate activations (outputs layers) for backprop, which consumes a lot of memor...

Posted by Rico's Nerd Cluster on March 6, 2022

Deep Learning - Ensemble

Ensemble

Ensemble An ensemble is a group of models (a.k.a base learners, weak learners) that are trained and combined to have better prediction, increased stability, and improved generalization compared to...

Posted by Rico's Nerd Cluster on March 3, 2022

Deep Learning - Neural Style Transfer

What Conv Net Is Learning

What Do Conv Nets Learn? For the ease of explanation, below I will use an example, where a conv layer has 3 input channels, and 5 output channels. As a recap: this layer has 3 filters, each filter...

Posted by Rico's Nerd Cluster on March 1, 2022

Deep Learning - Face Recognition

Siamese Network, Deep Face

Introduction Face verification (easier) vs face recognition (harder) Face verfifaction takes input image, name and an ID. Then, it ouptuts if the image corresponds to the ID. Face recogniti...

Posted by Rico's Nerd Cluster on February 25, 2022

Deep Learning - Face Recognition Prelude

2D Frontalization, 3D Face Alignment

Introduction DeepFace introduced a 3D alignment step that projects 2D face images into a frontal view [1]. This is called “frontalization”. A very cool 2D->3D problem. A frontal view is a view ...

Posted by Rico's Nerd Cluster on February 23, 2022

Deep Learning - Hands-On UNet Image Segmentation From Scratch

UNet

This article is inspired by this referece The source code of this project can be found here Data Loading When making a dataset, do NOT use jpeg and stick to png instead. Jpeg will compress the d...

Posted by Rico's Nerd Cluster on February 19, 2022

Deep Learning - Hands-On YOLO V1 Transfer Learning

YOLO V1 Theory & Transfer Learning

You Only Look Once (YOLO) V1 The main idea is to divide an image into a 7x7 grid. Each grid will detect the existence of 2 bounding box whose center is within the grid cell and outputs [p_1, bx_1,...