Rico's Nerd Cluster

「离开世界之前 一切都是过程」

Deep Learning - Hands-On ResNet Transfer Learning For CIFAR-10 Dataset

Data Normalization, Conv Net Training

ResNet-50 Transfer Learning COMPLETE CODE can be found here Data Loading Please see this blogpost for data loading Model Definition PyTorch Built-In Model 1 2 3 4 model = models.resnet50(weig...

Deep Learning - CNN Applications

TensorFlow Keras Sequential and Functional Models

TF Keras Sequential API TF Keras Sequential API can be used to build simple feedforward, sequential models. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 ...

Deep Learning - Classic CNN Models

LeNet-5, AlexNet, VGG-16, ResNet-50, One-by-One Convolution, Inception Network

LeNet-5 (65K params) The LeNet-5 architecture (LeCun, 1998) and is still very widely used. 5 is simply the version number. LeNet-5 has 7 layers. Input layer are 32x32 grayscale images. MNIST i...

Deep Learning - CNN Basics

Filters, Padding, Convolution and Its Back Propagation, Receptive Field

Filters Filters (aka kernels): “Pattern Detectors”. Each filter is a small matrix, which you can drag along an image and multiply pixel values with (convolution). They can detect edges, corners, a...

Deep Learning - Start Easy, Things I Learned From Training Small Neural Nets

Basic Torch Network With Some Notes on Syntax

Introduction To gain some insights into how hyper parameters impacts training, I created a simple neural network using PyTorch to learn 2D input data. Specifically, I’m interested in exploring the...

Deep Learning - PyTorch Basics

Neural Network Model Components, Common Operations

Data Type Conversions Common Data Types torch.arange(start, stop, step) can take either float or int values torch.range(start, stop, step) is deprecated because its signature is dif...

Deep Learning - TensorFlow Basics

Nothing Fancy, Just A Basic TF Network

Basic Operations Max Operations Immutable (tf.constant) vs Variable (tf.Variable), notice the different capitalization: tf.math.reduce_max(): find the max along certain dimension(s). 1 2 3...

Deep Learning - Softmax And Cross Entropy Loss

Softmax, Cross Entropy Loss, and MLE

Softmax When we build a classifier for cat classification, at the end of training, it’s necessary to find the most likely classes for given inputs. The raw unnomalized ...

Deep Learning - Hyper Parameter Tuning

It finally comes down to how much compute we have, actually...

How To Sample For Single Parameter Tuning Generally, we need to try different sets of parameters to find the best performing one. In terms of number layers, it could be a linear search: Defin...

Deep Learning - Layer Normalization

Normalization For Sequential Data

Layer Normalization Batch normalization has two main constraints: When batch size become smaller, it performs bad? Nowadays, we tend to have higher data resolution, especially in large NLP tra...