Rico贾若童的博客

Deep Learning - Inferencing

Autograd Profiler

Autograd Profiler PyTorch’s Autograd Profiler provides information on the resources (CPU and GPU) for each operation in a model. 1 2 3 4 5 6 import torch.autograd.profiler as profiler with profi...

Posted by Rico's Nerd Cluster on May 20, 2022

Deep Learning - Speedup Tricks

Torch Optimizer Tricks, Mixed Precision Training

General Speed-Up Tricks Data loading tricks: - compared to npz ,npy is faster to load during training because it uses contiguous disk memory and support mmap - use json for boxes with label...

Posted by Rico's Nerd Cluster on May 17, 2022

Deep Learning - Common Oopsies

Underflow, Weight Manipulation

Underflow torch.softmax(X) X is zero due to underflow. Sizing Be careful with the last batch if you want to initialize any tensor that’s specific to each batch’s sizes, because it could b...

Posted by Rico's Nerd Cluster on May 17, 2022

Deep Learning - Strategies Part 2 Training And Tuning

Bias And Variance, And Things To Try For Performance Improvement From My Experience

Orthogononalization Orthogonalization in ML means designing a machine learning system such that different aspects of the model can be adjusted independently. This is like “orthogonal vector” so th...

Posted by Rico's Nerd Cluster on May 17, 2022

Deep Learning - Strategies Part 1 Before Model Training

Error Metrics, Data Preparation Principles, Transfer Learning, Multi-Task Learning

Start Your Development Early, Then Iterate Even after many years in speech recognition, Andrew still had some difficulties bringing up a speech recognition system that’s super robust to noises. So...

Posted by Rico's Nerd Cluster on May 17, 2022

Deep Learning - Data Augmentations

Albumentations

Pre-processing Shuffle Data 1 2 3 4 5 6 7 8 9 shuffled_main_dataset = torch.utils.data.Subset( main_dataset, torch.randperm(dataset_size) ) # train_dataset is a Subset object # main_data...

Posted by Rico's Nerd Cluster on May 14, 2022

Deep Learning - PyTorch Data Loading

RESNET-50 Data Loading, Data Transforms, Custom Data Loading

Dataset and Data Loading Data Set and Data Loading in All-Together In Torch In PyTorch, data is stored in the DataSet object. We can read input data all together, or read them one by one. Then, f...

Posted by Rico's Nerd Cluster on May 11, 2022

Deep Learning - Bert

Masked Autoencoder (MAE) The main idea of the Masked Autoencoder (MAE) is to mask parts of an input (e.g., image features) and train a model to reconstruct the original input by leveraging the rem...

Posted by Rico's Nerd Cluster on April 10, 2022

Deep Learning - Bert

Introduction Bert (BiDirectional Encoder Representation Transformer) is great for tasks like question-answering, NER (Named Entity Recognition), sentence classification, etc. Bert is not a transla...

Posted by Rico's Nerd Cluster on April 10, 2022

Deep Learning - Neural Machine Translation

Hands-On Attention Project

Introduction And Data Preparation The goal of the project is experimenting with date translations, i.e., (“25th of June, 2009”) into machine-readable dates (“2009-06-25”). We need to truncate data...

Posted by Rico's Nerd Cluster on April 7, 2022

Rico's Nerd Cluster

Deep Learning - Inferencing

Autograd Profiler

Deep Learning - Speedup Tricks

Torch Optimizer Tricks, Mixed Precision Training

Deep Learning - Common Oopsies

Underflow, Weight Manipulation

Deep Learning - Strategies Part 2 Training And Tuning

Bias And Variance, And Things To Try For Performance Improvement From My Experience

Deep Learning - Strategies Part 1 Before Model Training

Error Metrics, Data Preparation Principles, Transfer Learning, Multi-Task Learning

Deep Learning - Data Augmentations

Albumentations

Deep Learning - PyTorch Data Loading

RESNET-50 Data Loading, Data Transforms, Custom Data Loading

Deep Learning - Bert

Deep Learning - Bert

Deep Learning - Neural Machine Translation

Hands-On Attention Project

FEATURED TAGS

ABOUT ME

FRIENDS