Rico贾若童的博客

Deep Learning - Overfitting

Bias, Variance, Overfitting, Regularization, Dropout

A Nice Quote 💡 Before we delve in, I’d like to quote from James Dellinger that really hits home: I think the journey we took here showed us that this knee-jerk response of feeling of intimidat...

Posted by Rico's Nerd Cluster on January 16, 2022

Deep Learning - Batch Gradient Descent

Batch Gradient Descent, Mini-Batch

A Neuron And Batch Gradient Descent A Neuron, has multiple inputs and a single output. First it gets the weighted sum of all inputs, then feeds it into an “activation function”. Below, the activat...

Posted by Rico's Nerd Cluster on January 14, 2022

Deep Learning - Activation and Loss Functions

Sigmoid, ReLU, GELU Tanh, Mean Squared Error, Mean Absolute Error, Cross Entropy Loss, Hinge Loss, Huber Loss, IoU Loss, Dice Loss, Focal Loss

Activation Functions Early papers found out that Rectified Linear Unit (ReLu) is always faster than Sigmoid because of its larger derivatives, and non-zero derivatives at positive regions. Howeve...

Posted by Rico's Nerd Cluster on January 8, 2022

Deep Learning - Auto Differentiator From Scratch

Auto Diff Is The Dark Magic Of All Dark Magics Of Deep Learning

Introduction Gradients here refer to scalar to matrix gradient. We need to accumulate gradients for mini-batch training. Elementwise Multiplication gradients: A * B = C del C / del A_ij...

Posted by Rico's Nerd Cluster on January 6, 2022

Deep Learning - Gradient Checking

First Step To Debugging A Neural Net

How To Do Gradient Checking In calculus, we all learned that a derivative is defined as: \[\begin{gather*} f'(\theta) = \frac{f(\theta + \epsilon) - f(\theta)}{\epsilon} \end{gather*}\] ...

Posted by Rico's Nerd Cluster on January 4, 2022

Deep Learning - Introduction

Why Do We Even Need Deep Neuralnets? Data Partition, ML Ops, Data Normalization

Why Do We Need Deep Learning Any bounded continuous function can be approximated by an arbitrarily large single layer. W hy? The idea is roughly that the linear combinations of activation function...

Posted by Rico's Nerd Cluster on January 4, 2022

Computer Vision - Pinhole Camera Model

This Blog Shows How A Small Magic Peek Hole Captures The World

Introduction Camera is intriguing. There have been many different types with different types of lenses (such as fisheye, wide angle Lens). However, the most original (and the simplest) form of cam...

Posted by Rico's Nerd Cluster on July 3, 2021

Computer Vision - Random Tree

Random Tree

Introduction Simple Example: Predicting Lottery Purchase with a Decision Tree Imagine we have two pieces of information about a person: Age Income The task is: we want to predict whether ...

Posted by Rico's Nerd Cluster on April 24, 2021

Computer Vision - Image Downsampling

Bicubic, Bilinear, Nearest Neighbor Interpolation, Ringing Effect

Introduction What is image downsampling?: Image Downsampling is to reduce spatial resolution and make an image smaller. Why performing interpolation during downsampling?: image downsampling essen...

Posted by Rico's Nerd Cluster on January 24, 2021

Computer Vision - ORB Features And Matching

This Blog Shows The Theory and Implementation From Scratch of ORB Features

Intro In SLAM and Computer Vision, distinguishing the same points across multiple camera views is necessary for visual odometry. These “distinguishable points” are called features. In general, we ...

Posted by Rico's Nerd Cluster on January 22, 2021

Rico's Nerd Cluster