Rico's Nerd Cluster

「离开世界之前 一切都是过程」

Deep Learning - Loss Functions

Mean Squared Error, Mean Absolute Error, Hinge Loss, Huber Loss, L1 Loss, Cross Entropy Loss, NLL Loss, Sparse Entropy, IoU Loss, Dice Loss, Focal Loss, Cauchy Robust Kernel

Regression Losses Mean Squared Error (MSE) \[\text{MSE} = \frac{1}{n}\sum_i (y_i - \hat{y}_i)^2\] Disadvantages: Sensitive to outliers because errors are squared. Assumes Gau...

Deep Learning - Activation

Sigmoid, ReLU, GELU Tanh

Activation Functions Early work observed that the Rectified Linear Unit (ReLU) often trains faster than sigmoid-like activations because it avoids saturation for positive inputs and has a simple g...

Deep Learning - Auto Differentiator From Scratch

Auto Diff Is The Dark Magic Of All Dark Magics Of Deep Learning

Introduction Gradients here refer to scalar to matrix gradient. We need to accumulate gradients for mini-batch training. Elementwise Multiplication gradients: A * B = C del C / del A_ij...

Deep Learning - Gradient Checking

First Step To Debugging A Neural Net

How To Do Gradient Checking In calculus, we all learned that a derivative is defined as: \[\begin{gather*} f'(\theta) = \frac{f(\theta + \epsilon) - f(\theta)}{\epsilon} \end{gather*}\] ...

Deep Learning - Introduction

Why Do We Even Need Deep Neuralnets? Data Partition, ML Ops, Data Normalization

Why Do We Need Deep Learning Any bounded continuous function can be approximated by an arbitrarily large single layer. W hy? The idea is roughly that the linear combinations of activation function...

Computer Vision - Pinhole Camera Model

This Blog Shows How A Small Magic Peek Hole Captures The World

Introduction Camera is intriguing. There have been many different types with different types of lenses (such as fisheye, wide angle Lens). However, the most original (and the simplest) form of cam...

Web Development - Reactive Design

React, Flask

React React itself is just a UI library and doesn’t bundle any network protocol or “live-update” mechanism under the hood. It re-renders your components whenever you call setState (or update state...

Web Development - Flask

React, Flask

Launch an App Since Flask 2.2, threading is by default: app.run(threaded=True) # default since Flask 2.2 the Werkzeug dev server starts one OS‑level process and spawns a new thread...

Computer Vision - Random Tree

Random Tree

Introduction Simple Example: Predicting Lottery Purchase with a Decision Tree Imagine we have two pieces of information about a person: Age Income The task is: we want to predict whether ...

Computer Vision - Ray Tracing, Ray Casting, Gaussian Splatting

Introduction In the 16th century one innovative artist, Albrecht Düre created a “Perspective Machine” to help artists draw perspectives accurately. He did this by creating a screened 2D frame, bet...