Rico's Nerd Cluster

「离开世界之前 一切都是过程」

Deep Learning - Activation and Loss Functions

Sigmoid, ReLU, GELU Tanh, Mean Squared Error, Mean Absolute Error, Cross Entropy Loss, Hinge Loss, Huber Loss, IoU Loss, Dice Loss, Focal Loss, Cauchy Robust Kernel

Activation Functions Early papers found out that Rectified Linear Unit (ReLu) is always faster than Sigmoid because of its larger derivatives, and non-zero derivatives at positive regions. Howeve...

Deep Learning - Auto Differentiator From Scratch

Auto Diff Is The Dark Magic Of All Dark Magics Of Deep Learning

Introduction Gradients here refer to scalar to matrix gradient. We need to accumulate gradients for mini-batch training. Elementwise Multiplication gradients: A * B = C del C / del A_ij...

Deep Learning - Gradient Checking

First Step To Debugging A Neural Net

How To Do Gradient Checking In calculus, we all learned that a derivative is defined as: \[\begin{gather*} f'(\theta) = \frac{f(\theta + \epsilon) - f(\theta)}{\epsilon} \end{gather*}\] ...

Deep Learning - Introduction

Why Do We Even Need Deep Neuralnets? Data Partition, ML Ops, Data Normalization

Why Do We Need Deep Learning Any bounded continuous function can be approximated by an arbitrarily large single layer. W hy? The idea is roughly that the linear combinations of activation function...

Computer Vision - Pinhole Camera Model

This Blog Shows How A Small Magic Peek Hole Captures The World

Introduction Camera is intriguing. There have been many different types with different types of lenses (such as fisheye, wide angle Lens). However, the most original (and the simplest) form of cam...

Web Development - Reactive Design

React, Flask

React React itself is just a UI library and doesn’t bundle any network protocol or “live-update” mechanism under the hood. It re-renders your components whenever you call setState (or update state...

Web Development - Flask

React, Flask

Launch an App Since Flask 2.2, threading is by default: app.run(threaded=True) # default since Flask 2.2 the Werkzeug dev server starts one OS‑level process and spawns a new thread...

Computer Vision - Random Tree

Random Tree

Introduction Simple Example: Predicting Lottery Purchase with a Decision Tree Imagine we have two pieces of information about a person: Age Income The task is: we want to predict whether ...

Computer Vision - Ray Tracing, Ray Casting, Gaussian Splatting

Introduction In the 16th century one innovative artist, Albrecht Düre created a “Perspective Machine” to help artists draw perspectives accurately. He did this by creating a screened 2D frame, bet...

Robotics - [Motion Planning 1] Overview And A star Variants

A Star

The “old school” path planning pipeline is: 1 Path Planning -> Trajectory Optimization A Star Setup: Maintain a priority_queue to store all nodes to expand The priority queue i...