Rico's Nerd Cluster

「离开世界之前 一切都是过程」

Math - Null Space and Singular Value Decomposition (SVD)

How to find Null Space? How to implement that using Singular Value Decomposition (SVD)?

Introduction The three most basic & important matrix factorizations in pure & applied linear algebra are: QR Decomposition (Gram-Schmidt Orthogonalization) LU Decomposition (Gaussian...

Math - Interpoloation

Linear, Cubic, Bicubic Splines

Interpolation “Interpolation” is essentially “how to get through lines and predict the value of a point in between?”. The first step is to connect points. For that, we need a spline. A spline is a...

Math - Positive Definiteness and Cholesky Decomposition

A matrix is positive definite, if it always sees the glass as half full. But why does the matrix still go to therapy? To break down its issues with Cholesky decomposition. Just Joking.

Positive Definiteness A matrix is Positive definite when for any x: \[\begin{gather*} x^T A x > 0 \end{gather*}\] For an nxn real symmetric matrix. It’s equivalent to say “it’s positive defin...

Math - Matrix Derivatives

Jacobian and Hessian, Vector-Vector Derivative

Jacobian and Hessian Jacobian Suppose we have a vector-valued function \[\begin{gather*} \begin{aligned} & f = [f_0(x), ... f_m(x)] \\ & x = [x_0, ... x_n] \end{aligned} \end{gather*}\] ...

Math - Eigen Value, Eigen Vector, and Eigen Value Decomposition

What is Eigen Value Decomposition?

Eigen Values and Eigen Vectors Basic Definitions \[\begin{gather*} Ax = \lambda x \end{gather*}\] $x$ is an eigen vector, $\lambda$ is an eigen value Important properties Only square ma...

Math Trigonometry Review

t a— layout: post title: Math - Trigonometry Review date: ‘2017-01-13 13:19’ subtitle: comments: true tags: - Math — Half Angle Formulas \[\begin{gather*} cos \theta = 2 cos^2 \frac{\theta}{2...

Math - Different Types Of Convolutions

Transpose Convolution (U-Net), Dilated Convolution (DeepLab)

Transpose Convolution Definition Transpose Convolution (a.k.a upsampling convolution) was designed specifically for upsampling, which is then used in the decoder part of an encoder-decoder networ...

Math - Gram Schmidt Orthogonolization And QR Decomposition

Gram Matrix; Super useful in finding forming an orthogonal vector basis, e.g., Singular Value Decomposition

Background Knowledge Glossary Inner product: $<a,b> = \vec{a}^T \cdot \vec{b}$, which is a.k.a “dot product” Outer product: $a \otimes b = \vec{a} \cdot \vec{b}^T$, which results in a ...

Math - Various Useful Forms Of Matrix Multiplication

Inner & outer Product, Correlation Matrix, Matrix Inverse etc.

Glossary Inner product: $<a,b> = \vec{a}^T \cdot \vec{b}$, which is a.k.a “dot product” Outer product: $a \otimes b = \vec{a} \cdot \vec{b}^T$, which results in a matrix. Hadamard (Sc...