Rico's Nerd Cluster

Deep Learning - Grad-CAM

A Step-by-Step Example with a Tiny Neural Network

Deep Learning - Integrated Gradients

A Step-by-Step Example with a Tiny Neural Network

Deep Learning - Learning Rate Schedules

Warmup, LambdaLR, Cosine Decay, Cosine Annealing, SGDR

Deep Learning - Truncated Normal Distribution

Rejection Sampling, trunc_normal_, ViT Positional Embedding Initialization

Deep Learning - Why 100 GB of NumPy Data Slowed Down Our ML Training

NumPy Memory Mapping, Linux Page Cache, Swap, Pause and Resume, and Ragged Arrays

Deep Learning - Contrastive Learning

Supervised Contrastive Learning (SupCon)

Deep Learning - Flash Attention

IO-Aware Attention, Tiling, Online Softmax

Deep Learning - Transformer Series 6 - Decoder-Only LLMs and KV Cache

Next-Token Training, Causal Mask, LM Head, KV Cache

Robotics - [Jetson 2] Running the Orin: Power Modes, Benchmarking, and Pitfalls

Power modes, sustained performance, memory reporting, Wifi, and how I burned a board

[ML] Neural Architecture Search (NAS) in RF DETR