Rico贾若童的博客

Deep Learning - Why 100 GB of NumPy Data Slowed Down Our ML Training

NumPy Memory Mapping, Linux Page Cache, Swap, and Ragged Arrays

1. The Problem Suppose we’re training a 3D LiDAR object recognition model to detect cars, pedestrians, and cyclists. We have three training configurations: centred crops, shifted crops, and tempor...

Posted by Rico's Nerd Cluster on October 16, 2026

Deep Learning - Contrastive Learning

Supervised Contrastive Learning (SupCon)

Supervised contrastive learning (SupCon) trains a model to produce embeddings in which images of the same class are close together and images of different classes are far apart. Ordinary classifica...

Posted by Rico's Nerd Cluster on October 13, 2026

Deep Learning - Flash Attention

IO-Aware Attention, Tiling, Online Softmax

What FlashAttention Does FlashAttention computes exact attention without storing the huge $N\times N$ attention matrix: \[\operatorname{Attention}(Q,K,V) = \operatorname{softmax}\left(\frac{QK^...

Posted by Rico's Nerd Cluster on October 10, 2026

Deep Learning - Transformer Series 6 - Decoder-Only LLMs and KV Cache

Next-Token Training, Causal Mask, LM Head, KV Cache

How a Decoder-Only LLM Is Trained Suppose we want to train a model to answer questions. Consider this training example: 1 [BOS] [USER] What do you see ? [ASSISTANT] I see a robot Here, [BOS] is...

Posted by Rico's Nerd Cluster on October 7, 2026

Robotics - [Jetson 2] Running the Orin: Power Modes, Benchmarking, and Pitfalls

Power modes, sustained performance, memory reporting, Wifi, and how I burned a board

This is part 2 of my Jetson notes. Part 1, Flashing the Orin Nano, covers getting JetPack onto the board. I Burned My $500 Nvidia-nano-orin This happened when I connected it to a Waveshare Rover ...

Posted by Rico's Nerd Cluster on October 4, 2026

[ML] Neural Architecture Search (NAS) in RF DETR

The main idea of the NAS RF-DETR is different resolutions do produce different size tensors and therefore different sized computation graphs, but they can still use the same learned weights. Traini...

Posted by Rico's Nerd Cluster on August 14, 2026

[ML] FiLM Block

FiLM (Feature-wise Linear Modulation ) Block A convolutional layer produces feature channels that represent learned concepts such as edges, shadows, textures, or object responses. FiLM uses the co...

Posted by Rico's Nerd Cluster on August 11, 2026

[ML] Soft Rounding - Differentiable Quantization for Neural Networks

Soft rounding is a smooth approximation to normal rounding. Normal rounding maps a continuous value to a discrete level: \[1.2 \rightarrow 1\] \[1.8 \rightarrow 2.\] This is useful for modeling ...

Posted by Rico's Nerd Cluster on August 8, 2026

[ML] Loss For Distribution Stats

loss for standard devations

Matching Real Point-Cloud Statistics with a Differentiable Distribution Loss Sometimes we want a synthetic point-cloud generator to produce range and intensity values that have the same statistica...

Posted by Rico's Nerd Cluster on August 2, 2026

[ML] Flow Matching

Optimal Transport, Concentration of Measures

1. Problem Setup and Definitions Imagine we now have a bunch of particles in the 1D world. They are all moving randomly. Along x axis, its current distribution can be characterized as a source dis...