Rico贾若童的博客

Entropy Bottleneck

Entropy Encoding

Problem Setup Suppose one encoder output (latent vector) is z = [0.13, -1.82, 0.07]. Storing these as float32 costs $3 \times 32 = 96$ bits. Entropy coding requires discrete symbols, so we first...

Posted by Rico's Nerd Cluster on February 23, 2026

PyTorch Mixed Precision Training

torch.zeros, GradScaler, GradCheck

Pytorch Setup 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 use_amp = True net = make_model(in_size, out_size, num_layers) opt = torch.optim.SGD(net.parameters(), lr=0.001) # if False, th...

Posted by Rico's Nerd Cluster on February 17, 2026

Deep Learning - Mixed Floating Point Training

FP16, BF16, Mixed Precision Training

Refresher: Floating Point Calculation A floating point is represented as sign bit | exponent | mantissa. 0 | 10000001 | 10000000000000000000000 represents 6 because: Sign bit 0 represents posi...

Posted by Rico's Nerd Cluster on February 14, 2026

[ML] Libraries For Point Cloud Compression

einops

einops Einops is a lightweight Python library that makes tensor reshaping, permutation, tiling, and reduction readable and explicit. It works with PyTorch, NumPy, and TensorFlow. rearrange — r...

Posted by Rico's Nerd Cluster on February 12, 2026

[ML] PyTorch Functions

autograd Function, Convolution, Normalization, Sum, Torch Cache Emptying

Torch.autograd.Function torch.autograd.Function lets you define a custom op with explicit forward() and backward() passes. Common built-ins like MaxPool are implemented this way. It’s especially u...

Posted by Rico's Nerd Cluster on February 9, 2026

[ML] D-PCC Decoder Layers

isohedron, losses

Decoder Overview The decoder reconstructs a dense point cloud from a compressed latent representation through multiple progressive upsampling stages rather than a single large expansion. Instea...

Posted by Rico's Nerd Cluster on February 6, 2026

[ML] D-PCC Encoder-Layers

sub-pixel convolution

D-PCC paper Terminology latent feature: a compressed internal representation that contains essential information needed to reconstruct the original data. Cardinality: the number of elements ...

Posted by Rico's Nerd Cluster on February 3, 2026

[ML] Point Cloud Transformer

Vector Attention

Terminology modulate - In ML, modulation means changing one signal using another signal. - So Attention $output=\alpha \odot v$ with elementwise product is modulation on value features. Th...

Posted by Rico's Nerd Cluster on February 2, 2026

[ML] OpenClaw

OpenClaw Video: https://www.youtube.com/watch?v=Otn-NbpQH1k Start Ollama: 1 2 3 4 5 6 ollama run http://localhost:11434/ ollama launch openclaw # on the control panel /model qwen2.5:7b...

Posted by Rico's Nerd Cluster on February 1, 2026

[CUDA - 6] CUDA Functions

attomicAdd, pragma unroll

atomicAdd There’s no such thing as “atomic memory” in CUDA—atomicity is a property of an operation, not a memory type. atomicAdd is an instruction you apply to a memory location (typically in gl...

Posted by Rico's Nerd Cluster on January 21, 2026

Rico's Nerd Cluster