Random notes
Notes, readings, and experiments on engineering, systems, AI, and anything else worth understanding. A personal log of what I’m learning and building.
Latest Note
DeepSeek V4.1 CED and CSA2 Architecture Notes
A detailed look at causal encoding, compressed sparse attention, cross-layer KV reuse, and the training and inference trade-offs behind the architecture.
Read moreRecent Notes
View by topicAdjacent-Query KV-Index Overlap in 256K DSA Training
Measured KV-index locality for adjacent queries in long-context training.
CUDA VMM and GPU Communication Layers
How virtual memory, fabric handles, NCCL, NVSHMEM, and MoonEP fit together.
Block-Scaled FP8 and FP4 Weight Quantization
Physical layouts, transpose consistency, and 1D versus 2D scaling.
MoonEP Public API Visual Guide
Buffer ownership, zero-copy views, communication plans, and lifecycle.
Scaling K1 Parallelism for Small B × H
Owner-helper scheduling and bounded ring mailboxes for CAKE KDA.
Engineering a Fast Hadamard Transform on Modern NVIDIA GPUs
Butterfly factorization, memory hierarchy, and CTA-dispatch amortization.
Browse by Topic