https://srini-anirudh.github.io/blogs/ First-principles, illustrated deep dives into how modern AI models are built, trained, and scaled. en Sat, 05 Sep 2026 00:00:00 +0530 https://srini-anirudh.github.io/blogs/the-agent-evaluation-playbook/ https://srini-anirudh.github.io/blogs/the-agent-evaluation-playbook/ Sun, 06 Sep 2026 00:00:00 +0530 How to evaluate agent trajectories, world-state changes, reliability, safety, graders, budgets, and long-horizon failures. https://srini-anirudh.github.io/blogs/what-autograd-actually-does/ https://srini-anirudh.github.io/blogs/what-autograd-actually-does/ Sat, 05 Sep 2026 00:00:00 +0530 Backpropagation and autograd from scalar chain rules to Jacobian products, saved tensors, activation memory, and checkpointing. https://srini-anirudh.github.io/blogs/why-transformers-do-not-explode/ https://srini-anirudh.github.io/blogs/why-transformers-do-not-explode/ Sat, 05 Sep 2026 00:00:00 +0530 Initialization, signal propagation, residual paths, normalization, optimizer dynamics, precision, and training stability. https://srini-anirudh.github.io/blogs/choosing-the-next-token/ https://srini-anirudh.github.io/blogs/choosing-the-next-token/ Sat, 05 Sep 2026 00:00:00 +0530 Temperature, beam search, sampling, truncation, penalties, grammars, and contrastive generation from first principles. https://srini-anirudh.github.io/blogs/teaching-a-language-model-to-follow-instructions/ https://srini-anirudh.github.io/blogs/teaching-a-language-model-to-follow-instructions/ Sat, 05 Sep 2026 00:00:00 +0530 Supervised fine-tuning as behavior cloning: data construction, loss masking, distribution shift, and capability change. https://srini-anirudh.github.io/blogs/how-lora-changes-a-model-without-rewriting-it/ https://srini-anirudh.github.io/blogs/how-lora-changes-a-model-without-rewriting-it/ Sat, 05 Sep 2026 00:00:00 +0530 PEFT from first principles: low-rank updates, LoRA, QLoRA, DoRA, adapters, memory costs, and tradeoffs. https://srini-anirudh.github.io/blogs/inside-an-ai-agent-harness/ https://srini-anirudh.github.io/blogs/inside-an-ai-agent-harness/ Sat, 05 Sep 2026 00:00:00 +0530 Anatomy of an agent harness: context assembly, actions, verification, containment, budgets, event logs, and compaction. https://srini-anirudh.github.io/blogs/memory-is-more-than-context/ https://srini-anirudh.github.io/blogs/memory-is-more-than-context/ Sat, 05 Sep 2026 00:00:00 +0530 Agent memory and context engineering as a bounded problem of writing, updating, retrieving, ranking, and forgetting. https://srini-anirudh.github.io/blogs/how-multi-agent-systems-actually-coordinate/ https://srini-anirudh.github.io/blogs/how-multi-agent-systems-actually-coordinate/ Sat, 05 Sep 2026 00:00:00 +0530 Multi-agent systems through diversity, specialization, communication topology, coordination cost, orchestration, and failure modes. https://srini-anirudh.github.io/blogs/reinforcement-learning-for-agents-from-first-principles/ https://srini-anirudh.github.io/blogs/reinforcement-learning-for-agents-from-first-principles/ Sat, 05 Sep 2026 00:00:00 +0530 Reinforcement learning for agents: environments, reward design, credit assignment, horizons, rollout systems, and evaluation. https://srini-anirudh.github.io/blogs/how-attention-moves-information/ https://srini-anirudh.github.io/blogs/how-attention-moves-information/ Sat, 05 Sep 2026 00:00:00 +0530 A first-principles derivation of QKV attention and its evolution through MHA, GQA, sparse, linear, latent, and multimodal forms. https://srini-anirudh.github.io/blogs/the-puzzle-of-overparameterization/ https://srini-anirudh.github.io/blogs/the-puzzle-of-overparameterization/ Sat, 05 Sep 2026 00:00:00 +0530 Double descent, benign overfitting, implicit bias, function space, grokking, and what they do and do not explain about LLM generalization. https://srini-anirudh.github.io/blogs/how-llms-are-pretrained/ https://srini-anirudh.github.io/blogs/how-llms-are-pretrained/ Sat, 05 Sep 2026 00:00:00 +0530 How pretraining corpora are acquired, filtered, deduplicated, mixed, tokenized, packed, scheduled, and evaluated. https://srini-anirudh.github.io/blogs/mixing-data-without-losing-capabilities/ https://srini-anirudh.github.io/blogs/mixing-data-without-losing-capabilities/ Sat, 05 Sep 2026 00:00:00 +0530 Post-training data mixtures, task interference, capability transfer, catastrophic forgetting, and practical mitigation strategies. https://srini-anirudh.github.io/blogs/how-language-models-learn-to-use-tools/ https://srini-anirudh.github.io/blogs/how-language-models-learn-to-use-tools/ Sat, 05 Sep 2026 00:00:00 +0530 Tool schemas, function-call training, retrieval over large catalogs, synthetic trajectories, constrained decoding, and executable rewards. https://srini-anirudh.github.io/blogs/the-making-of-an-ai-agent/ https://srini-anirudh.github.io/blogs/the-making-of-an-ai-agent/ Sat, 05 Sep 2026 00:00:00 +0530 Agent loops, trajectory data, imitation and reinforcement learning, long-horizon credit assignment, state, harnesses, and environments. https://srini-anirudh.github.io/blogs/what-changed-inside-the-transformer/ https://srini-anirudh.github.io/blogs/what-changed-inside-the-transformer/ Fri, 04 Sep 2026 00:00:00 +0530 A first-principles guide to the design pressures that turned the original Transformer into modern dense, MoE, and hybrid language models. https://srini-anirudh.github.io/blogs/scaling-laws-from-first-principles/ https://srini-anirudh.github.io/blogs/scaling-laws-from-first-principles/ Fri, 04 Sep 2026 00:00:00 +0530 Scaling laws from first principles: compute accounting, compute-optimal allocation, data quality, deployment economics, and empirical planning. https://srini-anirudh.github.io/blogs/reinforcement-learning-from-first-principles/ https://srini-anirudh.github.io/blogs/reinforcement-learning-from-first-principles/ Fri, 04 Sep 2026 00:00:00 +0530 Policy gradients, baselines, advantages, GAE, trust regions, and PPO derived and translated into language-model rollouts. https://srini-anirudh.github.io/blogs/teaching-a-model-what-we-prefer/ https://srini-anirudh.github.io/blogs/teaching-a-model-what-we-prefer/ Fri, 04 Sep 2026 00:00:00 +0530 How comparisons become rewards, why RLHF uses a reference-policy KL, and how the objective becomes DPO. https://srini-anirudh.github.io/blogs/how-reinforcement-learning-teaches-models-to-reason/ https://srini-anirudh.github.io/blogs/how-reinforcement-learning-teaches-models-to-reason/ Fri, 04 Sep 2026 00:00:00 +0530 RLVR, verifiers, GRPO, DeepSeek-R1, exploration, and the engineering lessons from open reasoning-model replications. https://srini-anirudh.github.io/blogs/thinking-in-tokens/ https://srini-anirudh.github.io/blogs/thinking-in-tokens/ Fri, 04 Sep 2026 00:00:00 +0530 How reasoning tokens provide serial compute, support search and refinement, and sometimes lead to overthinking. https://srini-anirudh.github.io/blogs/how-models-improve-without-changing-their-weights/ https://srini-anirudh.github.io/blogs/how-models-improve-without-changing-their-weights/ Fri, 04 Sep 2026 00:00:00 +0530 How to allocate inference compute across longer chains, sampling, search, and verification under per-prompt budgets. https://srini-anirudh.github.io/blogs/how-to-find-what-is-slowing-your-model/ https://srini-anirudh.github.io/blogs/how-to-find-what-is-slowing-your-model/ Fri, 04 Sep 2026 00:00:00 +0530 A practical guide to profiling ML systems with timelines, GPU counters, Roofline analysis, PyTorch Profiler, Nsight, and bottleneck-driven optimization. https://srini-anirudh.github.io/blogs/the-serving-playbook/ https://srini-anirudh.github.io/blogs/the-serving-playbook/ Fri, 14 Aug 2026 00:00:00 +0530 A first-principles guide to continuous batching, paged KV, prefix caching, chunked prefill, disaggregation, speculative decoding, and serving goodput. https://srini-anirudh.github.io/blogs/the-mechanics-of-llm-inference/ https://srini-anirudh.github.io/blogs/the-mechanics-of-llm-inference/ Fri, 17 Jul 2026 00:00:00 +0530 A first-principles guide to autoregressive inference, prefill, decode, KV cache, batching, bandwidth, latency metrics, quantization, and VLM workloads. https://srini-anirudh.github.io/blogs/the-parallelism-playbook/ https://srini-anirudh.github.io/blogs/the-parallelism-playbook/ Fri, 19 Jun 2026 00:00:00 +0530 A first-principles guide to DDP, ZeRO/FSDP, tensor, pipeline, context, expert, and hybrid parallelism. https://srini-anirudh.github.io/blogs/inside-a-training-step/ https://srini-anirudh.github.io/blogs/inside-a-training-step/ Fri, 15 May 2026 00:00:00 +0530 A first-principles walkthrough of transformer training: tensor shapes, forward and backward passes, FLOPs, activation lifetimes, memory accounting, and checkpointing. https://srini-anirudh.github.io/blogs/talk-is-not-cheap/ https://srini-anirudh.github.io/blogs/talk-is-not-cheap/ Fri, 17 Apr 2026 00:00:00 +0530 A first-principles guide to PCIe, NVLink, NVSwitch, RDMA, InfiniBand, topology, collective communication, and overlap. https://srini-anirudh.github.io/blogs/why-fast-gpus-still-wait-for-memory/ https://srini-anirudh.github.io/blogs/why-fast-gpus-still-wait-for-memory/ Fri, 13 Mar 2026 00:00:00 +0530 Arithmetic intensity and the Roofline model derived from first principles and applied to GEMM, attention, fusion, optimizers, and ML kernels. https://srini-anirudh.github.io/blogs/why-gpus-are-built-for-deep-learning/ https://srini-anirudh.github.io/blogs/why-gpus-are-built-for-deep-learning/ Fri, 13 Feb 2026 00:00:00 +0530 Why CPUs and GPUs spend silicon differently, from forms of parallelism and execution hierarchy to latency hiding, memory access, tiling, and Tensor Cores. https://srini-anirudh.github.io/blogs/the-vlm-architecture-gallery/ https://srini-anirudh.github.io/blogs/the-vlm-architecture-gallery/ Fri, 21 Nov 2025 00:00:00 +0530 Seventy-one vision-language model architectures on one standardized plate, from gated cross-attention to encoders absorbed into the base model. https://srini-anirudh.github.io/blogs/inside-a-modern-vision-encoder/ https://srini-anirudh.github.io/blogs/inside-a-modern-vision-encoder/ Fri, 17 Oct 2025 00:00:00 +0530 What changed in the visual half of VLMs: intake geometry, resolution, objectives, connectors, and the encoders models actually ship. https://srini-anirudh.github.io/blogs/one-transformer-two-modalities/ https://srini-anirudh.github.io/blogs/one-transformer-two-modalities/ Fri, 19 Sep 2025 00:00:00 +0530 A block-by-block comparison of language Transformers and Vision Transformers. https://srini-anirudh.github.io/blogs/an-image-is-a-sentence/ https://srini-anirudh.github.io/blogs/an-image-is-a-sentence/ Fri, 15 Aug 2025 00:00:00 +0530 An interactive field guide to ViT, DeiT, PVT, Swin, CvT, ConViT, CoAtNet, MViT, TimeSformer, and ViViT. https://srini-anirudh.github.io/blogs/across-the-cnnverse/ https://srini-anirudh.github.io/blogs/across-the-cnnverse/ Fri, 11 Jul 2025 00:00:00 +0530 A visual field guide to convolution and the lineage from LeNet through ResNet, EfficientNet, and ConvNeXt. https://srini-anirudh.github.io/blogs/evolution-of-ml-architectures/ https://srini-anirudh.github.io/blogs/evolution-of-ml-architectures/ Fri, 23 May 2025 00:00:00 +0530 Sixty-seven years of neural architectures, information flow, training cost, inference state, and gradient paths. https://srini-anirudh.github.io/blogs/how-models-know-where-they-are/ https://srini-anirudh.github.io/blogs/how-models-know-where-they-are/ Fri, 18 Apr 2025 00:00:00 +0530 Absolute, sinusoidal, relative, RoPE, NoPE, 2D and Fourier position features, and multimodal M-RoPE explained visually. https://srini-anirudh.github.io/blogs/what-an-optimizer-actually-does/ https://srini-anirudh.github.io/blogs/what-an-optimizer-actually-does/ Fri, 21 Mar 2025 00:00:00 +0530 A mechanical tour from SGD to Muon: optimizer updates, state costs, and distributed training bottlenecks. https://srini-anirudh.github.io/blogs/the-geometry-of-normalization/ https://srini-anirudh.github.io/blogs/the-geometry-of-normalization/ Fri, 14 Feb 2025 00:00:00 +0530 Normalization from BatchNorm to Peri-LN, including why the modality interface in a VLM is a normalization problem. https://srini-anirudh.github.io/blogs/why-neural-networks-need-nonlinearity/ https://srini-anirudh.github.io/blogs/why-neural-networks-need-nonlinearity/ Fri, 17 Jan 2025 00:00:00 +0530 From sigmoid and ReLU to GELU, SwiGLU, and squared ReLU, with animated derivations and model comparisons.