All 70 articles, sorted alphabetically
Sentence Similarity with BERT and Flask
Sentence Similarity with BERT and Flask
Read article →Evaluating LLM Outputs
LLM-as-judge BLEU/ROUGE limits and human-in-the-loop.
Read article →FlashAttention Explained
Why the same math runs 2-10x faster.
Read article →LLM Agent Architecture in Depth
A 2500-word walkthrough of LLM agent architecture: planner, tool registry, executor, working memory, long-term memory, reflection, guardrails, streami…
Read article →Continuous batching architecture
Deep-dive on LLM continuous batching: paged KV cache, iteration-level scheduling, preemption, fairness, chunked prefill, metrics.
Read article →Beam Search
How beam search maintains multiple candidate sequences to find a higher-probability overall completion.
Read article →LLM Caching Layers
KV cache, prefix cache, semantic cache — all different.
Read article →LLM Caching Strategies
Exact-match semantic and prompt-prefix caching.
Read article →Chain of thought reasoning architecture
Deep-dive on CoT reasoning: prompt patterns, self-consistency, tree-of-thought, verifier, hidden vs visible reasoning, cost tradeoffs.
Read article →LLM chunked prefill architecture
Deep-dive on chunked prefill for LLM serving: how splitting a long prompt into fixed-size token chunks under a single per-step token budget lets the s…
Read article →LLM Context Extension Architecture in Depth
A 2500-word walkthrough of LLM context extension: PI, NTK-aware, YaRN, LongRoPE, fine-tune, long-context eval, KV memory scaling.
Read article →LLM Context Length
What context length means, how it limits input + output, and modern techniques for extending it.
Read article →LLM Context Management
What goes in, what gets summarized, what gets dropped.
Read article →LLM continuous batching architecture
Deep-dive on continuous (iteration-level) batching for LLM serving: instead of static batches that wait for their slowest member, a scheduler recompos…
Read article →LLM evaluation architecture
Deep-dive on LLM evaluation: benchmark suite, chat arena, LLM judge, adversarial, regression set, domain-specific, cost + latency.
Read article →LLM Fine-Tuning Architecture in Depth
A 2500-word walkthrough of LLM fine-tuning: base + dataset, LoRA/QLoRA/Full/DPO methods, adapter merge, evaluation, serving.
Read article →FlashAttention architecture
Deep-dive on FlashAttention: why the GPU memory hierarchy (HBM vs SRAM) makes standard attention memory-bound, how tiling and the online softmax compu…
Read article →Greedy Decoding
How greedy decoding always picks the highest-probability next token, its determinism, and its limitations.
Read article →LLM Guardrails in Production
Input filtering output validation and PII detection.
Read article →LLM guided decoding architecture
Deep-dive on guided (constrained/structured) decoding: why validity can't be reliably obtained by prompting, how a schema or …
Read article →LLM KV-cache offloading architecture
Deep-dive on KV-cache offloading for LLM serving: paged block management, LRU eviction with reuse hints, prefix caching, pinned host staging, async co…
Read article →LLM KV-cache quantization architecture
Deep-dive on KV-cache quantization for LLM serving: why the KV cache dominates memory, quantize-on-write and dequantize-on-read, per-token vs per-chan…
Read article →MoE serving architecture
Deep-dive on Mixture-of-Experts serving: routing, expert parallelism, all-to-all, capacity, load balance, metrics, and ops.
Read article →Multi-LoRA serving architecture
Deep-dive on multi-LoRA serving: one shared base model with many small per-tenant LoRA adapters, the router and adapter cache/store that manage scarce…
Read article →PagedAttention architecture
Deep-dive on PagedAttention and KV-cache management in LLM serving: block managers and free pools, continuous-batching admission, prefix caching with …
Read article →Prefill/decode disaggregation architecture
Deep-dive on disaggregated LLM inference: why prefill and decode interfere, SLO-aware routing, layer-wise KV cache streaming over NVLink/RDMA, prefix …
Read article →LLM priority scheduling architecture
Deep-dive on priority scheduling for LLM inference: priority classes and deadlines, admission control, KV-cache-aware preemption (swap vs recompute), …
Read article →LLM prompt caching -- reusing computed context for cost and latency
Deep-dive on LLM prompt caching: KV cache reuse for repeated prompt prefixes, prefix matching, cost reduction (cached tokens billed less) and latency …
Read article →RAG architecture
Deep-dive on RAG: chunker, embedder, vector store, hybrid retrieval, reranker, generator, eval harness, feedback loop.
Read article →Repetition Penalty
How repetition penalty and frequency penalty discourage repeating tokens, and how they affect quality.
Read article →LLM reranker architecture
Deep-dive on LLM reranker: retriever candidates, cross-encoder scoring, relevance, top-k, latency budget, batching.
Read article →LLM model routing architecture
Deep-dive on LLM routing: feature extraction and learned routers, cascade vs predictive dispatch, quality gates and judge models, escalation ceilings,…
Read article →LLM sampling and decoding architecture
Deep-dive on LLM decoding: the logit vector, masking, repetition/frequency/presence penalties, temperature scaling, top-k vs top-p vs min-p truncation…
Read article →LLM Semantic Caching: Reusing Answers by Meaning, Not Bytes
How a semantic cache embeds queries, searches a vector index of prior Q&A, and serves on a similarity threshold — cutting cost…
Read article →Speculative decoding architecture
Deep-dive on speculative decoding: draft model + target verify + accept-reject, plus tree spec, Medusa heads, and self-speculation variants.
Read article →Speculative Decoding
How speculative decoding uses a small draft model to propose tokens that a large model verifies in parallel, cutting latency.
Read article →LLM streaming output architecture
Deep-dive on LLM output streaming: SSE, serving engine token-by-token, chunked encoding, cancellation, rate, structured events, timeouts.
Read article →Structured output architecture
Deep-dive on structured LLM output: JSON Schema to FSM compilation, token-mask decoding, tokenizer lifting, vLLM/TGI/llama.cpp guided decoding, JSON m…
Read article →LLM Temperature
How temperature scaling changes generation from deterministic to random, and typical values for use cases.
Read article →Tokenization
How tokenizers split text into tokens, byte-pair encoding, and why token count matters for cost and quality.
Read article →LLM tokenization architecture
Deep-dive on LLM tokenization: BPE training and vocabulary-size trades, the runtime encode pipeline with byte fallback, special-token security at trus…
Read article →LLM tool use -- connecting models to actions and data
Deep-dive on LLM tool use (function calling): the model requesting tools via structured output, tool schemas (name/description/params), the call-execu…
Read article →Top-k Sampling
How top-k sampling picks from the k highest-probability tokens, and when it's preferred over top-p.
Read article →Top-p (Nucleus) Sampling
How top-p sampling selects from the smallest set covering probability mass p, adapting to the model's confidence.
Read article →Transformer Architecture Introduction
The transformer architecture in plain terms: embeddings, attention layers, feedforward, output head.
Read article →Mixture of Experts in 2026
Mixtral, DeepSeek MoE, and the architecture's tradeoffs.
Read article →Positional Embeddings: RoPE, ALiBi, and the Quest for Perfect Long-Range Memory
Read article →Prompt Engineering Patterns
Zero-shot few-shot chain-of-thought and reasoning prompts.
Read article →Retrieval-Augmented Generation (RAG): Bridging the Gap Between a Model’s Training and Today’s News
Read article →Reducing LLM Hallucinations
RAG grounding citation requirements and verification chains.
Read article →Reinforcement Learning from Human Feedback (RLHF): The Secret Sauce That Made ChatGPT 'Helpful'
Read article →Rotary Position Embeddings (RoPE)
Why every modern LLM uses it.
Read article →Speculative Decoding: How Using a Tiny Model to 'Guess' Makes the Big Model 3x Faster
Read article →Structured Output with LLMs
JSON schema function calling and constrained decoding.
Read article →