Publications & Research
Technical reports, preprints, and research papers on LLM inference systems, ML infrastructure, and production AI — grounded in real engineering experience.
Parallelism Strategies for Large-Scale LLM Inference: A Practitioner's Analysis
A systematic comparison of tensor, pipeline, and sequence parallelism strategies for LLM serving at scale. Includes empirical benchmarks across model sizes, hardware configurations, and serving frameworks.
KV Cache Management in Production LLM Serving Systems
Analysis of KV cache design patterns, eviction policies, and prefix caching strategies in production deployments. Based on observations from real-world serving infrastructure.
Cost Modeling for LLM Inference at Scale
A framework for accurately modeling and forecasting LLM inference costs across cloud, dedicated GPU, and hybrid deployments. Covers compute, memory bandwidth, network, and operational overhead.
Related Deep Dives
Extended technical articles that serve as the foundation for the research above.
Architecting LLM Inference — Part 6: Parallelism
Tensor, pipeline, and sequence parallelism for large-scale LLM serving
Understanding PagedAttention and KV Cache
How vLLM's memory management changed LLM serving
ML Platform Design for Production AI
Feature stores, model registries, and the infrastructure layer under your models
Get Notified
New technical reports and preprints are announced on Substack first.
Subscribe to Newsletter