Notes on applied AI, software engineering, and technology.
Why a huge context window doesn't save a long agent session, and a working Python context manager that keeps it small, cheap and on task.
26 min read
Measure an LLM judge's position, length and self-preference biases, and its chance-corrected agreement with humans, before its scores drive a decision.
35 min read
How a cheap drafter and a single target forward pass make LLM decoding faster while keeping the target model's exact output distribution, with runnable code and real measurements.
26 min read
How grammar-constrained decoding masks logits to force valid JSON, built from scratch in Python, and the failure modes a schema can't catch.
25 min read
How KV caches and prefix caching work inside inference engines, and how to lay out agent prompts so provider prompt caching actually hits.
31 min read
Use idempotency keys to make timed-out POST retries safe from duplicate side effects.
20 min read
Migrate a monolith one route at a time, with clear data ownership and a rollback for each slice.
18 min read
Secure MCP tools against prompt injection with explicit authorization, validation, and tested boundaries.
22 min read
Build a versioned RAG test set and gate retrieval quality and answer faithfulness in CI.
22 min read
Deploy a RAG API to Azure Container Apps with managed identity, Key Vault, scaling, and health probes.
19 min read