AI
AI gateway rate limiting: token bucket vs sliding window
Implement dual-dimensional rate limiting for AI gateways to handle TPM and RPM limits while preventing 429 errors during traffic spikes.
// category archive
AI
135 articles
AI
LangGraph checkpoint serialization: SQLite vs PostgreSQL
Deploy the correct LangGraph checkpoint backend by comparing SQLite and PostgreSQL performance, migration steps, and failure recovery strategies.
AI
Vector temporal indexing for AI agent memory retrieval
Implement time-decayed retrieval to fix agent hallucination errors by weighting recency and relevance in vector databases for accurate episodic memory.
AI
RadixAttention memory layout strategies explained
Diagnose GPU OOM errors and low cache hit rates by comparing contiguous vs paged KV cache layouts for LLM inference workloads.
AI
Speculative Decoding LLaMA Speedup Measurement
Construct a reproducible benchmarking harness to measure speculative decoding speedup using wall-clock time and valid statistical sampling methods.
AI
KV cache offloading NVMe: vLLM vs TensorRT-LLM
Compare vLLM and TensorRT-LLM KV cache offloading architectures to select the correct system for your latency constraints and deploy long-context models confidently.
AI
Redis eviction policies for LLM semantic caches
Configure Redis eviction policies to align memory pressure with LLM token consumption patterns and reduce API costs.
cover
AI
Agent tool retry pattern with exponential backoff
Implement retry logic with exponential backoff and idempotency keys to prevent duplicate side effects in your AI agent tool calls.
cover
AI
GraphRAG indexing pipeline architecture for million-node gra
Build a fault-tolerant GraphRAG indexing pipeline architecture with stage isolation and resource planning to handle million-node knowledge graphs.
cover
AI
MLA attention mechanism code implementation in PyTorch
Implement Multi-head Latent Attention in PyTorch to compress KV cache memory by 83% while preserving model quality for long-context LLM inference.
cover
AI
Semantic cache OpenAI API with Redis reduces costs
Implement a semantic cache for OpenAI API calls with RedisVL to eliminate duplicate LLM requests and significantly lower your application's operating costs.
cover
AI
AI gateway circuit breaker pattern for LLM fallback routing
Deploy a state-aware AI gateway circuit breaker that routes around degraded LLM providers in under 50ms with zero request loss and intelligent failover.
AI
LangGraph human-in-the-loop approval patterns explained
Configure production-grade human-in-the-loop approval workflows in LangGraph with persistent state that survives restarts, handles concurrency, and ensures data integrity.
AI
Claude vs OpenAI Assistants: Build Fast GUI Agent (2026)
Desktop AI freezes when Claude streams token‑by‑token. Fix it with Claude vs OpenAI Assistants API: async loop, SQLite store, hybrid fallback for GUI agents.
AI
LangGraph checkpoints enable human approval workflows
Implement LangGraph checkpoints to build resilient human approval workflows with proper state persistence and error recovery patterns.
cover
AI
Tiered agent memory architecture: working vs episodic
Implement a three-tier agent memory architecture to reduce redundant API calls and maintain coherent conversations with eviction policies.
AI
RadixAttention prefix caching: vLLM latency reduction
Understand how RadixAttention prefix caching in vLLM eliminates redundant prefill computation to significantly reduce time-to-first-token latency.
AI
MCP server STDIO transport: build custom tools with Python
Build a Python MCP server using STDIO transport with raw JSON-RPC, diagnose real transport failures, and implement maintainable SDK-based patterns.