Personal Reflections & Tech Insights

Blogs

Event Sourcing for ML Reproducibility

Reproducing machine learning results can be a nightmare when data and models evolve. Event Sourcing offers a powerful way to build a complete history, making your ML pipeline a true time machine.

Making Distributed Systems Consistent with Raft

Building reliable distributed systems means handling failures while keeping data consistent. Raft consensus helps manage this complexity through a clear approach to agreement.

Estimating at Scale: Probabilistic Data Structures

Real-time AI analytics often needs to handle massive data streams. Probabilistic data structures offer a clever way to scale by trading exact accuracy for speed and memory efficiency.

Stateful Serverless for Long AI Workflows

Serverless functions are designed to be stateless and ephemeral, which often clashes with the long-running, multi-step nature of many AI workflows. Here's how to bridge that gap.

Transactional Outbox: Reliable Events for Microservices

Building reliable microservice communication is hard. The transactional outbox pattern helps ensure events are published consistently with database changes, avoiding common pitfalls.

Keeping Edge AI Models Fresh with OTA Updates

Edge AI models aren't static. Learn why Over-the-Air updates are critical for maintaining their performance and how they differ from typical software updates.

Multi-Tenant LLMs: Isolation and Resource Management

Serving large language models to multiple users efficiently introduces significant challenges around isolation, performance, and resource management. This article explores practical strategies to handle these complexities.

LLM Caching: When Key-Value Stores Fall Short

Caching LLM responses with traditional key-value stores often misses the mark because queries aren't exact. Semantic caching matches similar questions to provide better hit rates and reduce API costs.

Dynamic Batching: More AI Throughput, More Latency?

Dynamic batching can significantly boost AI model inference throughput by grouping requests. But it's a tradeoff: better throughput often means higher latency for individual requests.