Personal Reflections & Tech Insights

Blogs

FPGAs and TPUs: Why AI Inference Might Need More Than GPUs

GPUs dominate AI, but for inference, specialized hardware like FPGAs and TPUs can offer significant advantages in latency, cost, and power efficiency for specific workloads.

Event Sourcing for ML Reproducibility

Reproducing machine learning results can be a nightmare when data and models evolve. Event Sourcing offers a powerful way to build a complete history, making your ML pipeline a true time machine.

Estimating at Scale: Probabilistic Data Structures

Real-time AI analytics often needs to handle massive data streams. Probabilistic data structures offer a clever way to scale by trading exact accuracy for speed and memory efficiency.

RLHF at Scale: Human Feedback in Production

Implementing RLHF for LLMs in production isn't just about algorithms. It's a complex engineering and data challenge involving human feedback at scale.

Graph Neural Networks for Better AI Reasoning

Graph Neural Networks offer a powerful way to handle interconnected data, moving beyond traditional feature engineering to enable more sophisticated AI reasoning.

Semantic Caching for LLMs: Smarter Cache Hits

Traditional caching struggles with LLM prompts. Semantic caching uses embeddings to find similar queries, dramatically improving cache hit rates and reducing costs.

Homomorphic Encryption: Privacy for AI Data

Homomorphic encryption lets you compute on encrypted data without ever decrypting it, offering a powerful way to keep AI data private while still using it.

Stateful Serverless for Long AI Workflows

Serverless functions are designed to be stateless and ephemeral, which often clashes with the long-running, multi-step nature of many AI workflows. Here's how to bridge that gap.

Tracing AI Agent Decisions: The Observability Challenge

Building AI agents is one thing; understanding why they do what they do is another. Tracing their internal decisions is becoming a crucial, complex observability problem.