Backend engineer building AI infrastructure

How to Judge a RAG System

Grading a RAG answer is grading an open-book exam. Two separate questions: did it find the right pages, and did the answer actually use them.

Part eight of the RAG series. A wrong RAG answer has two possible culprits, and one number cannot tell them apart. Evaluation splits like grading an open-book exam: retrieval quality (context precision, did it bring back relevant pages and rank them first; context recall, did it miss any needed page) and generation quality (faithfulness, did the answer only say what the pages support; response relevancy, did it address the question asked). This defines a classic four-metric starting set from Ragas, each with its formula and a worked example, explains which need a ground-truth reference and which use an LLM as judge, and why an LLM-judge score is a moving target. Everyday analogies, a hand-drawn diagram, and a visualizer per metric.

• RAG

Peer-reviewer for REALM 2026

Reviewer for REALM 2026, the 2nd Workshop for Research on Agent Language Models at EMNLP 2026 (Budapest, Hungary), reviewing one submitted paper on language agents.

Area Chair for GroundLM 2026

Area Chair for GroundLM 2026, the workshop on grounding and retrieval-augmented language models at EMNLP 2026 (Budapest, Hungary), overseeing five submissions and writing meta-review recommendations.

• RAG

RAGOps: RAG Is a Distributed System, Not a Demo

A demo answers one question once. Production answers forever, over a corpus that never stops changing.

Part seven of the RAG series, and the first on running RAG in production. A demo retrieves from a fixed pile of documents and answers one question. Production may retrieve from a corpus that is changing continuously, so the hard part is often not the answer, it is keeping the index true to the source: continuous ingestion and incremental re-indexing for freshness, deletion that removes a document's chunks so the model cannot cite what no longer exists, a migration plan for the day you change the embedding model and old vectors no longer share a coordinate system with new ones, and a trace on every answer so you can see why it said what it said. Grounded in the RAGOps paper. Everyday analogies, a hand-drawn diagram, and interactive visualizers.

• RAG

Peer-reviewer for EMNLP 2026 Industry Track

Reviewer for the EMNLP 2026 Industry Track (Budapest, Hungary), reviewing one submitted paper on deploying natural-language-processing systems in real-world settings.

Peer-reviewer for 2nd Workshop on Efficient Reasoning

Peer reviewer for the 2nd Workshop on Efficient Reasoning at COLM 2026, reviewing three submitted papers.

• Inference

Judging at Hack-Nation: 6th Global AI Hackathon

Judged the 6th Global AI Hackathon, Hack-Nation's largest to date (5,500+ applications, some 2,500 builders across 12 hubs), run with the MIT Club of Northern California and the MIT Club of Germany. Scored seven projects on the jury.

Peer-reviewer for GroundLM 2026

Reviewer for GroundLM 2026, the workshop on grounding and retrieval-augmented language models at EMNLP 2026 (Budapest, Hungary).

• RAG

Peer-reviewer for Automated Software Engineering

Artifact evaluator for ASE 2026 (IEEE/ACM International Conference on Automated Software Engineering).

In the Machine Room

The last five parts were the recipe. This one is the kitchen at dinner rush: the two engineering tricks that get DSpark running inside a real serving system, and the speedups it delivers there.

Part six of the speculative decoding series, the finale. Parts one to five were the idea. This one is the engineering: two tricks that get DSpark running inside a live serving system. First, do not ship the whole dictionary of scores between machines while training the guesser, ship the compact summary and do the last step locally. Second, a scheduler that cannot wait, because real GPU speed rises in jagged steps and the pipeline cannot pause, so it predicts the batch size ahead of time. Plus the real production speedups. Plain words, an analogy, a diagram, and two auto-playing visualizers.

• Inference