skip to content
Posts · June 2026

Agent from Scratch Part 5: Evaluation

By Chenyi Zhang·28 June 2026 ·5 min read

agent from scratch· ai agents· llm· evaluation· fine-tuning· agent design

Working Is Not Enough

You have a garbage-in-garbage-out agent that technically runs. In most software, that’s a ship. In ML, that’s the starting line.

The question shifts from does it work to how well does it work — and that requires a completely different mindset.

Start With the Goal

Before you can evaluate anything, you need to know what the agent is for.

Without a clear goal, you’re stuck choosing between two bad options: a massive evaluation harness that tries to measure everything, or a frontier model powerful enough to paper over your ambiguity. Neither scales. Neither is reproducible.

No company today ships a reliably good omni-agent — one that handles any task, in any domain, with acceptable quality. The ones that work are narrow. They’re well-scoped.

Before you build a benchmark, write one sentence: what is this agent supposed to do? That sentence is the foundation of every evaluation decision you’ll make.

Understand Your Harness First

Evaluation data should reflect how your agent actually works — not how you imagine it works.

In this series, the agent has three primitives: skills, memories, and CLI commands. That means almost every successful task path looks like: a prompt plus a sequence of CLI calls.

This shapes what you measure. You’re not just checking whether the final output is correct — you’re checking whether the agent took a reasonable path to get there. A correct answer reached through five unnecessary tool calls is still a bad run.

Understand your harness before you design your eval. The harness determines what “good” looks like.

Don’t Create Data Manually

Here’s the most common mistake I see in evaluation pipelines: engineers sitting down to hand-write golden examples.

Manual data creation is expensive, slow, and almost always wrong in subtle ways. Human-written examples embed assumptions about what the agent should do rather than what it actually does well.

The better approach: save real user sessions. Every run your agent completes is a potential data point. What Meta has done at scale — distilling from real behavior — applies just as well to a small personal agent. Let real usage generate the raw data.

What to Annotate

Raw sessions aren’t benchmarks. You need to annotate them.

Two signals matter most:

  1. Step-level correctness — which step went wrong? Tool misuse? Bad reasoning? Wrong skill selected?
  2. Session-level success rate — how many steps did it take? Did it achieve the goal at all?

Step-level annotations are what you need for fine-tuning. Session-level annotations are what you need for your benchmark.

Keep both.

The Sandbox

Once you have good data, you need a place to replay it.

The sandbox is a controlled environment where you can run your benchmark against any change — a prompt tweak, a model upgrade, a new skill — and see whether things improved or regressed.

Without a sandbox, every change is a guess. With one, you have a feedback loop.

Keep your sandbox stable. If the environment itself is flaky, you’ll spend more time debugging the eval than the agent.

Fine-Tuning (When You Get There)

If you have clean step-level data and a stable sandbox, fine-tuning becomes tractable.

This includes:

  • Distillation — compress a larger model’s reasoning into a smaller, faster one
  • Knowledge transfer — transfer domain expertise from annotated sessions
  • SFT on good runs — reinforce the patterns that work

None of this is exotic. The eval pipeline is the fine-tuning pipeline. You’re building the same thing, just using it in two directions: measuring performance now, and improving it later.

AB Testing

One rule: no two changes claim the same reward.

If you’re changing the prompt and changing the model at the same time, you can’t attribute improvement to either. Run one change at a time. Compare against a stable baseline. Keep the sandbox environment constant between runs.

This sounds obvious. It gets ignored constantly. Don’t let two experiments race for credit on the same metric.

Loop It

Once single-pass evaluation is stable, you loop.

An agent that completes tasks, evaluates its own output, and improves its behavior over time is just evaluation running in a cycle. The loop version looks like:

Task → Execute → Evaluate → Flag / Update → Repeat

You can even have an agent run the evaluation loop itself — spawn it, give it the report, let it assess. Just make sure the agent doing the assessment can actually reason correctly. An agent that can’t compare 9.11 and 9.9 reliably should not be evaluating your other agents.

Read every report yourself. Question the outcomes. AI-generated evaluation reports carry the same failure modes as the systems they’re evaluating.

The Eval Pipeline at a Glance

Collect real user sessions
  → Annotate (step-level + session-level)
  → Separate good data for benchmark


Benchmark:
  → Run against stable sandbox
  → One change at a time (proper AB)
  → Compare against baseline


Fine-tuning (optional):
  → Use step-level annotations
  → Distill, transfer, or SFT
  → Re-benchmark after every change


Loop:
  → Agent runs → data accumulates → eval runs → agent improves

No magic. Just a disciplined feedback loop applied consistently.

GitHub

Implementation is at github.com/Czhang0727. The eval scaffolding is thin by design — complexity lives in the annotation quality, not the tooling.

The next post covers Hermes skill management — what happens when your agent accumulates too many skills and starts going off the rails.