AI agent evaluation: trajectories, adversarial tests and regression gates

The problem

An agent that queries data, fills in forms or acts on a user's behalf can't be validated by reading a few answers. You have to check its trajectory: which tool it chose, with which data, whether it stayed within its limits and whether it knew when to stop or ask for help.

What I build

Scenario golden dataset: real cases with the expected trajectory and outcome.
Trajectory metrics: tool choice, permission boundaries, correct termination and escalation to a human when needed.
Adversarial tests: users who go off-script, contradictory data and attempts to push the agent out of scope.
Boundaries in code, not in the prompt: for example, read-only access enforced by a parser.
Regression gates in CI: no change reaches production if it makes the metrics worse.

Experience

I build and operate production agents in the in-house AI department of a European multinational, on Amazon Bedrock AgentCore, with enterprise identity and strict boundaries. My public agent has 138 automated tests, including adversarial cases.

Format and price

A 2-3 week evaluation sprint, from €6,000. Your team keeps the harness, the metrics and the documentation.

FAQ

Does it work for agents built with any framework?

Yes. I evaluate the agent's observable behaviour, whether it's LangGraph, a vendor SDK or custom code.

Does this help with the EU AI Act?

A documented, repeatable evaluation is useful evidence of control and oversight. It doesn't replace legal advice.

Related case studies

An AI agent with boundaries enforced in codeLangGraph agent with SQL, document and maths tools, read-only access enforced by a parser, Langfuse tracing and 138 automated tests including adversarial ones.
Production RAG over US tax regulationRAG system over 50+ IRS documents for a US tax-compliance client: 9 retrieval configurations benchmarked, Recall@5 of 0.98 and on-premise deployment.

Other services

Technical audit of AI, RAG and LLM systemsIndependent audit of RAG, LLM and AI agent systems: architecture, measured quality, cost per operation and risks. Written report with severities. From €500.
Document AI automation for law firms and tax advisorsAI automation of filings and documents for law firms, tax advisors and compliance teams: LLM extraction, validation against official registries and human approval.
AI consultant in the Canary IslandsAI consultant based in Santa Cruz de Tenerife for businesses in the Canary Islands: AI system audits, agents, RAG and document automation. In person in Tenerife or remote.