An assistant that answers tax questions citing the exact source, measured with numbers before going live.
The challenge
A US tax-compliance client needed reliable answers over more than 50 IRS documents. Tax language mixes exact codes (forms, sections) with natural-language questions, and mistakes are expensive.
How I solved it
Golden dataset: 30 real questions with the answer and source passage annotated by hand.
Nine retrieval configurations compared one variable at a time: keyword search (BM25), semantic search (embeddings), hybrid with rank fusion, and cross-encoder reranking.
A custom citation-accuracy metric: it checks that each citation actually supports the claim.
Fully on-premise deployment with Docker, local models and a self-hosted vector database.
Result
The winning configuration (hybrid with reranking) finds the right document in the top five for 98% of questions (Recall@5 = 0.98; MRR = 0.80). The client left a 5.0 review: “Eduardo was fantastic… very pleased with how things turned out.”
Automating official filings for the legal sectorAI automation of official forms for law firms: LLM extraction, validation against official registries and human approval. 75-90% less manual time, zero incorrect filings.
An AI agent with boundaries enforced in codeLangGraph agent with SQL, document and maths tools, read-only access enforced by a parser, Langfuse tracing and 138 automated tests including adversarial ones.