Agentic flows and evaluation harness to automate reporting and high-quality customer replies with guardrails.
Support and reporting teams were spending hours triaging messages, generating routine summaries, and drafting replies. Output quality varied by agent, SLAs slipped on busy days, and leadership had little visibility into response accuracy or risk (PII, policy violations).
Designed an agentic workflow with FastAPI orchestration and LangChain tools. An intake router classifies requests, dispatches to specialized agents (report writer, reply composer), and validates outputs via an eval harness (content policy, PII checks, factuality). Failures trigger automatic fallback and human-in-the-loop escalation. Metrics are logged for continuous improvement.
✓ −60% manual time on routine reporting and templated replies
✓ +25% reply accuracy in evals vs. baseline macros
✓ <60s time-to-first-response for priority inbound
✓ 0 PII leakage incidents during pilot thanks to guardrails