Donovan Brady, VP of Developer Relations

Cloud engineering learned a hard lesson long ago: resilient systems are not built by pretending failure can be eliminated. They are built by assuming failure will happen, then designing the controls, observability, automation, and recovery paths needed to limit its impact.

As organizations adopt generative AI and agentic workflows, much of the conversation focuses on reducing hallucinations and improving model accuracy. Those efforts matter, but they only solve half of the problem. AI systems will still produce incorrect, incomplete, or unsupported outputs. They may retrieve the wrong context, misuse tools, follow bad instructions, or behave inconsistently across environments. For QA teams, the real question is no longer simply “Did the application work?” but “Can we trust, verify, trace, and govern the outcome?”

This session explores how principles from cloud architecture, AWS Well-Architected thinking, CI/CD, and resilient system design can be applied to modern AI quality. We will look at why evaluation harnesses are becoming the AI-era equivalent of deployment pipelines: controlled environments for testing prompts, data, reasoning, tool use, traceability, evidence, and output quality before AI reaches the business process.

Attendees will leave with a practical mental model for AI quality: stop treating hallucination as an exception and start treating it as a reliability condition to be tested, monitored, measured, and governed.

Sponsored By: