Presentation: Arrowheads and Circles: Building Self-Evaluating Loops in the Team and in the Product
Abstract: As AI systems take on more real work, the hard problem shifts from producing output to knowing whether to trust it. Large Language Models (LLMs) made generation cheap; they did not make trust cheap. This talk is about building self-evaluating loops around AI, guided by one rule at two levels: no participant grades work it has a stake in. First in the team, in the engineering process, through review by a second model prompted to disprove the first, contracts that check the boundaries between components, and pipelines that allow only one source of truth for each piece of data. Then in the product, inside Orion, a collaborative AI analyst platform whose claims ship with automatic checks, tested against live data and replaced rather than quietly edited when they go stale. Real production results included, failures and all. Attendees leave with a one-afternoon audit they can run on their own systems: which feedback signals actually change what the system does, and which are just decoration.
Speaker(s): Mr. Abijith Ramachandran,
Room: 300, Bldg: Ritchie School of Engineering and Computer Science, University of Denver, 2155 E Wesley Ave, Denver , Colorado, United States, 80210