Sep 16, 2026 · 30 min
Trusted AI needs observability before autonomy
EP 76: Dashboards Are Wrong in the Background. AI Is Wrong in Your Face | Barr Moses, Monte Carlo
The episode argues that reliable agents depend on strong data, context, evaluation, monitoring, and human oversight before organizations pursue scale.
- 1Organizations should choose a concrete business problem before building AI, then establish the data and context foundations it requires.
- 2Monte Carlo’s reinforcement loop uses traces, evaluations, and production signals to improve systems after deployment.
- 3Agent reliability depends on observing context, performance, behavior, and outputs, with stale data posing an especially hidden risk.
Don't miss
Barr Moses identifies stale context as a hidden failure mode that can undermine an agent before its output is generated.
The brief
Barr Moses, Monte Carlo’s co-founder and CEO, frames trusted AI as a production problem: enterprises must move beyond pilots and demos toward systems they can reliably operate.
Her practical starting point is not a model but a business problem. The right use case must be matched with the data, context, and operational foundations needed to support it.
Moses describes Monte Carlo’s reinforcement loop, which learns from traces, evaluations, and production signals as teams discover what goes wrong after deployment.
The conversation’s sharpest warning concerns stale context: agents can fail before generation begins, making context observability as important as performance, behavior, and output monitoring.
The broader argument is measured rather than anti-autonomy. Agents may eventually handle more operational fixes, but humans remain part of the oversight system.
Featuring
Mentioned
Cisco Systems, Inc.Listen to the full episode and explore every guest, topic, and moment on PodLume.