--- title: "AI Agent Observability With Ivana Huckova and Sven Großmann" description: "Grafana's Ivana Huckova and Sven Großmann discuss AI agent observability, coding-agent costs, PR analytics, secret redaction, and local guardrails." url: https://www.callstack.com/podcasts/ai-agent-observability-with-ivana-huckova-and-sven-grossmann updated: 2026-10-24T08:39:41.639Z --- # AI Agent Observability With Ivana Huckova and Sven Großmann Grafana's Ivana Huckova and Sven Großmann discuss AI agent observability, coding-agent costs, PR analytics, secret redaction, and local guardrails. _Guests: Ivana Huckova, Staff Software Engineer @ Grafana Labs; Sven Großmann, Principal Software Engineer @ Grafana Labs · Host: Łukasz Chludziński, Engineering Manager @ Callstack · Published 2026-10-12 · Duration 12:57 · Category: Observability_ Listen: [YouTube](https://youtu.be/lnpRpmedfZU) Tracking what a coding agent costs raises another question: what work did those tokens produce? Ivana Huckova and Sven Großmann from Grafana Labs join us in the studio at Agent Conf 2026 in Warsaw to discuss AI agent observability and the signals they use to inspect agent work. We talk about connecting coding sessions to pull requests, watching costs across tools, and using guardrails to redact secrets. The conversation also covers Grafana Assistant's dashboard generation and an open-source local app for observing coding agents. #### Building dashboards with Grafana Assistant The guests explain how Grafana Assistant can generate a dashboard from a prompt, using a weather dashboard as an example. We discuss how the assistant incorporates dashboarding practices, including panels that explain the data and give viewers context. The guests also describe how Grafana's long history as an open-source project gives models examples of how to use it. #### Guardrails, model-based checks, and secret redaction The conversation follows up on a security demo from the guests' conference talk. They explain how they use Agent Observability to monitor Grafana Assistant and apply guardrails. We discuss using an LLM as a judge, configuring its instructions, and testing changes to those instructions. Ivana adds another use for guardrails: redacting API keys and other secrets before they are sent to tools. #### Connecting coding sessions to pull requests and production Ivana describes analytics that show which agent-driven pull requests were merged, closed, or remain open. Those outcomes give developers a signal about how much of their agent work reaches the main branch. She is explicit that this is not a complete measure of productivity. We also discuss the possibility of following a pull request into production and correlating the change with product metrics that improve or regress. #### Agent costs and the incentives around token usage Agent governance includes visibility into spending. The guests discuss agents running up large bills and the different questions developers and managers ask about their usage. We also discuss how treating token consumption as an engineering performance target can create the wrong incentives. Engineers can run agents in the background to consume tokens without producing corresponding value. The conversation connects cost tracking with questions about the work those sessions produce. #### Local observability and choosing what to share Ivana describes an open-source local app where individual developers can inspect coding sessions and token usage. Teams can choose to connect to Grafana Cloud and send metadata or additional information, depending on what they want to share. The guests explain that the audience includes individual developers, small teams, and larger organizations. A developer may want to inspect their workflow or skills, while managers may want one view of spending across several coding-agent tools. #### Comparing models and using guardrails locally We ask whether the product can recommend a cheaper model for the same task and output. The guests describe running evaluations with different models as a way to compare results. They also make clear that recommending a cheaper model for equivalent output is not fully there at the time of the interview. Ivana closes the technical discussion by recommending local protections too. She discusses sandboxes and guardrails for restricting access to files and secrets, and says guardrails are part of the local app. #### About the guests Ivana Huckova is a Staff Software Engineer at Grafana Labs. Sven Großmann is a Principal Software Engineer at Grafana Labs. Both work on the AI team. At Agent Conf 2026, they presented "[When Vibes Aren't Enough: Observability for AI Agents](https://youtu.be/XvgprsJDo3o?si=mK-IR-5gXNapdVjN)." This studio interview follows up on agent observability, guardrails, and measuring coding-agent work. #### Resources - [Grafana Assistant documentation](https://grafana.com/docs/grafana-cloud/platform/grafana-assistant/) - [Grafana Agent Observability](https://grafana.com/products/cloud/agent-observability/) - [Grafana Agent Observability SDK and coding-agent plugins](https://github.com/grafana/agento11y) - [Ivana Huckova on X](https://x.com/ivanahuckova) - [Sven Großmann on LinkedIn](https://www.linkedin.com/in/sven-gro%C3%9Fmann-26387311b/) - [Agent Conf](https://www.agent.sh/) Insights ## Learn more about Observability Here's everything we published recently on this topic. [View all](https://www.callstack.com/insights) - [MCP and Supervisor Agents in eBay's Monitoring Workflow With Leonel Candido da Silva](https://www.callstack.com/podcasts/mcp-and-supervisor-agents-in-ebays-monitoring-workflow-with-leonel-candido-da-silva) (Oct 13 · Podcast): Leonel Candido da Silva explains eBay's agentic monitoring workflow: specialized agents, MCP, PagerDuty reports, and plans to learn from past incidents. - [Observability and OpenTelemetry in React Native](https://www.callstack.com/podcasts/observability-opentelemetry-in-react-native) (Aug 13 · Podcast): Learn how Embrace and OpenTelemetry can help React Native teams standardize observability and improve app monitoring across platforms. - [Observability for Application Deployments](https://www.callstack.com/events/observability-for-application-deployments) (Jun 24 · Talk): Discover practical observability techniques to pinpoint and fix slow deployments, optimize global performance, and deliver faster apps. - [Elevate Your React Native Observability With OpenTelemetry](https://www.callstack.com/events/elevate-your-react-native-observability-with-opentelemetry) (Jun 24 · Talk): Learn how to leverage OpenTelemetry to gain deeper, real-time insights into your React Native apps and significantly improve user experience. - [Consistently Improve App Performance With DMAIC and Reassure](https://www.callstack.com/blog/consistently-improve-app-performance-with-dmaic-and-reassure) (Mar 11 · Article): Learn how to keep your React Native app performant as it grows by following DMAIC process and monitoring performance regression with tools like Reassure.