For AI systems, reliability is defined by response quality, factual grounding, data security, and usability — and those signals need to live alongside the same observability data teams already trust to monitor performance and availability.

When evaluation scores are isolated in notebooks, spreadsheets, standalone tools, or CI logs, they’re hard to operationalize. By bringing AI quality metrics into Dynatrace AI Observability—next to latency, cost, errors, traces, and user behavior—teams can connect poor responses and hallucinations directly to the prompts, models, retrieval contexts, tool calls, services, and traces that produced them.

What is dt-evals?

dt-evals is an open source CLI for evaluating LLM and agent quality from real GenAI traces, agentic interactions. Teams can run online evaluations against live or recent interactions, score outputs with an LLM judge, and send structured results back to Dynatrace AI Observability so quality becomes visible, queryable, trendable, and actionable.

A minor prompt edit, model change, or retrieval update to an AI application can improve one behavior while quietly breaking another. The challenge to tracking down where and why these systems break is that evaluation results are often maintained outside the operational workflow, making it difficult to connect a low score to the exact trace, prompt, model version, retrieval context, tool call, or service that produced the unwanted behavior.

Dynatrace AI Observability closes this loop. With dt-evals and the AI Observability Evaluation Preview teams can pull recent gen_ai.*  spans, score real interactions with an LLM judge, and write structured evaluation results back as business events. These scores can be viewed with the originating trace, queried for custom analysis, trended in dashboards, and used to trigger alerts or workflow-driven remediation.

A failing faithfulness score is no longer just a number in a report. It’s now an operational signal.

What are LLM evaluations?

An LLM evaluation system scores an AI response against a range of quality and safety dimensions. Common examples include whether the answer is relevant to the question, faithful to the provided context, free of hallucinations, safe for users, complete enough to be useful, and resistant to prompt-injection attempts.

LLM evaluations are typically applied in two modes:

Offline evaluations run before release against a fixed test set or curated trace dataset. These are used to compare a proposed prompt, model, retriever, or agent-tool change against a known baseline before shipping. For example, replay 500 representative support questions in CI and block the release if faithfulness drops below the configured threshold.

Online evaluations run after deployment against sampled production or user traffic. Use online evaluations to detect regressions caused by live inputs, changing retrieval results, tool behavior, traffic mix, or model drift. For example, evaluate 10% of support-agent traces from the last hour and alert the team if hallucination failures exceed the configured window.

With dt-evals, you can run evaluations from the command line, use them in CI/CD, or schedule them to detect quality regressions autonomously after deployment as a post-processing quality gate for your AI agents and LLM output.

Run evaluations from the command line

dt-evals is an open source evaluation toolkit for teams that want to bring their own data, judge provider, and evaluation logic while keeping traces, scores, dashboards, and alerts connected.

Install the CLI:

npm install -g @dynatrace-oss/dt-evals

Or run it directly with npx:

npx @dynatrace-oss/dt-evals <command>

A typical first run has three steps:

1. Configure your environment and judge provider (Bring Your Own AI API key):

dt-evals configure

2. Verify your local setup and connection:

dt-evals doctor

3. Run evaluations on recent GenAI traces:

dt-evals run --since 1h --sample 10

This command evaluates traces from the last hour and samples 10% of them. In other words, dt-evals evaluates roughly one out of every ten matching traces, including the prompt and completion messages associated with each selected trace.

During configuration, you provide the connection to your Dynatrace environment and the credentials for the LLM judge provider you want to use. dt-evals does not require teams to send evaluations through a fixed provider. You bring your own judge credentials and control where evaluation execution happens.

For CI/CD use cases, run in CI mode:

dt-evals run --since 6h –ci

In CI mode, dt-evals emits machine-readable output and can fail the pipeline when a configured threshold is breached. This makes quality checks part of the same delivery process used for prompt changes, model upgrades, retrieval updates, and agent releases.

Bring your own LLM judge provider

The “LLM-as-judge” evaluation approach involves using an AI model to score another model or agent response. The LLM judge needs to come from an AI provider your team trusts and has approved for the type of data being evaluated.

dt-evals supports common LLM judge AI models and inference providers, including OpenAI, Anthropic, Google/Vertex/Gemini, AWS Bedrock, and Azure OpenAI. Depending on the package and configuration path you use. Teams provide their own credentials, choose the judge model, and can tune execution settings such as thresholds and concurrency.

This matters for both governance and cost control. Teams can decide which LLM provider is assigned to evaluate which traffic, how many judge calls run in parallel, and where evaluation results are stored.

Which quality and safety dimensions are evaluated by dt-evals?

dt-evals supports built-in LLM-as-Judge evaluators for a range of quality and safety dimensions, including:

• Relevance: Does the response answer the user’s question?
• Faithfulness: Is the response supported by the provided context?
• Hallucination: Does the response invent facts that are not present in the available context?
• Answer completeness: Does the response fully address the user’s request?
• Context relevance: Is the retrieved or supplied context useful for answering the question?
• Factual accuracy: Does the response match an expected or known-correct answer?
• Summarization quality: Does the summary preserve the important information?
• Conciseness: Is the response direct with no unnecessary detail?
• Fluency: Is the response clear and readable?
• Toxicity: Does the response contain harmful or abusive content?
• Bias: Does the response show unfair or inappropriate bias?
• PII leakage: Does the response expose sensitive personal information?
• Prompt injection: Did the input or response show signs of instruction manipulation?
• User frustration: Does the interaction suggest the user is blocked or dissatisfied?
• Drift: Are scores changing meaningfully compared with prior behavior?

A Retrieval Augmented Generation (RAG) application might focus on faithfulness, hallucination, context relevance, and answer completeness. A customer-facing support agent might focus on relevance, fluency, bias, toxicity, and prompt-injection risk. An internal assistant might add custom checks for tone, policy compliance, or whether the answer includes required next steps.

Add custom evaluations

Built-in metrics are useful, but most production AI systems also need checks that are specific to the business, domain, or workflow.

Custom evaluations let teams define their own judge prompts, scoring rules, labels, and thresholds. For example, a support team can create a custom evaluator that checks whether an answer includes a required troubleshooting step before recommending escalation. A financial services team can check whether responses include the required disclaimers. A platform team can check whether an agent uses the correct tool before answering.

The critical point is that custom evaluators run through the same pipeline as built-in evaluators. They can produce the same structured results, appear alongside other scores, and be used in dashboards, alerts, and release checks.

A typical configuration defines the target service, judge provider, sampling strategy, enabled metrics, and thresholds:

schemaVersion: 1
name: support-agent-prod

dynatrace:
  environmentUrl: https://your-env.apps.dynatrace.com
  platformToken: dt0s16.xxxxx

judge:
  provider: openai
  model: gpt-5.5

scope:
  service: support-agent
  since: 1h
  sampling:
    strategy: random
    percent: 10

metrics:
  enabled:
    - faithfulness
    - hallucination
    - relevance
    - drift

alerts:
  thresholds:
    faithfulness: 0.7
    relevance: 0.7

Evaluation results in the AI Observability app

Evaluation results appear directly in the AI Observability app, so teams don’t have to jump between a trace view, an eval report, and a separate dashboard to understand what happened.

In the Prompts view, teams can filter for prompts with evaluation scores and inspect row-level verdicts. Score badges such as relevance, fluency, bias, faithfulness, or toxicity make response quality easy to scan without opening every trace.

This is useful when triaging a regression. Instead of starting with a generic failure count, teams can quickly see which prompts failed, which evaluator failed them, and whether the issue is isolated or widespread.

From an individual prompt or trace, the Evaluations tab shows run-level details, including the evaluation name, score, provider, judge model, method, and supporting metadata.

That detail matters because a failed score is only useful if teams can explain it. Engineers and evaluation owners can move from a low score to the exact prompt, response, trace, model, evaluator, and rationale that produced it.

Query, trend, and alert on evaluation scores

Because dt-evals writes results back as structured events, evaluation scores can be analyzed with the rest of your telemetry.

Teams can ask questions such as:

• Which evaluator has the lowest average score?
• Which services are producing the most failed evaluations?
• Did quality drop after a model or prompt change?
• Are hallucinations increasing over time?
• Is quality improving at the cost of latency or token usage?

For example, to get average score by evaluator you could write this query:

fetch bizevents
| filter event.type == "gen_ai.evaluation.result"
| summarize avg_score = avg(gen_ai.evaluation.score.value),
    by: { gen_ai.evaluation.name }
| sort avg_score asc 

Failed evaluations by service and metric can be determined with this query:

fetch bizevents
| filter event.type == "gen_ai.evaluation.result"
| filter gen_ai.evaluation.score.label == "fail"
| summarize failures = count(),
    by: { dt.service.name, gen_ai.evaluation.name }
| sort failures desc 

Trending is where evaluation data becomes more useful than a point-in-time report. A single failed score can show an issue. A trend can show whether quality is drifting slowly, whether a release caused a sudden drop, or whether a fix actually improved behavior over time.

On the AI Evaluation & LLM App Performance dashboard, teams can track trends in quality score, pass rate, failed evaluations, drift detections, evaluator health, run cadence, and pass/fail volume over time.

How to turn quality regressions into alerts

Evaluation results can also drive alerts. For example, a support agent team may want to notify the AI team when hallucinations appear in production, or when faithfulness drops for more than a few minutes.

name: support-agent-prod

alerts:
  notifications:
    - name: hallucination-detected
      metric: hallucination
      condition: count > 0
      window: 5m
      channel:
        type: slack
        connection: ai-observability-slack
        channel: "#ai-alerts"

    - name: faithfulness-regression
      metric: faithfulness
      condition: fail_rate > 10%
      window: 15m
      channel:
        type: email
        connection: ai-team-email
        to: [ai-team@example.com] 

Deploy the alerts with:

dt-evals alerts list ./support-agent-prod.yaml
dt-evals alerts apply ./support-agent-prod.yaml

Once applied, Dynatrace runs these checks continuously as Workflows. If hallucinations appear in the last five minutes, the team gets a Slack alert. If more than 10% of faithfulness checks fail over 15 minutes, the AI team receives an email. This turns LLM quality from something teams inspect manually into something Dynatrace can monitor and route automatically.

For continuous alerting, evaluation runs need to happen continuously or on a schedule. You can run dt-evals in CI for release checks (see our example here), run it manually during investigation, or deploy a scheduled runner for ongoing production evaluation. An alert is only as fresh as the evaluation results it carries.

Once configured, quality signals can be routed to the teams that need to act. If hallucinations appear in the last five minutes, the team can receive a Slack alert. If more than 10% of faithfulness checks fail over 15 minutes, the AI team can receive an email. This turns LLM quality from something teams inspect manually into something they can monitor and route automatically.

Close the loop in the AI software delivery lifecycle

Evaluation gates are most useful when they meet developers where they already work. Because dt-evals writes evaluation results back into the observability data layer, those results are not limited to dashboards or post-release reviews. They can be queried, inspected, and acted on from development workflows, CI/CD pipelines, and agentic coding environments.

For example, a team can run dt-evals after any change to a prompt, model, retriever, or agent tool, and then use dtctl (Dynatrace CLI tool for AI Agents) to query the resulting evaluation data, inspect related traces, review dashboards, or validate whether a release threshold was met. In an AI-assisted workflow, tools such as Claude Code, Cursor, GitHub Copilot, or an internal agent harness can leverage MCP or CLI access to bring that same observability context into the developer’s daily workflow.

That closes the loop of the AI software delivery lifecycle: teams can evaluate behavior, control rollout decisions, remediate regressions, and feed production learning back into the next development cycle. Quality signals are no longer in a separate report; they’ve become a part of how AI software is built, shipped, and operated.

Bring evaluations into the release process

Evaluation support is not just for inspection after something breaks. It can also help prevent regressions before they reach users.

This makes AI quality part of the release process. Teams can gate changes based on relevance, faithfulness, hallucination risk, prompt-injection risk, toxicity, or custom metrics, rather than relying solely on latency and error rate.

What makes this meaningful is that it’s the same pipeline teams already run. AI quality gates sit alongside the latency, error rate, and SLO gates teams have been using for years. There’s no second CI system, no second platform to learn, no second dashboard to monitor. Quality becomes one more dimension of the release decision, gated the same way performance is gated, by the same platform, in the same pipeline.

Coming next

The current experience makes evaluation results visible and actionable inside Dynatrace AI Observability. Next, the focus is on making evaluation workflows easier to run at scale and easier to compare across changes.

Planned improvements include targeted and bulk trace evaluations, custom evaluation libraries, evaluator versioning and lineage, baseline comparisons, experiment views, native quality gates, and deeper visibility into online evaluations.

These capabilities will help teams compare prompt and model variants, understand quality versus cost and latency tradeoffs, and detect sustained quality regressions before they affect more users.

Start today

To get started, check out the Git repository. You’ll need:

• Node.js 20 or later
• A Dynatrace environment with GenAI spans and the AI Observability app installed

• Credentials for the judge provider you want to use
• A service, trace sample, or CI workflow you want to evaluate

Then, install the CLI:

npm install -g @dynatrace-oss/dt-evals

Configure your service and judge provider:

dt-evals configure

Run your first evaluation:

dt-evals run --since 1h --sample 10

With Dynatrace AI Observability and dt-evals, teams can bring LLM and agent evaluations into the operational loop, where they can trace behavior, score outputs, trend results, alert on regressions, and gate releases before silent failures reach production.

The post Evaluate LLM and agent quality in Dynatrace AI Observability with dt-evals appeared first on Dynatrace news.

As more agentic AI projects come online, the observability market is abuzz with familiar promises: tool consolidation, AI-powered insights, and faster remediation through smarter tools. On the surface, this sounds like progress. But beneath the excitement, many discussions are framed around the wrong question.

The real issue isn’t about how to adopt autonomous operations; it’s about ensuring AI agents are operating reliably and resolving problems without introducing new ones. When evaluating new observability solutions, the question should be:

Can this observability solution accurately analyze complex, dynamic telemetry in context so AI agents can act autonomously with trust, precision, and reliability?

As systems become increasingly agent driven, observability is crossing a structural boundary. Approaches designed for environments where only humans decide and act must adapt to a world where agents increasingly operate autonomously with human oversight, while keeping organizations informed.

Rethinking observability for the agentic age

Observability platforms were initially intended to support engineers in delivering reliable applications, services, and infrastructure to users, and alert them in the event of a problem. Dashboards, alerts, and correlation helped teams investigate incidents, piece together what happened, diagnose issues, decide on next steps, and resolve the problem. This model worked when changes were pushed manually.

The assumption was that more data, better correlation, and cleaner interfaces will lead to increased visibility and improved operational decision making.

Agentic AI systems break that assumption.

With faster release cycles and AI-generated code, manual investigations can no longer keep pace. Moreover, observability platforms must now provide actionable insights to both humans and AI agents.

As agents begin operating as autonomous participants in software environments by triggering mitigations, scaling infrastructure, and optimizing behavior in real time, observability can no longer function solely as a human interface. It must also provide AI systems with a reliable, contextual fact basis that agents can act on programmatically. Machines can’t rely on dashboards and alerts. They require a deterministic foundation of unified, real-time data that delivers accurate, context-rich answers at exabyte scale.

Agentic systems break the mold of “good enough”

Many observability platforms layer probabilistic AI on top of siloed data. They use LLMs to correlate signals and rank likely causes—but they can’t always determine correctness.

“Probabilistic” means that the same input will generate a different output based on a probability distribution of predefined outputs, delivering a different answer when the same problem occurs. This approach is also prone to hallucinations, requiring additional human validation, which can increase operational overhead and token costs, delay resolution of business-critical issues, and divert resources from strategic initiatives.

Enterprise-grade observability must now answer: Is this insight reliable enough for autonomous action?

AI built on siloed data is inherently unreliable. Autonomous systems depend on deterministic, contextual, and trustworthy data to act reliably.

“Deterministic” means that the same input always results in the same output by using factual data to trace the exact causal changes that created the issue. When agentic AI systems act on business-critical applications, the cost of being “mostly right” becomes operationally unacceptable.

This is where a subtle but critical divide appears in the market. Aggregating signals and correlating anomalies can surface patterns. Patterns alone are not a solid basis for decisions, and without deterministic understanding, AI systems inherit that uncertainty and can propagate it downstream.

To drive reliable enterprise autonomous operations, AI agents require a unified, AI-powered observability platform that can analyze exabytes of data in real time and across models to pinpoint root cause, delivering actionable answers in context of what’s affected and its business impact.

From correlated guesses to deterministic answers

This shift in the demands of observability hinges on a clear distinction:

• Probabilistic AI correlates signals that happened around the same time and therefore appear related, pulling information from fragmented data stores to propose a likely root cause.
• Deterministic AI uses causal analysis to pinpoint what happened and why, recommend remediation actions, and identify business impact.

Probabilistic AI is intended to narrow the search space and direct engineers toward potential resolution, but it still requires interpretation.

Deterministic AI establishes sequence, dependency, and impact, enabling systems to decide safely without waiting for humans to connect the dots.

Auto‑remediation, auto-prevention, and auto-optimization all depend on this leap. A platform that unifies telemetry only at the UI layer may deliver data and potential root cause, but it can’t compensate for fragmented understanding and missing context underneath. When context is pieced together after the fact, confidence is never guaranteed.

You can’t automate what you don’t precisely understand.

Context driven observability as the control plane for AI

In an autonomous enterprise, observability doesn’t sit beside execution; it’s embedded within it. This integration requires that teams adopt a new mindset toward observability architecture.

Because more AI workloads are happening at the source, telemetry must be optimized and streamlined before ingest, not after the fact, from the edge to the back end. Data access must be unified, context-aware, and always-hydrated on a massive scale. Answers must be explicit, not implicit, and they must be informed by automatic, real-time dependency mapping.

Likewise, intelligence must combine deterministic and agentic AI—not as add‑ons, but as a single reasoning system from ingest to execution.

In this model:

• AI agents can become the primary consumers of observability data.
• Humans can shift toward strategy, architecture, oversight, and exception handling.
• Observability evolves from a reactive lens into a control plane for autonomous operations.

Observability purpose-built for autonomous operations ensures successful agentic AI initiatives

This moment represents an architectural transition, not just an incremental upgrade cycle. Correlation-dependent observability that uses probabilistic AI can be extended, augmented, and rebranded, but it will always carry the limitations of approximation and human validation.

The next era belongs to an observability platform that’s built for machine understanding from the start: a unified, context driven architecture that delivers deterministic answers at machine speed, precision, and scale.

The post Beyond correlation to autonomous action: Why “good enough” observability fails in the age of agentic AI appeared first on Dynatrace news.

Imagine a team of AI agents building, deploying, and running software at machine speed—yet unable to see what’s happening in production. This is the new reality for enterprise technology leaders. As one Fortune 500 CTO told us, “Speed is now the primary driver of innovation, forcing organizations to rethink processes, compliance, and roles; it’s a necessity for innovation teams.”

Observability—real-time visibility into how software behaves in production—has become the critical enabler for both human-led and agent-led teams. Without it, AI agents are powerful but blind.

Key executive insights

1. Software production is being redefined by AI agents. This transformation is a structural shift, not a trend.
2. The world is bimodal again. Human-led and agent-led environments coexist.
3. AI agents are powerful but blind. Without rich context from production, they cannot deliver reliably.
4. Observability is a crucial enabler to gradually transform from human-led to agent-led operations. Observability is what allows organizations to industrialize software delivery with confidence.
5. The new KPI for agent-led teams is the percentage of human intervention required. The lower the number, the better the AI is working.

The market reality: A bimodal world

Organizations are accelerating AI adoption not because it is trendy, but because it is existential. Companies that fail to transform risk being outpaced by competitors that can deliver software faster, cheaper, and at higher quality. CTOs and CIOs are making statements like “speed over compliance” not out of recklessness, but because they recognize that without radical acceleration, their businesses face disruption.

At the frontier of this shift is a fundamentally new way of building software: AI-first development. In these environments, 100% of coding, testing, deployment, operations, bug fixing, and optimization are performed by AI agents. The human role shifts to specification, goal setting, supervision, and correction. Intellectual property moves from the code to the specification—code becomes a generated artifact, not the source of truth. With a complete, well-architected spec, agents can fully rebuild the software from it again.

This creates a bimodal operating environment:

• Human-led teams—the majority today—are existing operations, SREs, and developers augmenting their workflows with AI. They follow the traditional SDLC, increasingly supported by AI agents that auto-prevent, auto-remediate, and auto-optimize, which reduces manual effort and achieves more with the same resources.
• Agent-led teams—growing fast—are innovation groups operating in full AI development life cycle (AIDLC) mode. Swarms of AI agents build, deploy, and run software end-to-end. Humans write specifications and intent, not code. For these teams, the KPI is no longer “how many story points were solved?” but “what percentage of human intervention is required?”

Observability enables a reliable transition to autonomous operations

In the early 2010s, a similar bimodal pattern emerged with cloud: one team running thousands of servers on-premises, another in stealth mode on AWS. The pattern is repeating now with AI.

Why not switch everything to agent-led right away? Because existing systems follow processes, compliance, and technology stacks that can’t be immediately automated in an AI-first way. Moreover, it’s too risky to move all business-critical systems simultaneously. The safer path: start with an innovation team, build less critical applications first, and only when those are successful and trusted, begin migrating more of the business-critical services.

New foundation models that arrived in early 2026 have accelerated the path to fully autonomous operations, making agent-led teams realistic at small scale today, with large scale within sight. These systems focus on AI-first software generation first, with a clear goal to eventually master operational challenges (resilience, performance, scale, security) entirely with agents as well.

Observability plays a critical role not only in making both modes work reliably, but also in enabling the transformation from the first mode to the second. The context observability provides—understanding existing system behavior, dependencies, and requirements—is exactly what agents need to create the reliable and scalable software. Observability is what makes both modes work, and it is the critical bridge between them.

The core problem: AI agents are blind

AI agents can code, deploy, refactor, and operate software faster than humans ever could. But there is one thing AI cannot do without help: AI has no awareness of what happens in production. It’s blind to the real world: without real-time feedback from running software—in development and production —agents make decisions without context and without understanding their consequences. They operate at speed, but without sight.

77% of IT teams still lack full visibility across hybrid environments (IBM Institute for Business Value, 2025). If you can’t see it, you can’t scale it. Observability is not optional for AI-first operations, it’s a prerequisite.

The Dynatrace response: Real-time observability for both worlds

Dynatrace addresses both sides of this bimodal reality: a complementary response to the two speeds at which enterprises now operate.

For human-led teams: Autonomous operations at scale

This year, Dynatrace launched Dynatrace Intelligence: a full agentic operations system that orchestrates dozens of agents that auto-prevent, auto-remediate, and auto-optimize across site reliability, development, and application security. These AI agents deliver the following value in production:

• SRE Agent: Kubernetes troubleshooting, infrastructure optimization, and automated incident resolution – reducing mean time to resolution at scale.
• Developer Agent: Surfaces production context during deployment, validates changes, and prevents issues before they reach customers.
• Security Agent: Identifies vulnerabilities, triages threats, and accelerates security response, all in real time.

The deterministic foundation underneath: what separates Dynatrace agents from others is its deterministic foundation: real-time, full-stack, and cross-model root-cause analysis, anomaly detection, and forecasting, all grounded by data in a unified, purpose-built data lakehouse that delivers accurate, contextual answers from exabytes of information. This is not AI that guesses; it’s AI that reasons from facts. Benchmarks from internal testing and observed customer use cases: 12× higher success rate in SRE use cases, 3× faster problem resolution, 2.5× lower token cost.

Ecosystem integrations that extend intelligence beyond the platform: Dynatrace Intelligence extends into third-party tools to drive autonomous actions across development, SRE, and ITOps workflows.

For agent-led teams: develop and run software reliably

Dynatrace enables AI-first teams to let swarms of agents to build and run software reliably, providing real-world awareness from observability, run-time context across development, security, and operations, and self-optimization toward SLAs, cost, and resilience. Key capabilities include:

• Agentic observability: Closed-loop autonomous operations where observability agents coordinate with coding and deployment agents to self-heal.
• AI and cloud observability: Full-stack visibility across cloud infrastructure and AI workloads, covering resilience, performance, security, user experience, and LLM evaluations to assess the quality and reliability of agent outputs, helping identify potential inaccuracies, hallucinations, or risks.
• AI data lakehouse (Grail): Real-time context engine that provides long-term memory for agent decisions—sub-second, API-native, at an exabyte scale.

The goal: a closed loop where agents detect issues, resolve them, and ship the fix—autonomously, 24/7.

Dynatrace is on the same bimodal journey – our entire business runs on Dynatrace Intelligence in human-led mode, with agents taking over more tasks continuously, while our AI-first offering and new services are built and operated entirely by agent swarms, using our own observability to close the feedback loop.

Different approaches – unified platform

Across the platform, Dynatrace delivers end-to-end, full-stack visibility across cloud infrastructure, applications, and AI workloads, including agent behavior, decision paths, and cost, along with governance at machine scale. These capabilities serve human-led and agent-led teams differently, but from the same unified platform.

The measure of success in software delivery is shifting from human productivity metrics to a new KPI: the percentage of human intervention required. Observability is what makes that progress possible. The question for every technology leader is no longer whether to adopt AI-first, but how quickly they can close the visibility gap before competitors do to drive massive growth in innovation and productivity.

The post AI agents are redefining software development—but they’re flying blind without observability appeared first on Dynatrace news.

As agentic AI becomes mission-critical, systems that reason, act, and self-optimize introduce new operational challenges. Their dynamic and non-deterministic behavior makes them difficult to debug, they can drive unexpected cost spikes, and they inherently lack the auditability required for reliable, enterprise-grade use. Today, we’re excited to announce expanded support for leading agentic frameworks and protocols, along with a new dedicated AI Observability app. With this support, you can build, run, and debug agentic AI applications with confidence across AWS, Azure, and Google Cloud.

What’s new: Broader agentic technology support

Dynatrace supports a broad and rapidly growing ecosystem of agentic AI frameworks and protocols, unifying telemetry from these frameworks via OpenTelemetry and OpenLLMetry into a single, correlated observability model, delivering end‑to‑end visibility across clouds, models, tools, and agents from one platform.

• Amazon Bedrock AgentCore – Dynatrace offers observability for Amazon Bedrock AgentCore agents by collecting metrics such as token usage, model behavior, latency, and errors. This integration provides unified tracing, cost, performance, and guardrail monitoring, along with ready-made dashboards and intelligent anomaly detection and forecasting, helping teams quickly and effectively monitor, troubleshoot, and optimize complex autonomous agent workflows.
• Amazon Bedrock Strands – Dynatrace supports the Amazon Bedrock Strands Agents SDK, enabling comprehensive visibility into agentic AI systems. By instrumenting Strands-based AI agents with Dynatrace, organizations can monitor agent behavior, tool usage, and dependencies end to end. This helps ensure performance, reliability, and operational insight across distributed environments, supporting the confident development and operation of agentic AI use cases such as chatbots, recommendation systems, and autonomous workflows.
• LangChain Agents – Dynatrace provides observability for applications built with the LangChain framework, enabling the monitoring of performance, cost, and reliability of Large Language Model (LLM) applications and agents.
• Google Agent Development Kit (ADK) – Dynatrace provides observability for applications built with the Google Agent Development Kit (ADK), enabling visibility into agent execution, dependencies, and performance. This helps teams understand runtime behavior and maintain reliability as agent-based applications
• OpenAI Agents SDK – Dynatrace provides observability for observing applications built with the OpenAI Agents SDK, enabling monitoring of agent workflows, model interactions, latency, and errors. This supports improved operational insight, troubleshooting, and performance optimization for agentic AI applications.
• MCP AI Agent–  Dynatrace provides deep visibility into AI agents communicating via the Model Context Protocol (MCP). By observing both AI agents and MCP servers, organizations gain end-to-end insight into execution flows through tracing, enabling data-driven decisions, performance and cost optimization, and governance for complex agent workflows.

This agentic coverage is on top of the 40+ LLM technologies that Dynatrace already supports, including OpenAI, Amazon Bedrock, Google Gemini and Vertex, Anthropic, LangChain, NVIDIA, and more.

We’re working closely across AWS, Microsoft Azure, and Google Cloud ecosystems to ensure you have consistent, enterprise‑grade observability for your multi‑AI and multi‑cloud applications.

See it in action in the new AI Observability experience

The AI Observability app is now Generally Available, delivering a purpose-built experience for observing AI workloads end-to-end from agents and LLMs to orchestration layers, emerging protocols, and tools. It gives engineering teams deep, production-ready visibility into how AI systems behave in real time, allowing them to validate changes faster, reduce risk, and confidently ship AI-powered features at scale.

Unlike generic observability views, the AI Observability app is designed specifically for agentic and LLM-driven systems, making it easy to understand complex multi-step interactions, reason about cost and performance trade-offs, and troubleshoot issues across models, tools, and dependencies.

Key capabilities

• End‑to‑end observability for agentic AI
  • Monitor agent interactions, tool usage, dependencies, latency, and reliability
  • Track token consumption, cost trends, and caching impact
• Tracing and debugging for complex flows
  • Follow prompts, tool calls, and model invocations from the initial request to the final response
  • Jump from high‑level health to prompt‑level traces in a couple of clicks
• Actionable insights at scale
  • Rapid A/B testing across model and prompt variants for faster validation
  • Identify bottlenecks and optimize resource utilization with ready‑made dashboards and drill‑downs
• Security, privacy, and governance
  • Enterprise‑grade controls, auditability, and policy‑aligned routing
  • Guardrail outcomes (for example, toxicity, PII, or denied topics) are surfaced so you can monitor behavior and trends. (Note that guardrail enforcement occurs at the model/provider; Dynatrace captures and visualizes provider‑reported outcomes.)

Who this solution is for and why it matters

The Dynatrace AI Observability solution is for enterprise teams, including developers, DevOps, SREs, and business leaders who need deep, real-time insights into their cloud native  AI-powered applications and customer experience in a single unified view.

Who benefits the most from this solution?

• AI Engineering and Data Science: This group includes practitioners who develop and optimize models. They use LLM observability to track metrics related to model performance, such as identifying hallucinations and biases, validating changes, and improving prompt engineering practices.
• Software Developers: These individuals benefit from observability by gaining insights into application-level performance, which helps them debug and improve overall code quality. Observability tools allow for faster iteration in development cycles.
• Site Reliability Engineers (SRE): These teams ensure the reliability and performance of AI applications in production environments. They use observability to identify system-level bottlenecks and failures, and to respond swiftly to operational challenges.
• Application Security Teams: Although not traditionally the primary users, security teams can leverage AI observability to identify and mitigate emerging threats specific to AI applications, such as prompt-injection attacks and data leaks.
• Compliance and Governance Teams: Responsible for ensuring adherence to regulatory requirements and internal policies, these teams rely on observability to audit model behavior and to identify potential biases or harmful outputs.

What’s next: Agent topology view with Smartscape

We’re committed to further enhancing these capabilities. As agentic systems evolve into distributed networks of models, tools, and decisions, observability must move beyond traces and metrics. Our next focus is the Agentic Topology View, bringing Smartscape-grade visualization to agent execution flows so teams can see how agents interact, invoke tools, propagate errors, and improve performance end to end.

This agentic topology becomes the foundation for a deeper developer experience by connecting production telemetry with prompt management and evaluation workflows. By unifying agent topology, prompt lifecycle, and LLM-as-judge scoring in a single system, we’re helping teams systematically improve the reliability, performance, and quality of agentic AI at enterprise scale.

Get started today

• Install the AI Observability app from Dynatrace Hub
• Instrument your agents via OpenTelemetry/OpenLLMetry
• Start tracing prompts, tool calls, and model invocations—and turn insights into action

The post Announcing agentic framework support and General Availability of the Dynatrace AI Observability app appeared first on Dynatrace news.

Amazon Bedrock provides enterprises with fully managed access to leading foundation models through a single API, eliminating the complexity of managing underlying AI infrastructure. This simplicity accelerates innovation but also prompts enterprises to consider how best to govern and secure access to Amazon Bedrock as they’re using it at scale.

Without a secure AI gateway in place, organizations can quickly face challenges such as:

• Uncontrolled access and data exposure: Without integrated authentication and authorization, anyone with credentials can invoke models or send sensitive data without oversight.
• Compliance and audit gaps: Without consistent tracking and isolation, it’s difficult to demonstrate adherence to internal policies or regulatory requirements.
• Operational fragility: Developers must manage credentials and request signing manually, adding complexity and security risk.

These are the same challenges Dynatrace encountered while scaling its own generative AI workloads. In response, our engineering teams developed a secure AI gateway for Amazon Bedrock, which has proven effective in serving our global user base. We’re now sharing a reusable reference architecture for the AI gateway in close collaboration with our partners at AWS.

Enterprise-grade governance for real-world use cases

The Secure AI Gateway extends Amazon Bedrock with enterprise-grade governance and control. Built on Amazon API Gateway, the solution integrates seamlessly into existing enterprise environments and provides:

• Strong authentication and authorization through integration with corporate identity systems.
• Usage quotas and throttling to manage cost and ensure fair resource distribution.
• Multi-tenant support and tenant isolation with detailed usage tracking for security, auditability, and compliance.
• Zero-code compatibility with Bedrock features: Once the AI Gateway is deployed, all existing Bedrock capabilities remain available without any integration code changes.

Proven within Dynatrace’s own platform, this reference pattern provides enterprises with a practical path to securely operationalize Bedrock, maintaining the speed and flexibility developers expect while introducing the control and transparency that enterprise governance demands.

Find all the details and the full technical walkthrough here: AWS: Building a Secure AI Gateway to Amazon Bedrock.

AI Observability for continuous insights after deployment

Securing access is only the first step; ensuring everything continues to work as intended is the next. With Dynatrace observability for Bedrock-based workloads, your teams gain continuous insight into performance, reliability, and cost, verifying that governance controls remain effective and that AI workloads perform as expected.

You can read more about our solution here: Deliver secure, safe, and trustworthy GenAI applications with Amazon Bedrock and Dynatrace.

The post AWS publishes Dynatrace-developed blueprint for secure Amazon Bedrock access at scale appeared first on Dynatrace news.

Teams can transition from setup to insights in minutes using a lightweight OTLP configuration and ready-made dashboards.

Agentic observability is evolving

Agentic AI systems are quickly moving from proof-of-concept to production, giving customers the ability to automate complex workflows, invoke a variety of different tools and APIs, and coordinate tasks across multiple services. However, traditional monitoring overlooks critical AI-specific signals, such as token consumption, model behavior, and guardrail outcomes. Teams struggle to trace non-linear agent flows, establish baselines for dynamic systems, and maintain predictable costs as usage scales. Without purpose-built observability, organizations risk degraded experiences, higher costs, and compliance gaps as agent complexity grows.

As agentic AI moves from pilot programs to production, organizations are automating complex, cross-system workflows with Amazon Bedrock AgentCore. However, most monitoring stacks weren’t designed for emergent, tool-driven behaviors and, therefore, leave blind spots around correctness, safety, and cost. Teams struggle to trace non-linear flows, establish baselines for dynamic systems, build agentic workflows, and keep token-driven spend under control as usage scales.

The observability gap in AI agent deployments

While AI agents offer significant benefits, including improved employee productivity, increased efficiency, and competitive advantage, among others, an observability gap remains, creating the following challenges:

• Complex multi-step workflows
  AI agents run non-linear, multi-system sequences with inter-agent dependencies, making data flow and responsibility hard to trace. This obscures where time is spent and who is responsible for failures in the chain.
• Limitations of traditional metrics
  Basic operational metrics often overlook AI reasoning errors and quality issues that don’t significantly affect CPU or p95 latency. Without AI-specific telemetry, subtle degradations often slip through.
• Continuous underlying agent and LLM model version changes
  Your system might be robust today, but upstream model and version updates can alter behavior, latency, and costs, forcing continuous adaptation to prevent regressions and incidents. Proactive detection of model-induced changes is crucial to maintaining stable quality and safety over time.
• Scalability and quality challenges
  As deployments grow, telemetry volume and coordination overhead surge while token usage and API calls remain untracked. This breaks cost predictability and quality control, leading to issues such as hallucinations and model drift. Multi-agent logic evolves constantly, so “normal” is a moving target. Baselines drift, complicating anomaly detection and root-cause analysis.

Without addressing these challenges, organizations face risks, from degraded user experiences and spiraling costs to compliance violations and reputational damage.

New enhancements for teams building with Amazon Bedrock AgentCore

The new Dynatrace AI Observability app embeds Amazon Bedrock AgentCore observability into a dedicated end-to-end experience, featuring out-of-the-box analytics, auto-instrumentation, targeted GenAI metrics, debugging flows, and ready-made dashboards to address all observability gaps in agent deployments. Support is available for over 20 technologies, including Amazon Bedrock, OpenAI, Gemini/Vertex, Anthropic, and LangChain.

These enhancements enable teams to take advantage of the following benefits:

• End-to-end distributed tracing
  Trace every interaction from user prompt to model reasoning to tool calls, so you can pinpoint bottlenecks, errors, or costly loops in seconds. Filter by model, provider, token usage, latency, and more to accelerate root-cause analysis.
• Enriched GenAI telemetry data, out of the box
  Each LLM and tool invocation emits spans with prompts, completions, token counts (for both prompts and completions), finish reasons, model IDs, latency, and errors, utilizing GenAI semantic attributes. Orchestration layers (for example, actions, HTTP durations, and step names) are captured for the complete workflow context.
• Cost, performance, and safety insights
  Use intelligent forecasting to detect cost and performance anomalies in token consumption and latency. Monitor guardrails for toxicity, PII, and denied topics to build trust and meet compliance requirements.
• Simple OTLP setup, fast time to value
  AgentCore already emits telemetry; simply register the OpenTelemetry export to Dynatrace once. Use your Dynatrace OTLP endpoint and token, and you’re streaming signals into the Dynatrace Grail® data lakehouse with no code rewrites. Ready-made dashboards for Amazon Bedrock let you verify ingestion and gain instant insights.

What’s next

We’re investing in a deeper Amazon Bedrock model and provider insights, expanded guardrail analytics, and additional automation so you can attach remediation playbooks to cost or safety anomalies.

Additionally, we’ll introduce a new agent visualization and topology experience that visualizes your AgentCore agents, LLM services, tool backends, and dependencies, allowing you to understand real-time topology and data flows across the entire stack.

Navigate from the topology map to traces to follow agent behavior step-by-step across services, protocols, and external calls, pinpointing hotspots, ownership, and blast radius more quickly.

Expect tighter integrations with popular orchestration frameworks and more dashboards for common agent patterns, such as retrieval, multi-agent collaboration, and tool-heavy workflows.

Get started with Dynatrace AI Observability for Amazon Bedrock AgentCore agents

Ready to learn more? Have a look at our GitHub repository.

Start instrumenting your agents today. Open the Amazon Bedrock AI Observability dashboard in Dynatrace to verify telemetry and begin your analysis.

The post Announcing Amazon Bedrock AgentCore Agent Observability appeared first on Dynatrace news.

• Your AI investments can’t reach their potential without effective AI governance.
• AI governance is a challenge that demands unprecedented agility, proactive measures, and comprehensive oversight to manage complexity.
• With Dynatrace, you’re prepared for whatever comes next.

Stay compliant and build trust in your AI systems

AI regulation is tightening, and non-compliance is becoming a huge risk to broader, production-scale AI adoption. Penalties are only part of the impact: reputational damage, customer mistrust, and stalled innovation can cripple even forward-looking organizations. That’s why we’re introducing data governance and audit trails for AI observability: a scalable way to manage, monitor, and secure the AI data lifecycle with end-to-end lineage, retention controls, and evidentiary records of model and user interactions.

Our platform helps you turn governance into a competitive advantage. Built-in audit support helps with emerging regulations like the EU AI Act, and alignment to industry standard frameworks such as NIST AI and ISO/IEC 42001:2023.

The hidden challenges of AI data governance

The complexity of compliance

AI regulations are becoming stricter, and new regulations are on the horizon. Organizations must maintain detailed records of AI activities for years, ensure transparency of data and processes, and align retention policies with legal requirements. These measures are imperative for trust and safety, but they introduce significant challenges. For instance, AI-related events are often scattered across multiple systems, applications, and teams, complicating efforts to create a unified audit trail. Default retention periods can fall short of regulatory needs, and manual governance processes are error-prone and infeasible at scale.

The risk of non-compliance

Failing to meet regulatory standards risks hefty fines and penalties, but the market consequences, reputational damage, and loss of customer trust are even worse. Without the right tools, organizations will struggle to manage the growing complexity of AI data governance and reap the full benefits of AI investments.

Introducing Dynatrace data governance and audit trails

Dynatrace has a long history of empowering organizations to tackle complex challenges with AI-driven solutions. Building on this expertise, we’re introducing a new set of capabilities designed to simplify compliance, enhance transparency, and streamline data management. With Dynatrace, you can:

• Automatically retain AI-related events for up to 10 years in Grail®, our secure data lakehouse.
• Monitor and capture events from platforms like Amazon Bedrock, tracking everything from model deployments to fine-tuning activities.
• Leverage OpenTelemetry to collect real-time traces and metrics of AI workloads, along with every AI user interaction, giving you a complete picture of your AI ecosystem.

Close the compliance gap with embedded oversight

What sets Dynatrace apart is seamless integration with your existing workflows. With OpenPipeline® on Grail, you can route AI-related events to custom storage buckets with extended retention, automatically, and without forcing teams to change tools or processes. This allows long-term auditability and helps meet sector-specific compliance requirements that might require special retention and auditability measures.
Once configured, Dynatrace can automatically route and store events, creating a reliable and transparent audit trail. This helps to reduce fragmentation, tool sprawl, and manual effort traditionally associated with data governance.

Imagine being able to trace every user interaction, model training session, or deployment event with just a few clicks. Dynatrace makes this possible by consolidating fragmented data into a single, coherent view. Whether you’re responding to a regulatory inquiry or optimizing your AI models, you’ll have the insights you need, when you need them.

Simplified and instant data filtering with Dynatrace segments

Not all audit data carries the same compliance weight. For global enterprises with complex IT environments, the ability to instantly filter data by precise criteria is essential for accelerating compliance across diverse regulations, from strict local regulatory transparency obligations to lighter regimes elsewhere.

Dynatrace segments make it simple to break down and filter data to match your analysis needs and regulatory requirements:

• Targeted compliance views: Instantly filter audit data by region, environment, platform, model, or custom criteria to align with diverse regulatory requirements.
• Dynamic adaptability: Segments automatically update, for example, when new LLM models or environments are introduced, minimizing manual maintenance and keeping governance current.
• Reusable assets: Leverage a single dashboard or notebook across multiple use cases by simply applying different segments, reducing duplication of effort.
• Noise reduction: Exclude irrelevant data such as development or test logs to keep compliance and observability focused on what truly matters.
• Custom team-context: Provide different teams (for example, compliance, data science, operations) with clear, filtered views of their audit data, ensuring ownership and audit-readiness across departments.

From observability to trusted automation

The future of AI governance lies in proactive, automated solutions that not only meet today’s regulations but also anticipate tomorrow’s challenges. With Dynatrace, you’re not just complying—you’re building a foundation of trust and reliability that scales with your business. By capturing and integrating AI events into a unified platform, Dynatrace transforms compliance from a burden into a strategic advantage.

Get started today

Ready to simplify your AI data governance?

Try it out yourself on the Dynatrace playground. Or, learn how to configure AI governance in our documentation.

Read more

• Part one of the Rise of Agentic AI blog series covers the fundamentals of AI agents, models, and emerging communication standards such as Agent2Agent (A2A) and MCP.
• Part two of the Rise of Agentic AI blog series explores AI agent observability and monitoring, A2A and MCP communications, and how to scale and monitor Amazon Bedrock Agents.
• Part three explains how to monitor Amazon Bedrock Agents and how observability optimizes AI agents at scale.
• Part four covers full-stack observability for AI with NVIDIA Blackwell and NVIDIA NIM.
• Part five demonstrates how to build a simple agentic application using the OpenAI Agents SDK and instrument the data with Dynatrace.
• Part six explores AI Model Versioning and A/B testing for smarter LLM services.

The post The rise of agentic AI part 7: introducing data governance and audit trails for AI services appeared first on Dynatrace news.

Enterprise adoption of generative AI is showing no signs of slowing down, and it’s easy to understand why; organizations in every vertical aim to reap its benefits, including increased efficiency, routine task automation, and content generation, ultimately creating a competitive advantage. To better help organizations maximize the benefit and full potential of generative AI, Dynatrace has achieved the Amazon Web Services Generative AI (GenAI) Competency.

With this milestone, Dynatrace reinforces its position as a leading observability partner, backed by a proven track record of innovation and customer success on AWS. Building on its achievement of earning the AWS Machine Learning Competency, Dynatrace continues to drive advancements in generative AI.

Weighing the importance of this milestone for Dynatrace customers

This competency is more than just a badge; it’s a validation of how Dynatrace can help organizations safely, efficiently, and cost-effectively adopt generative AI in their business. AWS awards these competencies after rigorous technical validation and proven customer success. This means organizations can trust that Dynatrace solutions are designed to deliver measurable outcomes on AWS.

For existing customers, this competency reaffirms the Dynatrace commitment to continued innovation alongside AWS. This ensures the Dynatrace AI-powered observability platform evolves with the latest advancements in AI, future-proofing organizations’ existing investments as generative AI capabilities become core to modern cloud workloads.

For new customers, Dynatrace provides a trusted, proven foundation for observability and AI adoption on AWS. Whether an organization is exploring GenAI for customer engagement, automation, or new digital experiences, Dynatrace ensures these systems are reliable, secure, and optimized at every step.

Looking ahead with AI-powered observability on AWS

As organizations increasingly adopt generative AI, observability becomes a critical enabler. By leveraging Dynatrace causal AI, predictive insights, and seamless AWS integrations, organizations can maintain control over costs, risks, and performance while driving innovation, enhancing competitive advantage, and delivering exceptional customer experiences.

Whether you’re building, scaling, or fine-tuning GenAI application, Dynatrace and AWS Bedrock empower you to transform your observability. With end-to-end visibility into AI workloads, their interactions in full context of your business, and cloud-native applications, you can optimize performance, troubleshoot effectively, and maximize the value of your GenAI investments with greater confidence and precision.

Learn more

• Davis AI
• Dynatrace for Amazon Bedrock
• Explore the Bedrock Monitoring Playground to get hands-on with real-time dashboards that reveal how your Amazon Bedrock models behave in production.
• Install the AI Observability app to integrate Dynatrace’s AI Observability app into your environment to monitor GenAI workloads.
• Stop by booth #575 at AWS re:Invent 2025 to see Dynatrace in action.

The post Dynatrace achieves AWS Generative AI Competency: A new milestone in observability and AI appeared first on Dynatrace news.

Debug, optimize, and secure your AI models with confidence

As agentic AI applications and systems gain traction, delivering reliable, high‑performing LLMs and agents becomes challenging due to heterogeneous stacks, non‑deterministic behavior, and cost sensitivity across multi‑cloud runtimes. Reliable delivery and deployment to production requires end-to-end telemetry across the full chain:
UI/services → orchestration/agents (LangChain, LlamaIndex, MCP/A2A) → RAG pipeline (embedding + vector DB) → model gateway (OpenAI, Azure/OpenAI, Bedrock, Gemini, Mistral, DeepSeek) → GPU/infra. To support deterministic rollouts and continuous model improvement, teams need standardized tracing/metrics, guardrail signal capture, and automated cost and performance governance.

The hidden challenges of AI model management

The invisible bottlenecks

AI models, especially LLMs, are prone to issues like hallucinations, degraded performance, and incorrect outputs. Debugging these problems is often like finding a needle in a haystack. Existing tools fall short in providing a unified view to compare prompts, datasets, or model versions, making it hard to identify regressions or improvements.

Navigating the risks of rapid model evolution

The impact of deprecation and automatic upgrades on cost, performance, and quality

The rapid pace of innovation in the AI space means that providers like OpenAI and Anthropic frequently release new versions of their models, such as ChatGPT 5 or Anthropic Opus 4.1.

While these updates often promise better performance and new capabilities, they can also introduce significant risks for your AI services:

• Deprecation of older versions: Providers may discontinue support for older models, forcing you to adopt newer versions without sufficient time to test their impact.
• Automatic upgrades: Many AI providers automatically update their underlying models, which can lead to unexpected changes in behavior, degraded performance, or even broken workflows.
• Compatibility issues: Changes in model behavior, such as output format or token usage, can disrupt your application’s functionality, requiring adjustments to prompts, configurations, or integrations.

Tracking token usage and managing costs is another uphill battle. Add to this the risk of prompt injection attacks and data leaks, and it’s clear that traditional methods are no longer sufficient

The new AI Model Versioning and A/B testing

Ship better models with confidence. In a single view, compare models and versions to validate improvements and spot bottlenecks across latency, reliability, token usage, cost, and output quality, then drill into prompt-level differences to confirm why a variant wins. When something breaks, follow the request end to end with distributed tracing: from input through orchestration steps and model calls to completion, so you can pinpoint exactly where an error or slowdown originated.

Compare models and versions: Detect bottlenecks and validate improvements in a single view.

Trace prompt failures: Debug errors from input to output with our Distributed Tracing solution.

Monitor costs and token usage: Gain real-time insights into token consumption and cost implications.

Detect security and guardrail risks: Identify and alert on vulnerabilities like prompt injection attacks, toxic responses, or captured PII.

Attach your own attributes like user session, feedback, or dataset ID for additional debugging information.

How it works

With AI Model Versioning, you can track metadata such as model version, dataset ID, and hyperparameters.

A/B testing lets you expose different user segments to model variations, providing data-driven insights into performance metrics like accuracy and cost.

Instrument in minutes: Use the supported OpenTelemetry-based SDK to instrument your service to capture prompts, completions, token usage, errors, and guardrail signals.
You can also enrich spans with attributes like model.version, dataset.id, user/session, and feedback for deeper analysis. (You can read more about this here.)

Start analyzing out of the box: Once data is flowing, the AI Observability app provides ready-made dashboards and distributed tracing so you can compare models/versions, monitor costs and tokens, and debug prompt failures end to end. No extra setup is required; you can try it out on the Dynatrace Playground right now.

By combining observability, AI-driven insights, and organizational knowledge, we’re enabling systems that don’t just react but learn and adapt. Each critical issue or incident you resolve fuels a living knowledge base, paving the way for proactive incident prevention through alerting.

What’s next?

We’re committed to enhancing these capabilities further. Upcoming updates will include a dedicated app experience for multi-model and multi-cloud setups, advanced visualization tools, enhanced security features, intelligent forecasting, and alerting for cost/performance and guardrail optimization.

Get started today

Ready to revolutionize your AI services? Here’s how:

1. Sign up for a free trial.
2. Install the AI Observability app.
3. Explore the AI Model Versioning ready-made dashboard, or check it out on our playground

Together, let’s build smarter, more reliable AI systems.

Read more

• Part one of the Rise of Agentic AI blog series covers the fundamentals of AI agents, models, and emerging communication standards such as Agent2Agent (A2A) and MCP.
• Part two of the Rise of Agentic AI blog series explores AI agent observability and monitoring, A2A and MCP communications, and how to scale and monitor Amazon Bedrock Agents.
• Part three explains how to monitor Amazon Bedrock Agents and how observability optimizes AI agents at scale.
• Part four covers full-stack observability for AI with NVIDIA Blackwell and NVIDIA NIM.
• Part five demonstrates how to build a simple agentic application using the OpenAI Agents SDK and instrument the data with Dynatrace.
• Part seven introduces data governance and audit trails for AI services.

The post The rise of agentic AI part 6: Introducing AI Model Versioning and A/B testing for smarter LLM services appeared first on Dynatrace news.

As AI investment accelerates, a gap is emerging between ambition and execution. IDC projects¹ that by 2028, AI spending will make up 16.4% of total IT expenditures. However, Gartner, Inc.² predicts over 40% of agentic AI projects will be canceled by end of 2027. Likewise, a CIO survey found that 88% of AI pilots fail to reach production due to unclear objectives, insufficient data readiness, and a lack of in-house expertise. These findings place the expected return on research and innovation firmly at risk, as organizations invest in bespoke models and agentic AI that lack a clear, scalable outcome.

Nonetheless, another Gartner, Inc. article³ predicts that by 2028, 33% of enterprise software applications will include agentic AI, up from less than 1% in 2024, enabling 15% of day-to-day work decisions to be made autonomously by agentic AI systems. Consequently, a Forrester blog⁴ predicts that 40% of highly regulated enterprises will combine data and AI governance in a move toward a more integrated, transparent, accountable, and ethically responsible approach to AI.

These trends are not contradictory—they show how the market is searching for the right formula to adopt AI, and specifically agentic AI. Successful agentic AI outcomes are predicated on trust in AI’s reliability, security, and alignment with business goals and strategies to not fall behind competitors. Achieving that trust requires AI-native observability that’s deeply integrated with both data and strategic objectives.

Key insights for executives

• Every modern cloud-native enterprise project will also be an AI-native project – either because of first party AI or through invoking agentic AI services. Preparing for AI adoption is among the top drivers for cloud strategy and investment. Likewise, 63% of top-performing companies increase their cloud budgets to be able to leverage AI.
• Visibility into reliability and governance of AI interactions has emerged as a new responsibility for executives to realize the value of AI investments while managing risks. From analyst firms in the US to regulators in the EU – increased oversight, link to business goals and regulation of AI systems has become mandatory.
• Unifying observability signals with AI-powered analytics provides a strategic advantage for AI transformation. By converging observability and AI, teams can accelerate moving projects from pilot production and advance trust and transparency in AI.
• Dynatrace sets the standard for cloud- and AI-native software, including tracing and logging of AI behavior, predicting and optimizing AI resource utilization, and protecting from unintended AI behavior through runtime security.
• Dynatrace delivers unified, full-stack visibility across cloud infrastructure, AI workloads—from chat interfaces and prompts to models, tools, and GPUs running on Kubernetes—plus customer experiences and the business layer, all in a single pane of glass, to confidently deliver advanced, AI-powered cloud-native services via a rapidly growing number of 40+ technologies and integrations with hyperscalers and major agentic frameworks providers.

The rise of AI comes with a rise in complexity—and executive responsibility

Organizations generally find themselves maturing their AI implementations along five phases with growing complexity and risks:

1. Prompt engineering (generative Al hype). Single step human language prompts a large language model (LLM) for automated text processing and assistance.
2. Retrieval augmented generation (embedding Al in digital services). Multi-step prompt engineering and LLM access for customer support, automation, and decision-making.
3. Fine-tuned models. Additional model(s) put on top of existing ones for increased accuracy and domain-aware responses.
4. Multimodal GenAI. Combination of various modalities beyond text—such as video, audio, imaging and others—that further increase heterogeneity and processing power of services and their interdependences.
5. Agentic Al. Multiple AI agents and cloud native digital services intensively interacting with each other to autonomously fulfill a specific goal. Agentic AI can double the number of deployed digital service instances and massively increase IT complexity.

The necessity of AI observability for agentic AI reliability

As the complexity of AI implementations increases, observability becomes an essential feedback channel to properly orchestrate and moderate reliable agentic AI outcomes.

Even the early phase implementations show the need to observe AI, tune experience, manage cost, provide guardrails and govern AI responsibly. As the complexity grows, the risks also increase, making deep, context-rich observability of AI strictly mandatory.

7 important reasons for continuously observing AI

1. Business value. Validate AI investments against business goals and verify end-user value of AI services. Gain business insights from observability data.
2. Cost and performance control. Monitor and control expenses and sustainability associated with AI operations and investments.
3. Security. Increase awareness of interactions among AI services, reducing the risk of hacking and malicious influence. Leverage converged observability and security offerings to minimize risk and cost.
4. Compliance. Monitor that AI output is ethical, unbiased, and adheres to guardrails for meeting regulatory compliance requirements and providing traceability for audits. Expect high volumes of logs and traces to observe AI behaviors and keep audit trails.
5. Accuracy. Verify that AI agents function properly and precisely, generating quality output. Use observability to deeply check run-time behaviors and
6. Reliability. Provide traceability and root-cause analysis to verify AI agent health, scalability, performance, and availability.
7. Collaboration. Govern communications among agent-to-agent and agent-to-human, and provide the means to keep humans in control to override and take responsibility. Automate events from observability platforms that integrate with enterprise ecosystems.

With Dynatrace, executives can solve one of the biggest challenges of managing return on AI investment: Balancing innovation speed with risk, cost, and value.

Increase AI success with AI Observability from Dynatrace

AI is not a single component. Agentic AI in particular is composed of multiple layers and technologies, each observed within a holistic context. Dynatrace provides complete coverage of all layers that allows teams to observe the complete AI stack of modern cloud- and AI-native applications. The layers consist of the following:

• Business – track outcome: does it create productivity gains, does it deflect support tickets, does it act autonomously and is the investment worth it
• Infrastructure – utilization, saturation, errors
• Models – accuracy, precision/recall, explainability
• Semantic caches and vector databases – volume, distribution
• Orchestration – performance, versions, degradation
• Agentic layer – autonomous agents, MCPs
• Application health – availability, latency, reliability

Dynatrace automatically observes and analyzes complex multicloud and agentic AI systems. By securely unifying and storing all data in context, the Grail® data lakehouse with massively parallel processing unifies all data signals with full context and is continuously updated by Dynatrace Smartscape® real-time dependency mapping technology.

Davis® AI combines predictive, causal, and generative AI to provide deterministic answers and insights, which drive AutomationEngine actions and inform teams with recommendations to optimize productivity, performance, and cost. With these advantages, teams can embrace AI with confidence, make better decisions faster, and innovate at speed—without compromising trust, performance, reliability, or control.

Why Dynatrace for reliable agentic AI projects

Top Fortune 500™ organizations use Dynatrace to not only maximize return on investment (ROI) in AI technologies, but across their cloud- and enterprise stacks. Dynatrace leverages partnerships with hyperscalers and major AI framework providers to provide customers with observability for the latest technologies in this fast-moving space.

The recent announcement of our collaboration with NVIDIA is an example of our commitment to providing differentiated AI observability. Dynatrace AI observability delivers real-time, end-to-end observability into AI and LLM workloads—from infrastructure and applications to model performance and end-user experiences. This empowers enterprises to accelerate innovation, ensure compliance, and confidently scale mission-critical AI, all while maintaining reliability and efficiency across their cloud environments.

_____________________________________________

¹ _{IDC Market Forecast, “Worldwide Artificial Intelligence IT Spending Forecast, 2024–2028,” October 2024, https://my.idc.com/getdoc.jsp?containerId=US52635424&pageType=PRINTFRIENDLY.}

² _{Gartner Press Release, “Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027,” June 25, 2025, https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027}

_{GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.}

³ _{Gartner Article, “Intelligent Agents in AI Really Can Work Alone. Here’s How.,” by Tom Coshow, October 01, 2024, https://www.gartner.com/en/articles/intelligent-agent-in-ai.}

⁴ _{“Predictions 2025: An AI Reality Check Paves The Path For Long-Term Success,” Forrester Research, Inc., by Jayesh Chaurasia and Sudha Maheshwari, October 22, 2024, https://www.forrester.com/blogs/predictions-2025-artificial-intelligence/.}

The post Delivering agentic AI reliability: Why AI Observability is imperative appeared first on Dynatrace news.

Recently, OpenAI released a customer service agents demo built using the OpenAI Agents Python SDK that showcases an example multi-agent system at work. With the OpenAI Agents SDK, you can build agentic AI applications with the help of agents, handoffs, guardrails, tools (built-in and custom), and built-in tracing. These capabilities support the core pattern of knowledge, reasoning, and actioning as the foundation for scalable and trustworthy automation, first introduced by Dynatrace CTO Bernd Greifeneder.

In this blog post, we explain how to build an OpenAI agents SDK-based agentic application and instrument the agents and app with AI-powered observability from Dynatrace. Dynatrace can help you see agent executions, tool usages, and prompt flows from initial request to final response for quick root cause analysis and troubleshooting.

To illustrate the capabilities of the OpenAI Agents SDK and agent framework with Azure OpenAI on Azure AI Foundry, we have built our multi-agent solution using the OpenAI customer service agents demo mentioned above as a reference and modified it for our use cases.

About our sample agentic AI application

Our multi-agent system enables users to research, summarize and translate across a range of topics and content. The system consists of four agents:

• Welcome Agent: Engages the user, reasons with Azure OpenAI to analyze the prompt, identifies the intent, and passes it to the right agent to start processing.
• Researcher Agent: Searches the web and analyzes the results using OpenAI.
• Summarizer Agent: Summarizes content, including search results, text, PDF, CSV, and more, using Anthropic Claude.
• Translator agent: Translates queries and inputs into any user-requested language using OpenAI.

Next, we want the multi-agent system to perform two distinct scenarios:

1. Context history: In a specified chat session, the entire chat history and context is available for the duration of the session, while the individual prompts might be handed off to different agents for processing.
2. Composite queries: The app orchestrates multiple different agents for different purposes, such as Research, Translate, Summary, and Welcome, so users can engage to process a composite prompt with multiple sub-queries.

Understanding multi-agent frameworks and handoff workflows

There are some key differences between the agent frameworks. Unlike the A2A protocol, the OpenAI framework does not explicitly have a central registry for agents. Instead, OpenAI agents use the concept of “handoffs” orchestrated by the OpenAI Agent Framework.

OpenAI framework agent handoffs

While orchestrator-led coordination offers a more deterministic and structured workflow, agent-to-agent handoffs provide significant advantages in adaptability and modularity. These handoffs enable agents to collaborate dynamically, making it possible to handle complex, multi-step queries with greater flexibility. This approach focuses on a more decentralized and scalable system, allowing agents to specialize and respond to changing requirements in real-time.

Here are two example scenarios to illustrate the agent-to-agent collaboration in chat sessions, with context, as well as delivering multi-agent query processing.

Show the user prompts for a composite query and multi-agent workflow

For example: “Research Michael Jordan, then summarize in 40 words or less, and then translate to French.”

Multi-agent processing for CSV files uploaded

This example includes sample customer data to showcase multi-agent workflow processes with context and history in the chat session.

Overall, the agent-to-agent handoffs worked well (and with context) during all the session runs. Tracing and debugging can be achieved by instrumenting the SDK with OpenTelemetry and sending the data to Dynatrace’s built-in AI Observability solution for Azure OpenAI. You can easily capture the multi-agent workflow for a given prompt on the Azure AI Foundry platform dashboard. Find the code examples in our GitHub repository.

Set up tracing using Python

Using Python, you can set up the tracing by changing a few simple lines of code in your agent framework and core component:

from traceloop.sdk import Traceloop Traceloop.init( app_name="openai-cs-agents", api_endpoint="https://wkf10640.live.dynatrace.com/api/v2/otlp", disable_batch=True, headers=headers, should_enrich_metrics=True, ) 
with tracer.start_as_current_span(name="update_seat", kind=trace.SpanKind.INTERNAL) as span: 
    context.context.confirmation_number = confirmation_number 
    context.context.seat_number = new_seat 
    assert context.context.flight_number is not None, "Flight number is required" 
    return f"Updated seat to {new_seat} for confirmation number {confirmation_number}"

You can see the results right away in distributed tracing:

OpenAI orchestration

Within the OpenAI framework, there are two approaches to orchestrating agents:

1. Allow the LLM to make decisions: Use the intelligence of an LLM to plan, reason, and decide what steps to take.
2. Orchestrate with code: Use code to determine the flow of agents.

Overall, the OpenAI Agents SDK is comprehensive and easy to get running with some minor code changes, this time with OpenAI’s Codex assistant.

Multiple frameworks and toolkits are quickly ramping up to make multi-agent systems a reality. We foresee this space evolving and innovating rapidly.

The evolution of multi-agent systems

As agentic AI continues to advance, multi-agent applications are poised to play a transformative role in reshaping how applications operate. These systems enable dynamic, context-aware collaboration between specialized agents, empowering businesses to tackle increasingly complex workflows. From helping with automation, orchestrating large-scale data analysis, multi-agent systems will unlock new levels of efficiency, scalability, and innovation.

Tools like the OpenAI Agents SDK on Azure AI Foundry and Azure AI Studio are at the forefront of this evolution. By providing built-in capabilities such as agent handoffs, guardrails, and tracing, the SDK simplifies the development and monitoring of multi-agent workflows. These features make it easier for organizations to deploy responsible, secure, and robust AI systems and also ensure transparency and trustworthiness in their operations. These are key factors for widespread adoption.

Looking ahead, we can expect rapid innovation in this space. Emerging standards like MCP, A2A protocols, and frameworks such as OpenAI Agents are creating a vibrant ecosystem for multi-agent interoperability. The focus will likely shift toward even more intelligent and reliable orchestration, where agents autonomously plan, reason, and adapt to dynamic environments.

AI Observability for agentic AI applications

To keep pace with these advancements, we believe that observability must evolve in lockstep to ensure transparency across heterogeneous agent ecosystems. Advancements in observability tools, such as the Dynatrace AI Observability solution, are essential to help create more reliable and scalable AI frameworks at the enterprise level.

The future of multi-agent systems holds immense potential, with the OpenAI SDK marking the starting point. We’re just at the beginning of what’s possible. As this technology evolves, it will gradually become more stable and reliable, ultimately transforming the way we approach automation, collaboration, and AI-powered problem-solving across industries.

Check out our GitHub repo for detailed code examples for OpenAI Agents, AWS Strands, Google ADK, and start building your own AI Observability solutions today.

Read more

• Part one of the Rise of Agentic AI blog series covers the fundamentals of AI agents, models, and emerging communication standards such as Agent2Agent (A2A) and MCP.
• Part two explores how monitoring A2A and MCP communications results in better, more effective agentic AI. This blog post covers AI agent observability and monitoring, and how to scale and monitor Amazon Bedrock Agents.
• Part three explains how to monitor Amazon Bedrock Agents and how observability optimizes AI agents at scale.
• Part four covers full-stack observability for AI with NVIDIA Blackwell and NVIDIA NIM.
• Part six explores AI Model Versioning and A/B testing for smarter LLM services.
• Part seven introduces data governance and audit trails for AI services.

Together, these capabilities make it possible to achieve robust, scalable observability in agentic AI environments so teams can build reliable and trustworthy applications and services.

The post The rise of agentic AI part 5: Developing and monitoring multi-agent applications with OpenAI Agents SDK on Azure AI Foundry appeared first on Dynatrace news.

A generational technology shift is afoot—one where AI-powered workloads are the new currency of innovation. Agentic applications—built on foundation models, APIs, and autonomous workflows—are transforming how businesses create, deliver, and scale value.

Enterprise leaders are no longer asking if AI will play a central role in their strategy. Rather, they’re asking how fast they can operationalize AI to drive personalization, automation, and competitive advantage.

From observability to actionable intelligence

The key to unlocking AI’s full potential lies in understanding what your data is telling you. AI-native organizations don’t just need visibility; they need context, causality, and automation across their digital systems.

This is where Dynatrace excels. By combining full-stack observability with causal AI and advanced analytics, Dynatrace can help enterprises move beyond dashboards toward self-healing systems, intelligent automation, and AI-powered experiences that continuously improve.

Enhance agentic solutions with Dynatrace through the AWS Agentic AI Marketplace

To accelerate your journey in this AI-driven era, Dynatrace is proud to be part of the Amazon Web Services (AWS) Agentic AI Marketplace—Amazon’s emerging ecosystem designed to bring together agentic applications, reusable APIs, data sets, and generative AI solutions.

This integration delivers significant value, including the following:

• Seamless integration for AI workflows. Autonomous agents and builders can now readily discover and connect with Dynatrace, making it easier to compose sophisticated, intelligent workflows.
• Empower AI agents. AI agents building real-time, adaptive solutions can directly invoke Dynatrace observability, security, and automation capabilities, enabling them to make more informed decisions and take effective actions.
• Faster innovation. Enterprises benefit from faster time to value, as Dynatrace solutions are now natively composable within AWS AI services, such as Amazon Q, Amazon Bedrock, and Amazon SageMaker.

Ultimately, the AWS Agentic AI Marketplace empowers AI agents to dynamically discover, invoke, and orchestrate trusted services like Dynatrace, providing the foundation for an  autonomous and intelligent enterprise.

How Dynatrace enables business value in the agentic era

With the Dynatrace® AI-powered observability platform, organizations can benefit from the following capabilities:

• Smarter, safer automation. Dynatrace provides real-time insights into system health, anomalies, and dependencies—empowering agents to take autonomous action without compromising reliability or security.
• Accelerated AI decision-making. Dynatrace data streams feed foundational models and agentic workflows with high-fidelity, context-rich information—leading to faster, more accurate decision-making at scale.
• Personalized digital experiences. By understanding user behavior, performance trends, and business context, Dynatrace enables agents to dynamically personalize digital experiences in real time.
• Continuous optimization. From cost-aware workload placement to real-time cloud performance tuning, Dynatrace allows enterprises to scale AI workloads with confidence and efficiency.

Answering the call to innovation

The future of enterprise software is here: agentic, composable, and autonomous. For leaders navigating this massive shift, the question isn’t whether AI will play a central role, but how quickly teams can operationalize it to drive competitive advantage.

Now is the time to evaluate if your systems are truly ready—not just for AI adoption, but for AI collaboration. In this new ecosystem, it’s not simply about building AI; it’s about building with AI. This demands partners who can unlock the full potential of your data, enable your systems to adapt seamlessly, and accelerate your business outcomes. With Dynatrace, you gain a partner actively helping businesses operationalize trust, observability, and AI at scale.

The post How Dynatrace drives value in the age of AI in the AWS® Agentic AI Marketplace appeared first on Dynatrace news.

NVIDIA Blackwell systems provide high-performance infrastructure for enterprise AI, and now, thanks to the Dynatrace integration with the NVIDIA Enterprise AI Factory reference design, enterprises can add Dynatrace Full-Stack Observability to NVIDIA Blackwell infrastructure. This magnifies the value of the NVIDIA Blackwell platform by providing real-time performance insights, anomaly detection, and dependency mapping.

Keep high performance and security top of mind with unified observability and security

Dynatrace aligns with high data security and privacy standards typical of on-premises NVIDIA Blackwell deployments, particularly in regulated industries such as finance and healthcare. Its unified data model, Smartscape® topology mapping, and Davis® AI engine provide deep visibility into the full stack—from GPU metrics and containerized workloads to distributed applications and user experiences, enabling tailored observability for workloads running on  NVIDIA Blackwell. Integrating NVIDIA Data Center GPU Manager or other telemetry sources is straightforward, allowing teams to monitor GPU health, utilization, thermal thresholds, and memory bandwidth alongside traditional infrastructure metrics.

Dynatrace technology allows for automated discovery and instrumentation of services running on NVIDIA Blackwell-accelerated systems. Whether monitoring high-throughput GPU compute tasks, Kubernetes clusters, or microservices, Dynatrace ensures low-overhead performance monitoring with minimal manual configuration.

AI-powered, real-time insights improve performance and explainability

With Dynatrace Full-Stack AI Observability, you can monitor real-time performance, trace prompts end-to-end, and ensure compliance, optimizing cost and throughput for your AI and LLM workflows and agents, offering various use cases such as

• Monitor service health and performance, tracking real-time metrics and offering clear visibility into service incidents.
• Validate service quality by measuring response speed or identifying performance hotspots.
• End-to-end tracing and debugging pinpoint the root cause of errors and failures in the LLM chain, troubleshoot issues in complex pipelines, and trace dependencies across the entire system spanning multiple LLMs, RAG pipelines, and agentic frameworks.

Unified AI-powered observability

Dynatrace delivers full stack observability for your LLMs and Generative AI applications running on NVIDIA Blackwell systems. Its ability to provide visibility into complex, high-performance environments allows enterprises to fully leverage Blackwell’s capabilities while maintaining operational excellence and system reliability, improving the performance, explainability, and compliance of your AI workloads and agents.

Visit the Dynatrace Playground to learn more and gain hands-on experience with prepopulated data, so you can experience the possibilities of AI and LLM observability with Dynatrace. If you’re interested in using Dynatrace for your own AI workloads, visit our documentation and start benefiting from full stack observability for AI and LLM.

Read more

• Part one of the Rise of Agentic AI blog series covers the fundamentals of AI agents, models, and emerging communication standards such as Agent2Agent (A2A) and MCP.
• Part two explores AI agent observability and monitoring, A2A and MCP communications, and how to scale and monitor Amazon Bedrock Agents.
• Part three explains how to monitor Amazon Bedrock Agents and how observability optimizes AI agents at scale.
• Part five demonstrates how to build a simple agentic application using the OpenAI Agents SDK and instrument the data with Dynatrace.
• Part six explores AI Model Versioning and A/B testing for smarter LLM services.
• Part seven introduces data governance and audit trails for AI services.

The post The rise of agentic AI part 4: Dynatrace delivers full-stack observability for AI with NVIDIA Blackwell and NVIDIA NIM appeared first on Dynatrace news.

Agentic AI is revolutionizing—and dramatically accelerating—how engineering teams respond to incidents and debug use cases. By monitoring how AI agents communicate using standards such as Model Context Protocol (MCP), performance engineers can gain deep insights into how systems and AI models themselves are performing. And crucially, automate incident responses using coding assistants.

The next step in this scenario is to employ some MCP best practices, such as helping full-stack engineers to debug code at runtime without disrupting operations. Fortunately, Dynatrace has the answer with Live Debugger. First debuted earlier in 2025, Live Debugger gives developers instant access to code-level troubleshooting data in any environment, including production, and is now generally available.

With all these capabilities coalesced in an integrated development environment (IDE), developers can interact with AI using natural language to massively accelerate issue resolution.

Key takeaways:

• Unify AI debugging resources in one place. Debugging agentic AI workflows starts with combining Dynatrace features with custom MCPs in an IDE.
• Leverage an AI coding assistant to add context. An AI coding assistant, such as Cline, helps add vital context and improve prompt engineering using natural language queries.
• Use best practices to configure MCPs and Cline. Combining Cline with MCPs and on-the-fly debugging reduces guesswork and manual overhead.

MCP best practices TELUS uses to accelerate incident response

As an early adopter of Dynatrace Live Debugger, TELUS harnessed the power of Davis® AI, MCP, and real-time debug data to gain a deep understanding of their code, enabling every engineer to ask questions in natural language about their code behavior at runtime.

At a recent Dynatrace Guild meeting, TELUS SREs Dana Harrison and Cheng Li demonstrated how they’re combining Dynatrace features like Live Debugger with custom MCPs for the ultimate purpose-built troubleshooting setup in their Visual Studio Code IDE. By integrating multiple resources in one place, the TELUS team can blend next-generation debugging with agentic AI workflows. This integration reduces developer context-switching and enables developers to use straightforward, natural-language prompts.

With a fully instrumented environment that monitors every service end-to-end, the TELUS team leverages Dynatrace Live Debugger for Node.js and Java microservices to monitor running code in real time to reduce mean time to resolution. They initially tested this approach in non-production but are expanding into production with strict auditing and data masking.

The team also uses MCP in tandem with Cline AI, a coding assistant capable of gathering logs, metrics, and trace data through natural language commands. By unifying multiple data sources under MCP, TELUS avoids manually juggling different dashboards and tools. Developers can simply interact with the AI tools, which fetch relevant context on demand, even correlating Live Debugger snapshots with real-time logs. As a result, the entire debugging and incident investigation process remains inside the IDE, which significantly streamlines performance engineering and accelerates issue resolution.

How MCPs empower AI agents for debugging use cases

As an open standard, MCP connects AI agents to relevant data sources, such as repositories, tools, or external APIs. Instead of bespoke integrations for each data silo, MCP provides a universal interface to connect multiple relevant sources to feed the right context to the models and agents. This interface simplifies how agents access relevant context, leading to better task outcomes, execution, and more consistent performance across complex environments. For reference, you can try out the Dynatrace MCP server on GitHub.

MCP best practices: How Cline AI helps to bring in context

Cline AI is a coding assistant in Visual Studio Code that leverages multiple MCP servers to streamline data retrieval from various back-end systems. With plain-language queries (“Search for a specific service,” “Fetch logs from GKE,” or “Investigate errors”), developers can prompt Cline AI to automatically contact the relevant MCP server.

By becoming better at prompt engineering and handling the underlying API calls and correlating responses from Dynatrace MCP, native Kubernetes logs, or even third-party services, Cline AI provides an all-in-one investigation workflow right in the editor. This means developers don’t have to switch among multiple tools or memorize specialized APIs; they simply ask Cline AI what they need in natural language, and the MCP layer manages the rest.

Cline supports MCP-client features, such as dynamic tool discovery, prompt reusability, and adaptive resource access. It also enables powerful features like custom instruction, cline-rule, and memory banks.

TELUS’ best practices for configuring MCPs and Cline

The TELUS team followed these MCP best practices while configuring Cline.

Install Cline AI as VS Code extension

First, the TELUS team installed Cline AI as a VS Code extension, pointing it to their AI proxy, Fuel iX. Fuel iX lets them choose among various large language models based on the user’s requirements and desires.

Use careful prompt engineering and Cline rules to direct specific MCP tools

Cline AI can then interpret a developer’s natural-language prompt—such as “Investigate errors in our Node.js service”, “What is this service about”—and select the appropriate MCP endpoint(s) to gather data. Directing to specific MCP tools for more accurate results can be fine-tuned through careful prompt engineering, creating detailed .clinerules file definitions, and directives built into the MCPs themselves – including predicted requests and response formatting.

Configure each MCP server for a specific data domain

Each MCP server at TELUS is responsible for interacting with a specific data domain: for instance, they have a Dynatrace MCP server to fetch metrics and trace data, a GCP Logs MCP server to pull container and other forwarded logs, and additional servers for services like JIRA issues or PagerDuty.

Rely on centralized agentic AI commands, not vendor-specific APIs

By standardizing how these data sources are exposed, TELUS ensures that Cline AI never has to manage direct integrations or vendor-specific APIs. Instead, the assistant simply issues commands to the MCP servers, which internally handle authentication, query templates, and data normalization. This architecture not only allows TELUS to maintain clear boundaries between data retrieval logic and AI-driven workflows, but also accelerates developer onboarding, making it simple for any team member to debug or investigate issues from within VS Code by asking Cline AI, rather than switching between specialized tools or dashboards.

​​Use MCPs as a standardized endpoint

​In this context, MCP servers act as standardized endpoints, each responsible for a particular data source or application. By relying on Cline for coding assistance, the TELUS team can type a natural-language command (“Investigate issues in our Node.js service”), and Cline will automatically query Dynatrace Grail or GKE logs using the relevant MCP server.

This approach centralizes the complexities of data retrieval in one place and lets the AI assistant produce a consolidated summary or recommended fix, all within the IDE.

Step-by-step guide to set up Cline with Dynatrace

1. Install Cline through the VS Code Marketplace.
   
2. Configure Cline with the required credentials, usually an API endpoint and key.
   
3. Install your MCP. This example uses an internal package, but there are currently many MCPs published for installation.
   
4. Validate that the Dynatrace MCP server is up and running.
   
5. Ask Cline your questions. The agent formats them according to the needs of the MCP(s) you installed.
   
6. Example response of a production authorization service problem pulled and summarized from Dynatrace.
   

The objective of this combined setup (Live Debugger + MCP + Cline) is fast and intelligent troubleshooting. Instead of juggling multiple dashboards and CLI tools, an engineer can see code snapshots, error traces, logs, and even recently created JIRA tickets within one session. As mentioned, MCP standardizes queries across diverse systems, so the AI agents can correlate all relevant information, such as referencing a single trace ID to pull logs from GKE, configuration details from the Kubernetes cluster, or known issues in PagerDuty. This AI agent-powered workflow saves time and reduces context switching.

The TELUS team found that combining these insights with on-the-fly debugging significantly reduces the guesswork and manual overhead that typically slows resolution times.

The road ahead: Developer-first observability and IDE-centric debugging at TELUS with Dynatrace

Although TELUS currently uses Live Debugger in mostly non-production environments, they plan to enable it in production using strict governance. Their upcoming strategy involves role-based permissions for debugging sessions, mandatory security training on data privacy, and well-defined data retention policies to keep snapshots transient.

When combined with MCP’s universal integration points, like Dynatrace MCP, developers can quickly drill down into real production issues, see the exact variables at fault, and either push a fix or revert a misconfiguration with minimal downtime.

Looking ahead, the TELUS team plans to integrate Davis Copilot APIs to further simplify natural language interactions with Dynatrace.

In practice, this would allow an AI assistant to automatically translate human-readable queries into Dynatrace Query Language (DQL), making it even easier to retrieve the right metrics, traces, or logs. By layering Davis CoPilot™ on top of their existing MCP approach, TELUS aims to reduce manual DQL writing while offering developers a powerful yet streamlined way to perform more advanced data analysis through simple, intuitive prompts.

Read more

• Part one of the Rise of Agentic AI blog series covers the fundamentals of AI agents, models, and emerging communication standards such as Agent2Agent (A2A) and MCP.
• Part three explains how to monitor Amazon Bedrock Agents and how observability optimizes AI agents at scale.
• Part four covers full-stack observability for AI with NVIDIA Blackwell and NVIDIA NIM.
• Part five demonstrates how to build a simple agentic application using the OpenAI Agents SDK and instrument the data with Dynatrace.
• Part six explores AI Model Versioning and A/B testing for smarter LLM services.
• Part seven introduces data governance and audit trails for AI services.

More MCP best practices and resources for Dynatrace Live Debugger

• Dynatrace MCP on GitHub
• Dynatrace Live Debugger on Dynatrace Hub
• Dynatrace for Developers Visual Studio plugin on Microsoft Marketplace
• Dynatrace for Developers product page
• Dynatrace Automation Guild on the Dynatrace Community
• 5 powerful use cases beyond debugging for Dynatrace Live Debugger
• Debug complex performance issues in production with ease with Live Debugger, Open Pipeline, and OpenTelemetry

The post The rise of agentic AI part 2: Scaling MCP best practices for seamless developers’ experience in the IDE with Cline appeared first on Dynatrace news.

By now, everyone is aware of generative AI fueled by large language models (LLMs) and generative pre-trained transformers (GPTs). The next level of innovation is agentic AI and the autonomous AI agents that drive it. Using Model Context Protocol (MCP) to facilitate agent-to-agent communication, these systems are revolutionizing how enterprises automate tasks and orchestrate complex workflows.

Powered by LLMs, vector databases, retrieval augmented generation (RAG) pipelines and additional tools, these AI agents are expanding extensively, giving rise to multi-agent systems, cross-agent protocols, and context-sharing standards. But these autonomous agents also introduce new challenges in monitoring, debugging, and security.

We’ll examine in detail the fundamentals of AI agents, models, and the emerging standards that help them communicate, like Agent2Agent (A2A) and Model Context Protocol (MCP).

Key takeaways:

• Autonomous AI agents are the backbone of agentic AI. These services combine to deliver adaptable automated tasks.
• AI agents depend on LLMs and orchestration logic. These technologies maintain the agent’s state, session memory, context, and reasoning strategies.
• Agents depend on protocols, such as A2A and MCP, to effectively communicate. Models and agents need these protocols to manage multi-agent communication.

What is agentic AI?

Agentic AI is an artificial intelligence system made up of independent agents that can take initiative and perform sequences of actions to complete tasks by reasoning, learning, and adapting to changing circumstances.

Dynatrace Chief Technologist Alois Reitbauer described agentic AI this way:

“It’s really delegating a task to software the way you would delegate it to a human. Say if you wanted to do travel booking, give it some complexity and freedom and some decision points it can make. Like, I have to go to Vegas, I need a hotel, I need a couple of good restaurants to go to, we’re going to be 50 people, fix it with my schedule.” – Alois Reitbauer in The New Stack

Agentic AI systems rely on AI agents to perform the tasks that lead to the desired outcome.

What are AI agents?

An AI agent is a self-directed autonomous application that harnesses large language model (LLM) reasoning, tool usage, and context-awareness from numerous data sources to carry out tasks.

Agents can think and act independently without outside intervention. Agents can think through chain-of-thought, plan, execute (Reason+Act=ReAct), and refine their actions as needed. Businesses are looking into adopting these autonomous agents for applications such as customer service automation, supply-chain optimization, and content generation.

How do AI agents operate?

AI agents operate similarly to a Michelin-starred chef in a busy kitchen: They continuously gather information, plan, execute, and adjust to reach their desired end goal.

In the chef analogy, the cook surveys orders and available ingredients, decides on a suitable recipe, and then refines the approach based on feedback or resource constraints.

Agents do the same thing in a computational context. Specifically, they observe the world (for example, a user request or a set of data), perform internal reasoning about the best course of action, then carry out the steps needed to fulfill the request. This cycle allows them to respond adaptively to changing conditions, much as a chef would substitute ingredients or modify a dish mid-preparation.

Underpinning this iterative loop is the orchestration layer, which maintains the agent’s state, session memory, and reasoning strategies (such as ReAct, Chain-of-Thought, or Tree-of-Thoughts). Large language models (such as OpenAI’s GPT, Anthropic Claude, Google Gemini, Amazon Nova) provide the core reasoning capability for the agent. The model “thinks” about the user’s query. But the agent gains its power by incorporating additional frameworks or tools that can fetch external information or execute actions in the real world. One way to fetch and provide tools and information is through a unified protocol called Model Context Protocol (MCP).

Additionally, the orchestration layer ensures that multiple rounds of reasoning, tool usage, and tool outputs are all tracked and synthesized before the agent returns a final response to the user. Agents follow these steps in a structured way, so they can produce more accurate, context-rich answers and easily manage complex tasks.

What is the difference between models and agents?

A model (like a large language model) simply generates outputs based on its training data and the given prompt, typically without any built-in mechanism for session memory, external actions, or complex decision loops and validations.

An agent, on the other hand, includes the model but goes further. It maintains a stateful process (managing multi-turn conversations and thought processes), uses external tools to gather fresh data or perform actions, and follows a defined orchestration logic (such as ReAct and chain-of-thought). Thus, while a model is a core reasoning component, an agent adds the surrounding structure and capabilities needed for autonomous, goal-directed behavior.

What is Agent2Agent (A2A)? How multiple agents communicate with each other

As enterprises slowly adopt multiple specialized agents, interoperability of these services becomes crucial to create reliable experiences. To achieve this, A2A from Google helps to create an open protocol that enables agents—regardless of vendor or framework—to securely exchange information, coordinate actions, and integrate capabilities. By specifying tasks, capabilities, and artifacts in a standardized JSON-based lifecycle model, A2A fosters multi-agent collaboration across otherwise siloed systems.

A2A protocol enables agents to share updates and delegate tasks without overhead. However, direct communication between agents only solves half the problem: These agents also need relevant, up-to-date data and context to drive decisions and be equipped with the right toolset to execute actions.

Without a unified method for accessing diverse data sources, even the most capable multi-agent ecosystem remains limited in scope. The open-source project Model Context Protocol (MCP) fills this gap.

What is Model Context Protocol? How MCPs empower agents

As an open standard, the Model Context Protocol (MCP) connects AI agents to relevant data sources, such as repositories, tools, or external APIs. Instead of the above mentioned integrations for each data silo, MCP provides a universal interface like USB-C to connect multiple relevant sources to feed the right context to the models and agents. This universality simplifies how agents access relevant context, leading to better task outcomes, execution and more consistent performance across complex environments. For managing complex tasks like the ones highlighted above, the Dynatrace MCP server on GitHub helps to get real-time end-to-end observability and MCP data into your daily workflow.

What’s next: Monitoring A2A and MCP for better agentic AI

As these technologies evolve, we can expect deeper integrations between agent orchestration protocols (A2A and MCP) and open observability frameworks, delivering end-to-end visibility from data ingestion to cross-agent collaboration. Likewise, as standards converge, organizations will rapidly compose advanced AI solutions while retaining full transparency and control, paving the way for even greater scalability, resilience, and confidence in autonomous agents.

Read more

• Part two of the Rise of Agentic AI blog series explores AI agent observability and monitoring, A2A and MCP communications, and how to scale and monitor Amazon Bedrock Agents.
• Part three explains how to monitor Amazon Bedrock Agents and how observability optimizes AI agents at scale.
• Part four covers full-stack observability for AI with NVIDIA Blackwell and NVIDIA NIM.
• Part five demonstrates how to build a simple agentic application using the OpenAI Agents SDK and instrument the data with Dynatrace.
• Part six explores AI Model Versioning and A/B testing for smarter LLM services.
• Part seven introduces data governance and audit trails for AI services.

The post The rise of agentic AI part 1: Understanding MCP, A2A, and the future of automation appeared first on Dynatrace news.