From generic AI to task‑focused operational agents

Dynatrace ready‑made agents are purpose‑built capabilities that apply Dynatrace intelligence to specific, recurring operational tasks. Each agent focuses on a clearly defined problem, such as explaining why a service is slow, summarizing unusual behavior in an environment, or helping you understand what changed and why it matters. These agents are designed to take a question or a signal based on the exact data that is in your environment and organization and turn it into a useful answer you can act on.

Because Dynatrace agents are ready‑made, there is no need to define prompts, train models, or design behavior from scratch. Each agent already knows:

• What type of input to expect,
• Which Dynatrace signals and context it should use,
• And what output types are most useful for each addressed problem type.

All available ready-made Dynatrace agents can be found in Dynatrace Hub.

Trigger agent actions with Dynatrace Workflows and the Dynatrace MCP Server

Ready‑made agents can be triggered automatically as part of Dynatrace Workflows or available wherever you already work via the Dynatrace MCP Server.

Using agents in Dynatrace Workflows

Dynatrace Workflows lets you run agents in response to events or on a schedule. Instead of manually asking questions about potential problems and remediation steps, the workflow autonomously responds to changes in your environment.

For example, the Kubernetes Troubleshooting Agent runs nine parallel queries for data enrichment, and Dynatrace Intelligence turns all the information into a structured diagnosis. Customize the agents to your needs, including instructions for human approval steps and automated remediation.

The fastest way to get started is with Dynatrace ready-made agentic workflow templates, currently available in a preview release. Instead of building from scratch, you get proven automations that summarize issues, suggest remediation, and deliver insights directly to the tools your teams already use.

Power users can go further by building their own agentic workflows that combine Dynatrace Intelligence actions with any trigger, data source, or integration in Workflows. Use cases range from auto-scaling Kubernetes clusters based on Dynatrace Intelligence forecasts to generating query-cost-optimization recommendations for stakeholders, to virtually any other automation your environment requires.

Using agents through the Dynatrace MCP Server

The Dynatrace MCP Server makes the agents available outside the Dynatrace web UI, without requiring you to deploy or operate any additional infrastructure. You can connect Dynatrace to any MCP‑compatible client in minutes, with no server to install, host, or maintain.

Through the tools exposed by the MCP Server, you can use natural language to query data in Grail®, check system health, and get problem analyses and remediation recommendations. This brings Dynatrace directly into the tools you already use, such as your IDE, Claude Code and Cowork, Microsoft Copilot, Slack, or automation platforms like n8n. The Dynatrace MCP Server also powers integrations with systems like Azure SRE, AWS DevOps, GitHub Copilot, Atlassian Rovo Ops, Amazon Q, and others.

This means agents are no longer tied to a single interface. You can ask Dynatrace questions and get grounded, production‑ready answers wherever you work, using the same agents and intelligence that power Assist and workflows.

Dynatrace Assist: a simple way to test ready-made agents

The quickest way to use a ready‑made agent and see how it works before you start creating a workflow is with . Dynatrace Assist lets you ask questions about your environment using natural language, without switching tools or setting anything up.

A simple way to start is with a real problem you already have. For example, when a service becomes slow, open Assist and ask a question such as “Summarize the open problems and highlight those that need immediate attention.” Assist interprets the question, evaluates the environment you’re working in, and pulls together relevant data and context using Dynatrace Intelligence. Instead of manually navigating metrics, traces, logs, and dependencies, you get an explanation grounded in what is actually happening in your system.

Continuing your conversation with Assist, you can refine the question or follow suggested drill‑downs. Assist supports this as a single flow, helping you move from an initial question to deeper analysis and, where applicable, to next steps. You’re not configuring an agent or defining behavior. You’re simply asking a question and letting Dynatrace coordinate the right intelligence and ready‑made agents behind the scenes.

This makes Assist your lowest‑friction entry point for using Dynatrace agents. You get a concrete result quickly, using the same data and context you already rely on in your daily work.

What’s next?

If you haven’t already, open Dynatrace Playground, or your Dynatrace tenant, and ask Dynatrace Assist a question to see the ready-made agents in action.

The post Dynatrace AI agents begin working for you on day one, and are built to grow with you appeared first on Dynatrace news.

Imagine commissioning le Carré and Fleming to build your perfect undercover agent: quietly embedded in the system you’re watching. You hand in your mission brief, which includes the target, objective, and behaviors to track. Your agent observes without drawing attention, reporting insights back to you. On cue, the information flow you’ve carefully orchestrated turns signals into actionable intelligence that helps pre-empt risk.

Dynatrace doesn’t write spy fiction. However, even better, Dynatrace now lets you write your own smart agentic workflows that deliver intelligent reports and react to changes in your environment based on your objectives.

Adding generative AI to your workflow

Using the power of gen AI, Davis CoPilot® transforms your workflows into agentic instructions. Davis CoPilot lets you explore data using conversational language, translating complex data and queries into summaries, and provides intelligent recommendations across Dynatrace.

Integrated into Dynatrace Workflows, Davis CoPilot is your toolkit for building smart automations, bringing the power of generative AI into your mission-critical workflows.

By embedding Davis CoPilot in your workflows, you can associate any automation with any number of conversational automations. Your workflows can even perform actions autonomously when combined with precise Davis® AI forecasting, for instance, scaling resources based on forecast demands.

In real time, these workflows monitor live data, summarize critical issues, and identify remediation paths or emerging threats. When scheduled, these smart workflows help you outsource routine tasks, such as alerting stakeholders of costly queries.

Let’s look at some examples of how these smart workflows can help you in your daily work.

Build agentic workflows that respond to critical events

Proactive guiding through complex problem remediation

Let’s assume you want to build an automation that cuts through alert noise and analyzes a problem as it occurs, guiding you through the remediation. When a new problem is detected, Davis CoPilot extracts the problem details, summarizes the situation, and provides tailored remediation guidance. By embedding it into a smart workflow, you can select your preferred automation to automatically syndicate this information, populate a ticket in ServiceNow or Jira, or post it to a dedicated Slack channel.

See how you can set up a workflow automation that automatically sends summaries and remediation guidance when a new problem is detected.

Monitor emerging threats to help you orchestrate a response

Next, you can build an agentic workflow that helps you monitor emerging threats and assess their risk to your environment as vulnerabilities are detected in your tenant. In plain language, you instruct your agent to extract IOCs, query security events in your environment, and correlate them with observability data in your environment. Information provided by the external threat feed is automatically matched against the live context in your tenant. The agent has now collected all the necessary information and provides a reliable risk assessment, along with a plan to orchestrate a response, directly in your Slack channel, ensuring around-the-clock visibility and a rapid response.

Write the future: Build an agentic workflow that autonomously auto-scales your resources

Dynatrace helps you build agents that reason autonomously. The key is to deliver data as precise as Dynatrace forecast capabilities. In this example, we linked the power of Davis AI to forecast demand, with generative AI and GitHub automations. Davis AI predicts the number of resources the hyperscaler infrastructure will need based on forecasted demand. When Davis AI notices a scaling need, Davis CoPilot interprets the data and autonomously edits the manifest using the GitHub automation. Giving you one end-to-end workflow that automatically scales resources up or down based on forecasted needs. To see this in action, watch how this workflow autonomously edits a manifest based on Davis AI suggestions to auto-scale a Kubernetes cluster.

Schedule agentic workflows to optimize routine tasks

Do you feel like sleeping in a little later? Maybe stretching your lunch break a little longer? Running that extra hill without sacrificing your productivity? Scheduling Davis CoPilot into your smart workflow is a great way to automate recurring tasks and save time.

Build an automation that predicts resource consumption

A recurring challenge for SREs is analyzing the full environment to predict future bottlenecks or over-resourcing and continuously translating the data to update stakeholders. Even with great observability in place, you need to ensure that you interpret the data and make timely decisions to inform future provisioning.

By combining Davis AI forecasting automation with Davis CoPilot, you can build an agent that answers key questions, such as which workloads are most resource-intensive, which resources show the most variance, and which require frequent scaling. This automation is capable of highly reliable forecasts, even when data points are limited. The automation interprets the data and emails actionable recommendations directly to you and anyone else who needs to stay informed.

To see this in action, watch the section of this video that explores predicting resource consumption.

Smart workflows that optimize query costs

Scheduling tasks can even help you keep costs lean and efficient. For admins or budget owners, staying within financial limits while maintaining performance is a constant challenge. In this example, we built a smart workflow that identifies the top 20 most expensive queries from the last 24 hours. Davis CoPilot analyzes each query and sends optimization recommendations directly to the query authors via email.

To implement this yourself, tailored to the most expensive queries executed on your tenant, go to our documentation

Conclusion: Adapt your workflows to any stage of your automation journey

These are just a few examples; the applications for it are endless. We’ve designed this workflow action to cater to your organization’s automation appetite. You may want to transform how you keep business stakeholders informed about what’s happening in your environment, leveraging Dynatrace’s highly accurate insights, which are translated into plain language and actionable next steps.

Alternatively, you may be ready to transition towards autonomous operations, where automation not only supports but also acts in a controlled and reliable manner. Davis CoPilot embedded into your workflows opens the door to your agentic journey.

Start your agentic journey and join the Davis CoPilot for Workflows Preview

Davis CoPilot for Workflows is available as a Preview. Sign up now and see how generative intelligence embedded into your workflows transforms your automation. Today, it helps you react faster, optimize more effectively, and collaborate seamlessly. Tomorrow, it will go even further: anticipating needs, orchestrating actions, and enabling truly autonomous reasoning.

The post Write the future: Create your own agentic workflows appeared first on Dynatrace news.

Every organization is adopting GenAI across its infrastructure and application stacks. It’s important that IT operations teams seek a seat at the table because large swaths of models will be deployed across every technology. For example, the use of cloud migrations, GenAI large language models, small language models, and specialized models will drive productivity, cost savings, and business returns. Every customer is considering and attempting to measure their business returns from their AI investments; transparency into the data, system and model performance and drift, security, and quality are critical areas where IT operations, DevOps, SREs, and platform engineering teams can play a critical role in optimizing business returns and reducing business risks. So, where should you start the conversation?

Executives can use observability to reduce business risks and increase AI ROI by understanding how observability capabilities play a role in delivering across the core AI value categories of productivity, customer impact, cost optimization, innovation, and quality. For example, observability improves customer satisfaction by reducing the mean time to resolution and mean time to understanding. In addition, it can improve cross-team collaboration and data access to deliver cost efficiencies.

To reduce business risks and increase ROI in GenAI use cases, technology executives should plan to manage rising complexity, and, as part of continuous evaluation, executives should consider GenAI performance across the following dimensions:

• System performance: Monitoring the system performance of GenAI applications encompasses measuring operational performance characteristics similar to those of traditional applications, including at the software and infrastructure layers and the model. Model system performance monitoring includes the measurement of metrics such as model response latency, error rates (including failure to respond), and API failures.
• Quality performance: It is crucial for organizations to monitor the output quality of GenAI and AI applications. Quality includes accuracy of responses and model drift, where data used to train models no longer produces accurate or relevant results.
• Governance: Model governance of GenAI often encompasses monitoring and enforcing legal requirements and the organization’s ethics policies. Ongoing monitoring is necessary, including the adoption of guardrails to prevent the delivery of outputs that don’t comply with laws or company policies.
• Security: In addition to the theft of private information or loss of intellectual property, organizations must protect against security risks that are specific to GenAI applications. Prompt injection and jailbreaks are two emerging attacks. Monitoring tools that detect these and other security issues are critical to risk management.
• Cost: Monitoring the cost of delivering a GenAI application is a multitiered undertaking. Depending on the application, organizations may incur costs for each query and response to a model, in addition to costs associated with the underlying infrastructure required to deliver the application. The ability to collect the right cost information and analyze it on a per-application basis will be key to the ability of an organization to determine ROI.

Organizations must base the measurement of each performance dimension on its ability to derive outcomes that drive business value. Each GenAI application should support a targeted outcome, such as improved productivity, increased revenue, new revenue streams, or enhanced customer satisfaction. Connecting the dots between GenAI performance dimensions and business value requires defining measurements that matter to the business and collecting, correlating, and analyzing the data to understand the app’s ability to deliver that value.

For technology executives, AI observability is fast becoming essential for managing the operational complexity and business outcomes from AI initiatives. It provides the visibility needed to demonstrate ROI, ensure reliable AI applications, and make informed decisions based on critical data that supports every AI use case.

The post Optimizing AI ROI from DevOps and IT Operations: The rising need for AI/LLM observability appeared first on Dynatrace news.

We’re excited to announce that Davis CoPilot Chat is now available across the Dynatrace platform. Davis CoPilot™, launched in October 2024 to support Dynatrace users with access to their data, now extends across the platform, streamlining user onboarding and providing comprehensive support and contextual insights from various Dynatrace® Apps. With the new Davis CoPilot conversational interface, users can leverage natural language to quickly get answers to their questions, making it easier than ever for users to interact with Dynatrace.

Intuitive access to information boosts team productivity

We understand that taking advantage of the numerous features and functionalities offered by platforms like Dynatrace can be challenging. To help you navigate this and boost your efficiency, we’re excited to announce that Davis CoPilot Chat is now generally available (GA). This new feature provides information and guidance exactly when and where you need it, making your Dynatrace experience smoother and more efficient.

Davis CoPilot leverages the power of generative AI to answer your questions through a globally accessible chat interface. We’re proud to say that Davis CoPilot is multilingual: you can ask questions and get answers in many different languages, including French, Spanish, German, Portuguese, Chinese, Japanese, and, of course, English. Davis CoPilot provides immediate, accurate responses, eliminating the need for extensive searches and reducing dependency on support channels. This makes knowledge more readily available and boosts productivity and user experience for both new and experienced users.

Davis CoPilot Chat follows our recent announcement of the general availability of Quick Analysis in Notebooks and Dashboards, which makes data accessible to technical and non-technical users alike. This means you can interact with data stored in the Dynatrace Grail™ data lakehouse just by using natural language.

Simplify onboarding and quickly find what you’re looking for with Davis CoPilot

You can start using the Davis CoPilot conversational interface immediately. Simply enable Davis CoPilot and assign the relevant user permissions, and the Davis CoPilot button will appear in the Dock.

Start a new conversation with Davis CoPilot Chat by selecting it in the Dock or by pressing CTRL/CMD + I and entering your question.

New users can quickly get up to speed with Dynatrace by asking Davis CoPilot for help with basic commands, setup instructions, and troubleshooting tips. This reduces the learning curve and enables new users to become productive faster. The conversational interface provides step-by-step guidance, making the onboarding process smoother and more efficient.

If you’re already familiar with Dynatrace, you can rely on Davis CoPilot to provide detailed explanations for a wide range of expert questions related to exploring new use cases, advanced configuration topics, and building custom apps.

Here are some examples of questions you can ask Davis CoPilot:

• Onboarding: How do we start sending OpenTelemetry data to Dynatrace?
• Understanding Dynatrace: What is the difference between an event and a problem in Dynatrace?
• Exploring Dynatrace solutions: How can we comply with the Digital Operational Resilience Act (DORA) using Dynatrace?
• Configuring your environment: How do I set up an alert based on an anomaly detector?
• Developing custom apps: How can I import external table data and visualize it using the Dynatrace App Toolkit?

Get contextual assistance at the press of a button

Davis CoPilot seamlessly integrates into our use-case-specific Dynatrace Apps, offering you contextual insights and guidance at the press of a button. While we plan to release additional contextual app integrations in the coming months, several will be available a few weeks after launch, allowing Davis CoPilot to provide you with insights into:

• Kubernetes warning signals
• Individual problem details and the relationships between problems
• Database performance optimization

Simplify Kubernetes: Davis CoPilot decodes warning signals

Understanding the background and root cause of warnings often requires in-depth subject matter expertise. That’s why we integrated Davis CoPilot into Kubernetes. Instead of manually looking up error messages, Davis CoPilot translates warning signals into clear, understandable language. In addition, Davis CoPilot offers a list of typical root causes and related remediation steps. This way, newcomers can quickly become proficient, and experts can elevate their expertise to hero status.

Problems demystified: Davis CoPilot provides insights into root causes

In Problems, Davis CoPilot provides clear summaries of problems, their root causes, and the suggested remediation steps. Davis CoPilot explains individual issues in clear language from the problem details page and can perform a comparative analysis when multiple problems are selected from the list view. This helps you identify common root causes and propose corrective steps without relying on a team of experts and waiting for hours for critical insights. If you want to learn more, have a look at Wolfgang Beer’s latest blog post and learn more about recent advancements in the Problems app.

Optimize database performance: Understand query execution plans

Query execution plans provide detailed information on how a database will execute an SQL query. While these provide the raw data on how to improve query performance and reduce resource consumption, they require expert knowledge to read and interpret. Now, in Databases, Davis CoPilot can provide natural language explanations of execution plans, breakdowns of relevant details, and recommendations on how to improve statement performance. This gives non-expert database users, such as developers, the knowledge they need to optimize their application performance and database utilization.

Empower your teams with Davis CoPilot today

The launch of Davis CoPilot Chat marks the second milestone of our journey. We’re committed to continuously enhancing the assistant’s capabilities with upcoming features, including query explanations, workflow actions, and troubleshooting guides.

Get started with Davis CoPilot today and transform how you and your teams interact with Dynatrace:

• Try Davis CoPilot Chat on the Dynatrace Playground
• Activate Davis CoPilot in your environment and grant your users access.

Thanks for joining us on this exciting journey. We look forward to your feedback and to seeing how Davis CoPilot helps your teams achieve their goals.

Davis CoPilot Chat, as well as the Dynatrace Apps integrations mentioned in this blog post, will be available starting with the release of Dynatrace SaaS version 1.307.

The post Davis CoPilot expands: Get answers and insights across the Dynatrace platform appeared first on Dynatrace news.

“Engineers today lack an easy way to track the tokens and prompt usage of their LLM applications in production. By using OpenLLMetry and Dynatrace, anyone can get complete visibility into their system, including gen-AI parts with 5 minutes of work.”

Nir Gazit, CEO and Co-Founder Traceloop

Why AI model observability matters

The adoption of LLMs has surged across various industries, particularly since the introduction of OpenAI’s GPT model. While these models yield impressive results, the challenge of maintaining their operation within defined boundaries has increased.

AI model observability plays a crucial role in achieving this by addressing these key aspects:

1. Model performance and reliability: Evaluating the model’s ability to provide accurate and timely responses, ensuring stability, and assessing domain-specific semantic accuracy.
2. Resource consumption: Observing computational resource availability and saturation, whether deployed in cloud-native environments like Kubernetes or CPU-enabled servers.
3. Data quality and drift: Monitoring the quality and characteristics of training and runtime data to detect significant changes that might impact model accuracy.
4. Explainability and interpretability: Providing information on model versions, parameters, and deployment schedules, which is essential for interpreting and understanding model answers.
5. Security and compliance: Actively preventing security threats at both the application and model levels to ensure responsible and compliant AI usage.

The challenge of AI model observability

One challenge in AI model observability is the diverse tooling landscape required to gain critical insights. OpenTelemetry has become a standard for collecting traces, metrics, and logs. However, seamless support for various SDKs and AI model frameworks, such as LangChain and Pinecone, remains essential.

Combining Dynatrace with Traceloop’s OpenLLMetry addresses the heterogeneity challenge by supporting a range of popular LLMs, prompt engineering, and chaining frameworks. OpenLLMetry, an open source SDK built on OpenTelemetry, offers standardized data collection for AI Model observability.

How OpenLLMetry works

OpenLLMetry supports AI model observability by capturing and normalizing key performance indicators (KPIs) from diverse AI frameworks. Utilizing an additional OpenTelemetry SDK layer, this data seamlessly flows into the Dynatrace environment, offering advanced analytics and a holistic view of the AI deployment stack.

Given the prevalence of Python in AI model development, OpenTelemetry serves as a robust standard for collecting observability data, including traces, metrics, and logs. While OpenTelemetry’s auto-instrumentation provides valuable insights into spans and basic resource attributes, it falls short in capturing specific KPIs crucial for AI models, such as model name, version, prompt and completion tokens, and temperature parameters.

OpenLLMetry bridges this gap by supporting popular AI frameworks like OpenAI, HuggingFace, Pinecone, and LangChain. Standardizing the collection of essential model KPIs through OpenTelemetry ensures comprehensive observability. The open source OpenLLMetry SDK, built atop OpenTelemetry, enables thorough insights into your Large Language Model (LLM) applications.

As the collected data seamlessly integrates with your Dynatrace environment, you can analyze LLM metrics, spans, and logs in the context of all traces and code-level information. Maintained under the Apache 2.0 license by Traceloop, OpenLLMetry is a valuable asset for product owners, providing a transparent view of AI model performance.

The diagram below illustrates how OpenLLMetry captures and transmits AI model KPIs to your Dynatrace environment, empowering your business with unparalleled insights into your AI deployment landscape.

Dynatrace OneAgent® is perfectly capable of automatically injecting and tracing code-level information for many technologies, such as Java, .NET, Golang, and NodeJS. However, Python models are trickier.

In the Dynatrace web UI, you can track your AI model in real time, examine its model attributes, and assess the reliability and latency of each specific LangChain task, as demonstrated below.

The captured span by Traceloop automatically displays vital details, including the mode utilized by our LangChain model gpt-3-5-turbo, the model’s invocation with a temperature parameter of 0.7, and the utilization of 53 completion tokens for this individual request.

With the growth of AI, maintaining transparency is essential

Observing AI models like Large Language Models (LLMs) in production is crucial for enhancing performance, reliability, security, and user trust. This includes the monitoring of AI-related costs to ensure they remain within acceptable margins. The Dynatrace platform, coupled with Traceloop’s OpenLLMetry OpenTelemetry SDK, offers comprehensive visibility from model inception to completion.

As AI adoption grows, maintaining transparency is essential for regulatory compliance. While Dynatrace automates tracing for various technologies, Python-based AI models require OpenTelemetry. OpenLLMetry bridges this gap, supporting popular AI frameworks and vendors to ensure standardized data collection. OpenLLMetry provides an open source SDK for LLM observability, seamlessly integrating with Dynatrace for in-depth analysis.

References

• Dynatrace Documentation: AI model observability
• Traceloop documentation: OpenLLMetry

The post Enhanced AI model observability with Dynatrace and Traceloop OpenLLMetry appeared first on Dynatrace news.

AI observability is becoming imperative as businesses in all sectors are introducing novel approaches to innovate with generative AI in their domains. Advanced AI applications using OpenAI services don’t just forward user input to OpenAI models; they also require client-side pre- and post-processing. A typical design pattern is the use of a semantic search over a domain-specific knowledge base, like internal documentation, to provide the required context in the prompt. This is achieved by using OpenAI services to compute numerical representations of text data that ease the computation of text similarity, called “embeddings,” for the documents as well as for the user input.

Furthermore, tools like LangChain leverage large language models (LLM) as one of their basic building blocks for creating AI agents (think of AI agents as APIs that perform a series of chat interactions that target a desired outcome) which perform complex and potentially large queries against an LLM like GPT-4. They then connect to third-party services such as online calculators, web search, or flight status information to combine real-time information with the power of an LLM.

One of the crucial success factors for delivering cost-efficient and high-quality AI-agent services following the approach described above is using AI observability to closely observe their cost, latency, and reliability.

Dynatrace enables enterprises to automatically collect, visualize, and alert on OpenAI API request consumption, latency, and stability information in combination with all other services that are used to build AI applications. This includes OpenAI as well as Azure OpenAI services, such as GPT-3, Codex, DALL-E, or ChatGPT.

AI observability example: OpenAI token consumption

Our example dashboard below visualizes OpenAI token consumption. It shows critical SLOs for latency and availability, as well as the most important OpenAI generative AI service metrics, such as response time, error count, and the overall number of requests.

Dynatrace OneAgent^® discovers, observes, and protects access to OpenAI automatically, with no manual configuration, revealing the full context of used technologies, service interaction topology, security-vulnerability analysis, and the observability of all metrics, traces, logs, and business events in real time.

How Dynatrace traces OpenAI model requests

Let’s use a simple NodeJS example service to show how Dynatrace OneAgent automatically traces OpenAI model requests. OpenAI offers an official NodeJS language binding that allows the direct integration of a model request by adding the following lines of code to your own NodeJS AI application:

const { Configuration, OpenAIApi } = require("openai");

const configuration = new Configuration({

apiKey: process.env.OPENAI_API_KEY

});

const openai = new OpenAIApi(configuration);

const response = await openai.createCompletion({

model: "text-davinci-003",

prompt: "Say hello!",

temperature: 0,

max_tokens: 10,

});

Once the AI application is started on a OneAgent-monitored server, the application is automatically detected, and the traces and metrics for all outgoing requests are collected. OneAgent automatic injection of monitoring and tracing code works not only for the NodeJS language binding but also when using the raw HTTPS request in NodeJS. While OpenAI offers official language bindings only for Python and NodeJS, there is a long list of community-provided language bindings.

OneAgent can automatically monitor all C#, .NET, Java, Go, and NodeJS bindings. However, we recommend following the OpenTelemetry approach to monitoring Python with Dynatrace.

The screenshot below shows the traces that OneAgent collects, along with all the latency and reliability measurements for each of the outgoing GPT model requests.

Dynatrace further refines the OpenAI calls by automatically splitting specific services for the OpenAI domain, as shown below.

Once this is done, the Dynatrace Service Flow shows the flow of your requests, starting with your NodeJS service and calling the OpenAI model, as shown below.

As shown in the example above, Dynatrace OneAgent automatically collects all latency and reliability-related information along with all the traces showing how your OpenAI requests traverse your service graph.

The seamless tracing of OpenAI model requests allows operators to identify behavioral patterns within their AI service landscape and to understand the typical load situation of their infrastructure.

This AI observability knowledge is essential for further optimizing the performance and cost of services.

By adding some lines of manual instrumentation to a NodeJS service, cost-related measurements are also picked up by OneAgent, collecting the number of OpenAI conversational tokens used.

Observing OpenAI request cost

Each request to an OpenAI model, such as text-davinci-003, gpt-3.5-turbo, or GPT-4 reports back how many tokens were used for the request prompt (the length of your text question) and how many tokens the model generated as a response.

OpenAI customers are billed based on the total number of tokens consumed by all the requests they make. By extracting these token measurements from the returning payload and reporting them through Dynatrace OneAgent, users can observe token consumption across all OpenAI-enhanced services in their monitoring environment.

Here is the instrumentation used to extract the token count from the OpenAI response and to report the three measurements to the local OneAgent:

function report_metric(openai_response) {

var post_data = "openai.promt_token_count,model=" + openai_response.model + " " + openai_response.usage.prompt_tokens + "\n";

post_data += "openai.completion_token_count,model=" + openai_response.model + " " + openai_response.usage.completion_tokens + "\n";

post_data += "openai.total_token_count,model=" + openai_response.model + " " + openai_response.usage.total_tokens + "\n";

console.log(post_data);

var post_options = {

host: 'localhost',

port: '14499',

path: '/metrics/ingest',

method: 'POST',

headers: {

'Content-Type': 'text/plain',

'Content-Length': Buffer.byteLength(post_data)

}

};

var metric_req = http.request(post_options, (resp) => {}).on("error", (err) => { console.log(err); });

metric_req.write(post_data);

metric_req.end();

}

After adding these lines to your NodeJS service, three new OpenAI token consumption metrics are available in Dynatrace, as shown below.

Davis AI automatically detects ChatGPT as the root-cause

One of the superb features of Dynatrace is Davis® AI, which automatically learns the typical behavior of monitored services. Once an abnormal slowdown or increase of errors is detected, Davis AI triggers root cause analysis to identify the cause.

Our simple example of a NodeJS service entirely depends on the ChatGPT model response. So, whenever the latency of the model response degrades or the model request returns an error, Davis AI automatically detects it.

In the example below, Davis AI automatically reported a slowdown of the NodeJS prompt service and correctly detected the OpenAI generative service as the root cause of the slowdown.

The Davis problem details page shows all affected services for which the OpenAI generative service was the root cause of the slowdown, along with the ripple effects of the slowdown.

The problem details also list all Service Level Objectives that were negatively impacted by the slowdown.

AI observability with Dynatrace brings peace of mind when using OpenAI models

The massive popularity of generative AI cloud services, such as OpenAI’s GPT-4 model, is forcing companies to rethink and redesign their existing service landscapes. Integrating generative AI into traditional service landscapes comes with all kinds of uncertainties. Using AI observability from Dynatrace to observe OpenAI cloud services helps you gain cost transparency and ensure the operational health of your AI-enhanced services.

Also, full transparency and observability of AI services will play a significant role in upcoming AI regulations at a national level and for risk assessments within your own company.

For further details, you can view the full source of the NodeJS service on GitHub.

The post Dynatrace automatically monitors OpenAI ChatGPT for companies that deliver reliable, cost-effective services powered by generative AI appeared first on Dynatrace news.