The growing complexity of modern multicloud environments has created a pressing need to converge observability and security analytics.

Security analytics is a discipline within IT security that focuses on proactive threat prevention using data analysis. As attackers become more skilled, threats can become more detrimental to organizations if they go undetected. A proactive approach to application security is essential: by constantly collecting and analyzing data, security analytics enables teams to catch problems before they escalate.

In the past five years, delivering innovation more efficiently has remained at the forefront of customer demand. As environments began to scale with cloud-native and microservices architectures to meet this demand, security approaches didn’t evolve accordingly. The result: Environments are more vulnerable to threats and more difficult to secure. In fact, according to a recent survey, nearly 70% of chief information security officers agreed that vulnerability management has become more difficult as the complexity of their software supply chain and cloud ecosystem has increased.

The ripple effect of increased risk compounds the problem. With more alerts and greater difficulty identifying false positives, security teams experience frustration and burnout. A lack of common tooling, common language, and collaboration further inhibits productivity and prolongs remediation.

At the 2024 Dynatrace Perform conference in Las Vegas, Gerhard Byrne, Dynatrace principal product manager, and Susan St. Clair, principal security solutions engineer, discussed how organizations can take a more proactive approach to threat detection and incident resolution.

During their breakout session, Byrne and St. Clair demonstrated how the Dynatrace platform accelerates remediation by enriching security data with observability context and actionable insights to protect environments against exploitation or lateral movement.

Threat hunting expectations vs. reality

In a perfect world, threat hunting and incident resolution would be a linear, straightforward process. Ideally, after fetching data and filtering, an organization could enrich the findings with observability data to get better insights into the nature of the alert. With these insights, the team could identify the threat and understand the nature of the incident. This allows them to react accordingly and return the system to a secure state.

But with the complexity of modern cloud environments — including the associated software supply chains and siloed toolchains — and the increasing sophistication of today’s attackers, threat hunting is unpredictable. In reality, security teams aren’t aware of all the unknown unknowns in their environments. After fetching, filtering, and enriching information with observability data, security teams might change their hypothesis about what’s occurring. This revision of assumptions might happen multiple times, causing engineers to lose track of previous hypotheses, patterns, and evidence. Keeping threats documented is a challenge: Engineers typically open numerous tabs to maintain context, which is tedious and can create error. Remediating a vulnerability can thus take far longer than anticipated, which can be detrimental when the risk is high.

“As defenders, we need to embrace different paths and possibilities like our adversaries are doing today,” Byrne said. “Just going down a checklist will not help you find new threats.”

Streamline threat hunting and accelerate resolution with Dynatrace Security Investigator

During the conference, Dynatrace announced the new Security Investigator app on the platform. The app enables security teams to investigate threats faster, obtain accurate and observability-enriched results, and maintain context throughout the weaving paths on which security investigations might lead.

Security Investigator demo

St. Clair began her demonstration of the app with the following scenario: She receives a Slack alert that an anomaly was detected and there has been unauthorized access to a Kubernetes cluster monitored via OneAgent.

To begin, St. Clair filtered the alert for a time window of a couple of hours to focus her analysis. Using Dynatrace Query Language in Grail, St. Clair determined what log data was available to her. With each execution, data appears in a query tree. “As I’m building out this investigation, each of these nodes is being created for me automatically,” she said. “As part of that documentation, I can easily go back and forth to see what was executed.”

St. Clair then used the Dynatrace Pattern Language (DPL) to make the data more usable. She used the DPL Architect to create her own patterns in addition to the platform’s out-of-the-box patterns for parsing the data. After running an audit log pattern, she wanted to understand why she received the Slack alert and which object the attacker had accessed so she could then focus on the unauthorized responses. Running this query, suspicious IPs arose, and she saved them as evidence in a new folder she created within the app.

As she continued to execute queries, St. Clair’s hypothesis began to change. She saw substantial traffic in a specific port, which was not necessarily malicious, but it was abnormal enough to warrant additional investigation. Fortunately, the query tree automatically creates new “branches” to support changing hypotheses and help engineers keep track of evidence and patterns. The query tree thus frees security professionals from tedious manual documentation, allowing them to focus entirely on finding the unknown unknown.

Finally, St. Clair found running malware on the system, pivoting the investigation from the initial authentication alert. By the end of an investigation, she had a visual representation of the process from start to finish.

“I can keep track of where I went. [The data is] documented, shareable, collaborative, and available for further investigation,” St. Clair said.

The road ahead: enriched security analytics with Dynatrace

Organizations can benefit most from their security investigations using the abundant data in the Dynatrace platform. The platform’s broad and deep observability identifies where problems initiate as well as their dependencies using PurePath. Teams can use Real User Monitoring and Session Replay to track user-facing activity in real time. This helps them understand how an attacker accessed an application, how they interacted with it, how traces originated, and more. Teams can also create management reports and use Dynatrace Notebooks to easily share their security investigation data with their peers.

“Security is a team sport,” Byrne said. “The next time you’re in a war room, you can be the person who provides insights and conclusions based on the wealth of data that is available at your fingertips with Dynatrace platform.”

The post Advanced security analytics to resolve incidents quickly and streamline threat hunting appeared first on Dynatrace news.

Business and technology leaders are increasing their investments in AI to achieve business goals and improve operational efficiency. From generating new code and boosting developer productivity to finding the root cause of performance issues with ease, the benefits of AI are numerous. However, without observability, the AI wave—which has become more than just hype—comes with risks.

The first risk is not adopting AI at all. Organizations that miss out on implementing AI risk falling behind their competition in an age where software delivery speed, agility, and security are crucial success factors. But organizations must also be aware of the pitfalls of AI: security and compliance risks, biases, misinformation, and lack of insight into critical metrics (including availability, code development, infrastructure, databases, and more).

At the 2024 Dynatrace Perform conference in Las Vegas, Dynatrace chief technology strategist Alois Reitbauer joined founder and chief technology officer Bernd Greifeneder to discuss how organizations can mitigate their risk and embrace AI properly. Reitbauer delved into key use cases for Dynatrace hypermodal AI before Greifeneder introduced the platform’s new AI observability capability that will revolutionize how organizations can maximize value from their AI implementations.

Expanding hypermodal AI for all use cases with Davis CoPilot™

The Dynatrace composite approach to AI, known as hypermodal AI, enables organizations to extract the maximum ROI from their data efficiently and cost-effectively. Hypermodal AI combines predictive, causal, and generative AI so that organizations can not only predict behavior and identify root causes but also effortlessly store, query, and access their data using natural language and at a massive scale.

Moreover, with Davis CoPilot™, organizations can use generative AI to harness the full extent of hypermodal AI for every use case. “Every new question we ask comes with additional analysis, prediction models, and more,” Reitbauer said. “By packaging [these capabilities] into hypermodal AI, we are able to run deep custom analytics use cases in sixty seconds or less.”

Performance analytics

Dynatrace hypermodal AI empowers development teams to dig deep into database statements and remediate issues quickly. The first use case Reitbauer presented outlines a developer who hears from the database team that a problem might be occurring with the database. The developer opens a Dynatrace notebook, creates a CoPilot section, and asks for all the database statements executed over the past 72 hours using a natural language query.

In response, CoPilot delivers the data showing the variety of statements executed over the given timeframe. Armed with this data, the developer can then change the query to ask for the statements over a period (for example, the previous week). CoPilot then delivers the data, enabling the developer to easily identify regressions, increases, and more.

Security analytics

Davis CoPilot proactively mitigates risk by enabling security teams to validate hypotheses and hunt for threats before they impact the organization or end users. The second use case involves a security engineer who becomes aware of a new threat and wants to know if any of the organization’s systems might be affected. The engineer can efficiently access this data via a natural language query in a CoPilot Notebook: “Summarize all MITRE security events of the last 72 hours.”

In this example, there is a suspicious increase in scripting events. The key advantage of hypermodality is that users can frictionlessly switch from business events to logs to drill down further into their data. After updating the query to ask for log data, the engineer was able to identify attack attempts.

Experience analytics

Hypermodal AI also helps teams analyze issue reports, write scripts, and pinpoint root cause to maintain superior customer experiences. In the third use case, site reliability engineers (SREs) use Davis CoPilot to examine the number of problems over time to understand user impact. An SRE can translate a DQL query from a colleague into natural language, modify it, and examine the result. CoPilot efficiently provides valuable problem data so the SRE can then collaborate with the customer experience team to remediate the issue.

FinOps

The fourth use case illustrates how FinOps can also benefit from hypermodal AI. With as a significant priority for business and technology leaders, FinOps engineers can use CoPilot to understand the number of nodes on a Kubernetes cluster. Generative AI partners with predictive AI to deliver insights into the number of nodes for a specific cluster they will need in the future, thus improving efficiency while ensuring a frictionless customer experience.

Business workloads

The final use case considers an e-commerce business whose analyst team wants to understand the minimum daily number of orders they need to fulfill, in addition to predicting how much packaging they will need for an average week. First, they import their order data into the Dynatrace platform using the business events capability. From there, they can use predictive AI to forecast their daily workload. They can also modify the query to view their predicted workload on a weekly basis, forecast order amounts, and efficiently plan their packaging.

Mitigate risk for all use cases with AI observability

Reitbauer’s exploration of hypermodal AI and its variety of applicable use cases demonstrated that AI can deliver exponential value. As organizations consider the next AI services they will build, visibility into their implementations is critical to ensuring success.

At Perform 2024, Dynatrace announced the immediate availability of AI Observability, a capability that enables organizations to observe how their AI implementations are behaving. The complexity of modern cloud environments necessitates broad and deep observability into every layer of the tech stack to mitigate risk and deliver software quickly and securely. AI implementations are no exception.

AI Observability provides insights into all layers of the tech stack for which organizations might be using AI, including infrastructure, GPUs, semantics, vector databases, orchestration, and more. This also comes with context into the rest of your stack, with 700+ integrations out of the box: “[With AI observability,] you can observe not only the AI itself, but the applications and services that you bring to your customers,” Greifeneder said.

But contextual analytics don’t stop here. “AI Observability brings the ROI back to the business,” Greifeneder continued. “You can figure out if [your AI implementations] are bringing the value, customer experience, performance, and efficiency you need.”

AI observability is also a critical capability because of the increasing risk of duplicated code that comes with generative AI implementations. “Generated code becomes less maintainable. It poses more security risks due to its convenience,” said Greifeneder. “Dynatrace observes these applications that have been created with or augmented by generated code.” The Dynatrace composite approach to AI thus keeps applications secure while allowing organizations to reap the benefits of artificial intelligence in a variety of use cases.

Embrace AI completely, properly, and confidently with AI observability

As organizations navigate AI and explore how they can use the technology for their unique business goals, the Dynatrace platform empowers organizations to embrace AI properly. With its combination of predictive, causal, and generative AI, Dynatrace hypermodal AI enables cost-effective and efficient processes that promote collaboration, swift issue resolution before they reach end users, and the delivery of superior customer experiences.

The post Mitigating risk with AI observability: Dynatrace empowers organizations to embrace AI for all use cases appeared first on Dynatrace news.

Cloud complexity and data proliferation are two of the most significant challenges that IT teams are facing today. Computing environments are scaling to new heights, resulting in more data that makes pinpointing root causes and vulnerabilities even more challenging. Modern cloud complexity is becoming nearly impossible for human beings to manage without AI and automation. Application observability helps IT teams gain visibility in their highly distributed systems, but what is developer observability and why is it important?

In a recent webinar, Dynatrace DevOps activist Andi Grabner and senior software engineer Yarden Laifenfeld explored developer observability. Grabner and Laifenfeld discussed how observability and dynamic debugging together empower developers to get the most out of the observability data that Dynatrace provides.

Why is developer observability important for engineers?

“Observability is about answering questions,” said Laifenfeld. “DevOps, SREs, developers… everyone will ask questions. Observability is about answering.”

The challenges that developers face with modern cloud environments are myriad. The scale and the highly distributed systems result in enormous amounts of data. When an incident occurs, developers need to know what data to look at, where the incident occurred, and other relevant metrics. Manually sifting through data to answer these questions is time-consuming and takes time away from innovation. These environments are also highly dynamic: “With environments constantly spinning up and down, a problem that occurred five minutes ago may no longer be relevant,” Laifenfeld said.

Observability is necessary for understanding complex environments and making sense of immense amounts of data. But not all teams use the same observability data in the same way. Laifenfeld described observability as an onion: each layer represents a different degree of granularity that different teams consider important. “The DevOps people looking end-to-end. They don’t care about individual services, but rather how they interact and go together,” she said. “They also care about infrastructure: SREs require system visibility and incident management. But developers need code-level visibility and code-level data.”

With traditional monitoring tools, the granular data that developers require typically involves manual preparation. This includes defining what classifies as an error, identifying the data from the error that will fix the issue, and prepping logs before deployment. “That’s not how I envision code-level observability,” Laifenfeld said. “Developer observability, as I see it, is pressing a button and getting observability right at your fingertips. I think Dynatrace and Rookout together are going to enable this future.”

Developer observability, Kubernetes, and expanding left

The shift left movement has changed developers’ scope of responsibility. As software delivery tasks traditionally performed at the end of the software delivery lifecycle – such as testing and deployment – have shifted to the beginning of the lifecycle, developers are now working with Kubernetes more directly. Laifenfeld argued that developers shouldn’t bear the burden of the additional workload when their focus is their code: “Learning Kubernetes as a developer is not easy,” she said. “Developers don’t need more responsibilities; they need more capabilities. More ways to understand their environments and what’s going on.”

Developer observability supports the idea of expanding left. Laifenfeld connects with the expand left concept because it makes unknown concepts more accessible to developers. “Observability should be easy, comfortable, and intuitive,” she said. “If developers are used to working in their integrated development environments, observability should bring unknown concepts to them and make it feel [accessible]. You won’t need to know Kubernetes to understand what’s going on with your code in Kubernetes.”

KubeCon North America is this week. Laifenfeld and Grabner are excited to attend and discuss all things observability. With topics ranging from best practices to cloud cost management and success stories, the conference will be a valuable resource for understanding observability and getting started.

Developer observability use cases

Dynatrace makes achieving a unified view of your environment easy. As Grabner put it, the “secret sauce” is AI-powered automatic data discovery. “Dynatrace [works by] ingesting all your data, understanding where it comes from, extracting metadata, analyzing it for you, and giving you the data you need whenever you need to fix a problem,” he said. By ingesting data from either cloud-native sources (such as OpenTelemetry or Kubernetes events) or what you already have instrumented (such as Open Standards), Dynatrace enriches data with context to provide a 360º view of your environment.

Opening the Dynatrace playground tenant, Grabner walked through some key observability use cases.

Data at your fingertips

Say you would like to focus on a specific Kubernetes workload called “adservice.” Dynatrace allows you to easily pull up that entity and understand key context such as metadata, where it’s running, when there was a deployment, and more. In this example, Grabner saw that the adservice workload was running on EKS and could see the relevant metrics, logs, services, events, error logs, and more.

Easily identify problems, receive context, and understand impact

Identifying and solving problems is nearly impossible with massive volumes of data to sift through. Instead of making problem-solving like finding a needle in a haystack, Dynatrace makes it easy by identifying problems for you. When Davis AI detects a problem, teams can easily filter the data and identify the issue and where it occurred.

In Grabner’s example, he could see there was a JavaScript error increase and could understand the impact of the error. “I could see that 116 users were affected and that there were 2,000 calls to my API,” he said. “[Dynatrace] tells me which app was affected, the root cause, and what led to the problem.” Enriching problem data with context allows teams to not only understand the scope and impact of the problem, but also drill down to understand the impact of a failing component on other dependencies.

Detect anomalies automatically

The Dynatrace platform uses seasonal baselining to automatically detect performance spikes or degradations in your apps and services. Dynatrace learns your environment by understanding baseline performance on certain days of the week or times of the month. When an anomaly crops up, the platform automatically detects the performance change and notifies you immediately.

In Grabner’s example, he understood that there was an increased Java error rate on the front end of the application. How do you know if this problem has business impact? This is where service-level objectives (SLOs) come in. Dynatrace enables teams to specify SLOs, such as latency, uptime, availability, and more. Therefore, when a problem arises, teams can easily identify if the problem affects these objectives. “I call this pre-crime alerting,” said Grabner.

Dynamic debugging

Developers can leverage Dynatrace to understand code-level problems and debug them without stopping a program from running. Laifenfeld used a to-do list app as an example. “I have a list of everything I have connected to the platform, and I write in what I want to debug,” she said. “Then, I add a breakpoint. A breakpoint won’t stop your program but will collect local variables, stack trace, process metrics, etc., and bring that to you while your program continues to run.”

“This gives us the whole picture of what happens in production,” Laifenfeld continued. “It’s dynamic, smartly detects problems, and once we add the breakpoint, it tells us what’s happening, giving us what we need and when we need it.” By delivering code-level context and easy, live debugging to developers, Dynatrace empowers developers with the capabilities they need to deliver high-quality software, faster.

Unlocking a 360º view with unified observability

As digital transformation continues picking up speed and multiclouds become more complex, understanding your environment is critical now more than ever. By bringing observability capabilities to developers, teams can unlock unprecedented insights from their services and applications.

The post Application observability meets developer observability: Unlock a 360º view of your environment appeared first on Dynatrace news.