In the previous blog post of this series, we discussed the crucial role of Dynatrace as an orchestrator that steps in to stop the testing phase in case of any errors. Additionally, Dynatrace equips SREs and application teams with valuable insights powered by Davis^® AI. In this blog post of the series, we will explore the use of Site Reliability Guardian (SRG) in more detail.

SRG is a potent tool that automates the analysis of release impacts, ensuring validation of service availability, performance, and capacity objectives throughout the application ecosystem by examining the effect of advanced test suites executed earlier in the testing phase.

Validation stage overview

The validation stage is a crucial step in the CI/CD (Continuous Integration/Continuous Deployment) process. It involves carefully examining the test results from the previous testing phase. The main goal of this stage is to identify and address any issues or problems that were detected. Doing so reduces the risk of production disruptions and instills confidence in both SREs (Site Reliability Engineers) and end-users. Depending on the outcome of the examination, the build is either approved for deployment to the production environment or rejected.

Challenges of the validation stage

In the Validation phase, SREs face specific challenges that significantly slow down the CI/CD pipeline. Foremost among these is the complexity associated with data gathering and analysis. The burgeoning reliance on cloud technology stacks amplifies this challenge, creating hurdles due to budgetary constraints, time limitations, and the potential risk of human errors. Additionally, another pivotal challenge arises from the time spent on issue identification. Both SREs and application teams invest substantial time and effort in locating and rectifying software glitches within their local environments. These prolonged processes not only strain resources but also introduce delays within the CI/CD pipeline, hampering the timely release of new features to end-users.

Mitigate challenges with Dynatrace

With the support of Dynatrace Grail™, AutomationEngine, and the Site Reliability Guardian, SREs and application teams are assisted in making informed release decisions by utilizing telemetry observability and other insights. Additionally, the Visual Resolution Path within generated problem reports helps in reproducing issues in their environments. The Visual Resolution Path offers a chronological overview of events detected by Dynatrace across all components linked to the underlying issue. It incorporates the automatic discovery of newly generated compute resources and any static resources that are in play. This view seamlessly correlates crucial events across all affected components, eliminating the manual effort of sifting through various monitoring tools for infrastructure, process, or service metrics. As a result, businesses and SREs can redirect their manual diagnostic efforts toward fostering innovation.

1. Configure an action for the Site Reliability Guardian in the workflow. The action should focus on validating the guardian’s adherence to the application ecosystem’s specific objectives (SLOs). Additionally, align the action’s validation window with the timeframe derived from the recently completed test events.
   
2. As the action begins, the Site Reliability Guardian (SRG) evaluates the set objective by analyzing the telemetry data produced during advanced test runs. At the same time, SRG uses DAVIS_EVENTS to identify any potential problems which could result in one of two outcomes.
   
   Outcome #1: Build promotion
   Once the newly developed code is in line with the objectives outlined in the Guardian—and assuming that Davis AI doesn’t generate any new events—the SRG  action activates the successful path in the workflow. This path includes a JavaScript action called promote_jenkins_build, which triggers an API call to approve the build being considered, leading to the promotion of the build deployment to production.
   
   Outcome #2: Build rejection
   If Davis AI generates any issue events related to the wider application ecosystem or if any of the objectives configured from the defined guardian are not met, the build rejection workflow is automatically initiated. This triggers the disapprove_jenkins_build  JavaScript action, which leads to the rejection of the build.
   
   Moreover, by utilizing helpful service analysis tools such as Response Time Hotspots and Outliers, SREs can easily identify the root cause of any issues and save considerable time that would otherwise be spent on debugging or taking necessary actions.  SREs can also make use of the Visual Resolution Path to recreate the issues on their setup or identify the events for different components that led to the issue. In both scenarios, a Slack message is sent to the SREs and the impacted app team, capturing the build promotion or rejection.The telemetry data’s automated analytics, powered by SRG and Davis AI, simplify the process of promoting builds. This approach effectively tackles the challenges that come with complex application ecosystems. Additionally, the integration of service tools and Visual Resolution Path helps to identify and fix issues more quickly, resulting in an improved mean time to repair (MTTR).

Validation in the platform engineering context

Dynatrace—essential within the realm of platform engineering—streamlines the validation process, providing critical insights into performance metrics and automating the identification of build failures. By leveraging SRG and Visual Resolution Path, along with Davis AI causal analysis, development teams can quickly pinpoint issues, and further rectify them ensuring a fail-smart approach. The integration of service analysis tools further enhances the validation phase by automating code-level inspections and facilitating timely resolutions. Through these orchestrated efforts, platform engineering promotes a collaborative environment, enabling more efficient validation cycles and fostering continuous enhancement in software quality and delivery.

In conclusion, the integration of Dynatrace observability provides several advantages for SREs and DevOps, enabling them to enhance the key DORA metrics:

• Deployment Frequency: Improved deployment rate through faster and more informed decision-making. SREs gain visibility into each stage, allowing them to build faster and promptly address issues using the Dynatrace feature set.
• Change Lead Time: Enhanced efficiency across stages with Dynatrace observability and security tools, leading to quicker postmortems and fewer interruption calls for SREs.
• Change Failure Rate: Reduction in incidents and rollbacks achieved by utilizing “Configuration Change” events or deployment and annotation events in Dynatrace. This enables SREs to allocate their time more effectively to proactively address actual issues instead of debugging underlying problems.
• Time to restore service: While these proactive approaches can help improve Deployment Frequency and Change Lead Time, telemetry observability data with Dynatrace AI causation engine Davis AI can aid in improving Time to restore service.

In addition, Dynatrace can leverage the events and telemetry data that it receives during the Continuous Integration/Continuous Deployment (CI/CD) pipeline to construct dashboards. By using JavaScript and DQL, these dashboards can help generate reports on the current DORA metrics. This method can be expanded to gain a better understanding of the SRG executions, enabling us to pinpoint the responsible guardians and the SLOs managed by various teams and identify any instances of failure. Addressing such failures can lead to improvements and further enhance the DORA metrics. Below is a sample dashboard that provides insights into DORA and SRG execution.

In the next blog post, we’ll discuss the integration of security modules into the DevOps process with the aim of achieving DevSecOps. Additionally, we’ ll explore the incorporation of Chaos Engineering during the testing stage to enhance the overall reliability of the DevSecOps cycle. We’ll ensure that these efforts don’t affect the Time to Restore Service turnaround build time and examine how we can improve the fifth key DORA metric, Reliability.

What’s next?

Curious to see how it all works? Contact us to schedule a demo and we’ll walk you through the various workflows, JavaScript tasks, and the dashboards discussed in this blog series.

If you’re an existing Dynatrace Managed customer looking to upgrade to Dynatrace SaaS, see How to start your journey to Dynatrace SaaS.

The post Automate CI/CD pipelines with Dynatrace: Part 4, Validation stage appeared first on Dynatrace news.

Managing a Dynatrace Platform Subscription (DPS) budget requires balancing your organization’s usage of Dynatrace capabilities against a pre-defined annual budget commitment. Without adequate flexibility in the subscription model, your organization might fail to benefit from capabilities that could transform your observability and security processes. Conversely, with too much flexibility, costs may exceed your budget.

Subscription administrators typically limit team access and capability usage to avoid billing surprises. This approach is time-consuming and requires ongoing management. Also, if limits are set too low, some critical components in your infrastructure might go unmonitored, potentially negatively impacting your business. While if limits are set too high, you might pay for more monitoring than you need and exceed your budget. While there is no penalty for exceeding your DPS budget (the same rate card is applied for on-demand usage without additional fees or premium pricing), administrators need the ability to manage their subscriptions within a planned budget.

Cost monitors notify you of changes to your forecast usage

Cost monitors offer a different approach to these challenges. By using predictive AI with smart forecast algorithms that predict usage, you’re notified whenever forecasted usage is projected to exceed your defined budget or when unexpected usage spikes occur.

Cost monitors run in the background daily, automatically monitoring usage forecasts and costs. They notify you when unusual forecasts and cost events occur so you can focus on monitoring your applications, not your subscription. Cost monitors automatically notify subscription administrators without manual setup or configuration. However, you retain full control over configured thresholds and notification paths if you desire to change them. They even provide APIs so you can integrate them into your monitoring ecosystem as needed.

Smart forecasting capabilities

Cost monitors leverage the smart forecasting capabilities of the Dynatrace Platform Subscription to predict your costs through the end of your subscription period. The budget summary includes a median forecast value with an upper and lower range for each day of your subscription into the future. By monitoring the forecast cost for the end of your subscription, you can proactively plan for growth and be notified of spikes in usage to stay on top of daily spending. Usage is forecast daily and you’re notified of significant changes or forecasts that exceed your defined threshold values. This allows you to plan and make changes accordingly.

Cost monitors also track daily costs at the capability and environment levels, alerting you when daily costs exceed predicted levels. There are no thresholds to manage for cost events; cost monitors are designed to ensure that all meaningful increases are caught and notified without generating false alerts. This predictive AI technique allows you to stay on top of daily costs and be notified of unexpected cost increases.

Cost monitors also generate email notifications for specified recipients when forecast and cost events occur. By default, you are notified of all forecast and cost events. License administrators can additionally define a specific email distribution list for these notifications. This approach ensures that a responsible team member is notified about all subscription-relevant events without any upfront configuration. Proactive alerting can be used to draw attention to unusual usage in non-production environments or an unexpected increase in use for a specific capability, allowing you to plan or manage accordingly.

What’s next

Dynatrace will continue to monitor and fine-tune the cost-event detection algorithms used by cost monitors to ensure that notifications are accurate.

If you’re already using the new Dynatrace Platform Subscription model (available as of April 26, 2023), cost monitors are already running on your account! For full details on how to get the most from them, please see our Cost monitor documentation.

Additional information about the Dynatrace Platform Subscription model is also available in our DPS documentation. If you’re interested in migrating to a Dynatrace Platform Subscription, please contact a Dynatrace account representative.

The post Avoid billing surprises with smart Dynatrace cost monitors appeared first on Dynatrace news.

Without native log support, overhead and complexity grow

OpenTelemetry, the Cloud Native Computing Foundation (CNCF) incubating project, introduced standards that enable companies to instrument, generate, and export telemetry data. When combined with out-of-the-box correlation, such telemetry data provides context-rich observability. Dynatrace has supported the OpenTelemetry project for years as a key contributor and contributed to its rise to a popular open source observability framework for cloud-native software. Many global enterprises have instrumented their code to emit traces, metrics, and logs in a standardized and vendor-neutral way using OpenTelemetry.

While ingestion of OpenTelemetry traces and metrics into Dynatrace is supported, companies often prefer to collect logs in the OpenTelemetry format. Earlier this year, OpenTelemetry announced that their logs API/SDK specification is stable, making it ripe for broader adoption. This enables unified observability because logs are indispensable for troubleshooting apps, monitoring infrastructure, auditing or investigating security incidents, tracking business events, and many other use cases.

Without such a holistic view of system behavior and performance, organizations typically need to invest in separate instrumentation and integration efforts for each telemetry type, which brings additional overhead, costs, and complexity.

Unify OpenTelemetry logs, traces, and metrics in Dynatrace

Dynatrace now includes full support for OpenTelemetry logs, which provides unified observability for organizations with vendor-neutral and open-source tech stacks. Our commitment to this open standard allows you to cover all three pillars of observability with minimal configuration effort because OpenTelemetry traces, metrics, and logs can be exported to Dynatrace using the same OTLP exporter.

By ingesting OTLP logs into Dynatrace, you can utilize the Grail™ data lakehouse and its massively-parallel processing analytics engine. This allows you to eliminate log forwarding and collection solutions, which not only add maintenance overhead and complexity but can also become bottlenecks in performance and log volume throughput.

With the added support of logs to OpenTelemetry traces and metrics, Dynatrace now gives you a unified and holistic overview of observability signals, with integrated linking of traces and logs. This enables you to connect the traces in your stack directly with root-cause information in logs. One customer recently shared why working with all telemetry signals together really makes sense for them.

Native support for OpenTelemetry (OTLP) logs also supports enterprises that have highly diverse technical architectures. While Dynatrace OneAgent^® is often the preferred way of discovering and ingesting logs from traditional hosts or Kubernetes environments, there are certain environments for which OneAgent is not a viable option. As an alternative, OpenTelemetry lets you extend Dynatrace technology coverage with log data.

Generic ingest of log data now works with OTLP

Ingesting OTLP logs is now supported via the OpenTelemetry Logs Ingest API. In SaaS deployments, you can use this approach to ingest log data into Grail and analyze it via Log Management & Analytics in your environment. The same API is also available for Log Monitoring Classic for Dynatrace Managed deployments (via an Environment ActiveGate).

All you need to do is configure your OpenTelemetry collector or any other OTLP log source to send logs to the OpenTelemetry Logs Ingest API endpoint.

Start using the full OpenTelemetry set today

• OpenTelemetry log ingest is supported in Dynatrace version 1.270 and Environment ActiveGate version 1.269
• Read more about OpenTelemetry logs in Dynatrace, ingest transformations and start ingesting logs over the API

The post OpenTelemetry logs in Grail unlock full observability appeared first on Dynatrace news.

Organizations in every industry are engaged in some form of digital transformation, integrating technology into all areas of the business. Digital transformation strategies are fundamentally changing how organizations operate and deliver value to customers. Some of the benefits organizations seek from digital transformation journeys include the following:

• Increased DevOps automation and efficiency. Digital tools and technologies provide a more efficient way of doing things. They help organizations streamline and automate complex and time-consuming procedures and improve overall performance.
• Improved customer experience. Consumers expect personalized, proactive, and convenient service, which traditional application architectures often struggle to provide.
• Competitive advantage. A comprehensive digital transformation strategy can help organizations better understand the market, reach customers more effectively, and respond to changing demand more quickly.
• Enhanced business operations. Digitizing internal processes can improve information flow and enhance collaboration among employees. With teams all using the same data, it’s easier for the organization to make data-driven decisions.

However, digital transformation requires significant investment in technology infrastructure and processes. It often involves replacing legacy systems and workflows that have been in place for years or even decades. Because it’s an ongoing process, digital transformation requires that teams continually adapt and evolve as new technologies emerge and customer expectations change. As a result, a successful digital transformation strategy requires that organizations make changes to organizational culture and provide employee training.

In a panel discussion at Perform 2023, Debbie Umbach, vice president of corporate marketing at Dynatrace, explored the successful digital transformation strategies of three Dynatrace customers: Best Buy, a multinational consumer electronics retailer; a top global retail and corporate finance service provider in the banking industry; and the Department of Veterans Affairs (VA), the second largest U.S. federal agency.

Customer Panel: Digital Transformation

Watch video

Digital transformation challenges: Goals for a successful digital transformation journey

Every organization faces unique challenges. Finding the solutions to these challenges becomes the goals for each panelist’s successful digital transformation strategy.

The global banking leader is seeking better ways to detect and manage alerts from multiple sources, improve platform resilience, and to better understand the business cost of a technical failure.

The goals of VA are to streamline operations and become more proactive so they can head off problems before their users find them, thus creating delightful user experiences for their diverse community.

Best Buy is designing its journey to cut through the noise of its multicloud and multi-tool environments to immediately pinpoint the root causes of issues during peak traffic loads.

Digital transformation security risks

Increasingly, observability of DevOps workflows is converging with application security. The proof point for one organization is Log4Shell, a critical zero-day vulnerability discovered in a popular Java library in 2021.

With Dynatrace Application Security, VA was able to immediately detect whether the vulnerability was present in any of its systems. Additionally, contextualized real-time dependency mapping and automatic analysis enabled them to quickly assess the risk and take action to protect user data and critical infrastructure.

Digital transformation success stories: Operational benefits

In addition to immediate threat response, VA teams were also able to unlock more automation and reduce mean time to repair (MTTR) so they could resolve issues before they affected users or compromised systems.

Best Buy automates service level objectives (SLOs) for milestones such as staging gates and version control, which enables them to streamline DevOps workflows and cut red tape.

An additional game changer for Best Buy is Dynatrace anomaly detection and tool consolidation. Previously, they had 12 tools with different traffic thresholds. With Dynatrace, they now have a single unified platform that measures all the thresholds looking for differences, which makes it easy to find a performance spike in a torrent of data from different sources.

Advice for a successful digital transformation strategy

In the process of their digital transformation journeys, the panelists learned some lessons they wanted to pass on. After getting the right observability and analytics platform in place, the primary key to success is enabling teams to access it en masse. Their advice includes the following practices:

• Get technical and platform teams on board. With the key players advocating organizational change, teams can adopt new technologies and methods faster.
• Enable teams with ongoing education so they get the most out of Dynatrace capabilities. This keeps teams from falling into old habits and enables them to continually innovate and expand.
• Turn off the tools you no longer need. Although teams often get attached to old tools and methods, it’s easier, in the long run, to stop using them.
• Take advantage of Dynatrace University. The 3–5-minute videos make it easy for teams to quickly learn how to do things at their own pace as they’re solving problems.
• Don’t let perfect get in the way of good. Whether it’s cloud migration or monitoring, don’t be afraid to try something. It’s better to get some traction and fail fast, then recover and continue to build.

Watch the full session to learn more about how these organizations use Dynatrace to execute their digital transformation strategies. Customer panel: Successful digital transformation journeys

The post Digital transformation strategies: Success stories from three digital transformation journeys appeared first on Dynatrace news.

Service-level objectives help IT teams define technical success and align with top-line business objectives.

But not all service-level objectives (SLOs) are created equal. So how do development and operations (DevOps) teams and site reliability engineers (SREs) distinguish among good, great, and suboptimal SLOs? In the 2023 Perform session “SLOs done right: A practitioners guide,” Michael Cabrera, SRE lead at Vivint, and Andreas Grabner, DevSecOps activist at Dynatrace, break down the state of SLOs and discuss how teams can adopt successful SLOs, avoid less-than-ideal objectives, and ultimately build better SLOs.

The state of service-level objectives

While SLOs play a critical role in helping DevOps and SRE teams align technical objectives with business goals, they’re not always easy to define. In many cases, issues are tied to operational overload. Enterprises now have access to myriad metrics they can track and measure, but an abundance of choice doesn’t equal actionable insight.

Indeed, 54% of SREs say they handle too many metrics, making it increasingly difficult to find the most relevant ones for a particular service, according to the Dynatrace State of SRE Report. 22% of SREs said they don’t know what makes a good SLO, while 18% said they don’t know which metrics to track. The same number said they’re not sure how or where to get started.

The result? From issues with an overwhelming set of metrics to problems with implementation, 99% of SREs said they encounter challenges when defining and creating SLOs.

The four components of a good SLO

To help SREs and DevOps teams create and deploy effective SLOs, Cabrera and Grabner examine the following four factors that make a good SLO:

1. Monitors signals

The first attribute of a good SLO is the ability to monitor the four “golden signals“: latency, traffic, error rates, and resource saturation. While not every SLO must report on all these signals simultaneously, good service-level objectives help teams uncover and understand how at least one of the four signals can affect current operations.

2. Includes the end user

While SLOs for back-end silos and processes help eliminate internal issues, back-end-based objectives tell only half the story. As a result, effective SLOs must account for end users. “You might have an internal dashboard where everything is green and it makes you feel good,” Grabner noted. “But on the outside, people are suffering. That means you can’t define SLOs purely on back-end metrics.”

3. Bridges the business gap

Good SLOs should also help bridge the gap between chief executive officers’ concerns and those of chief technology officers. By providing data relevant to both parties — such as user adoption metrics for CEOs and application crash data for CTOs — organizations can find common ground.

4. Prioritizes incident response

SLOs should target problems that represent potential business impact. This enables teams to prioritize incident response and ensure immediate issues are resolved before working to address more long-term concerns.

SLOs that teams should avoid

Any SLO that accurately measures and reports data can benefit organizations. In practice, however, SLOs’ value varies significantly based on how teams design, deploy, and manage them.

Grabner and Cabrera offer the example of an iOS app experiencing crash issues after a team deploys a new version. Dynatrace OneAgent provided information about failure rates, latency, and throughput, along with iOS data for users, crashes, and error rates. Creating objectives tied to these metrics alone, however, didn’t benefit end users. Put simply, these metrics are the symptom rather than the root cause; knowing how many crashes have taken place and how often provides evidence that applications or other IT systems aren’t working as intended, but it doesn’t offer insight to help fix the problem at scale.

As a result, SLOs can’t simply alert on metrics. Instead, they must alert on both front- and back-end processes to help teams understand where problems occur and how they’re related, setting the stage for meaningful correction.

How to build a better SLO

So, how do teams build a better SLO?

Building better SLOs starts by understanding the dual nature of these service-level objectives. To be effective, SLOs must deliver business and technical results. Cabrera and Grabner cited the example of a tandem approach in monitoring application performance. On the technical side, SLOs account for metrics such as availability, response time, and error rate. On the consumer side, SLOs focus on application adoption, overall application rating, and the total number of mobile crashes.

By combining these data sets, SRE and DevOps teams can make decisions that improve back-end performance and end-user experience. When it comes to creating a better SLO, “Make it visible and tangible,” Grabner advised.

It’s also worth considering implementing automation to help scale SLOs and encourage shift-left operations that allow teams to quickly and easily build their own SLO frameworks.

The post SLOs done right: how DevOps teams can build better service-level objectives appeared first on Dynatrace news.

There are many definitions of environmental sustainability, most of which converge on a common theme. Collectively and individually, we have a responsibility to act to protect global ecosystems and support health and wellbeing, now and in the future. Sustainability seeks a balance between human activity and the natural world.

As a result, environmental sustainability is one of the three pillars of environmental, social, and governance (ESG) initiatives. ESG embodies a set of criteria that guides an organization’s strategy, measures the organization’s impact, and informs potential investors across these three pillars. Specifically, the environmental criteria focus on an organization’s utilization of natural resources and the resulting impact on the environment. McKinsey summarizes the importance of this focus. “Every company uses energy and resources; every company affects and is affected by the environment,” the article states. ESG metrics are increasingly important to investors as they evaluate risk. In turn, these metrics are increasingly important to organizations because they measure and disclose their performance.

Balancing act or tipping point?

What motivates us to act? The answer, to varying degrees, is favorable investment potential, regulatory requirements, competitive advantage, cost savings, and moral imperative. And the time to act is now. In fact, in an article for the Wall Street Journal, Gartner analyst Stephen White anticipates that organizations will rapidly adopt performance metrics tied to energy consumption. “Sustainability is increasingly becoming a board-level issue with broad companywide mandates flowing down from chief executives to tech leaders,” White says.

Regulatory mandates are another driver that organizations can’t ignore. As a result, the Securities and Exchange Commission (SEC) in the US and the European Parliament in the EU are adopting stricter reporting rules that will apply to a larger number of companies. Globally, the International Sustainability Standards Board (ISSB), established at the UN Climate Change Conference in Glasgow, has developed requirements for climate-related disclosure. Sustainability reporting informs investors about sustainability risks and helps focus a company’s sustainability actions.

Your IT carbon footprint

Corporate carbon footprint calculations consist of multiple facets, including transportation, waste management, fuel, and electricity consumption. To that end, measuring, understanding, and reporting results are important precursors to intelligent reduction actions.

From an IT perspective, the sustainability focus is on carbon emissions from electricity consumption, specifically related to on-premises, hybrid, and multicloud computing. Some interesting facts:

• Moving a workload to the cloud can reduce its carbon footprint by up to 96%.
• Cloud computing has a greater carbon footprint than the airline industry.
• Average cloud server idle time exceeds 70%.

Yet most organizations don’t have the tools to measure, much less reduce, their IT carbon footprint.

Measuring your carbon footprint: from intention to action

Cloud providers offer tools to measure carbon emissions from the use of their cloud services. But these tools don’t support multicloud environments, and they don’t account for the footprint of on-premises services. More importantly, these tools are fundamentally backward-looking. They lack both the time and dimensional granularity required to derive carbon-emission analytics and optimization insights. Environmental sustainability emphasizes our collective responsibility to take action. For this, we need real-time intelligence and analytics, not just historical reports.

As executive mandates reach technology leadership teams, your organization’s carbon reduction goals will become more tangible. As a result, you’ll need to shift from intent to action, from passive reporting to active reduction initiatives.

Introducing Carbon Impact

The Carbon Impact app from Dynatrace measures and reports the carbon footprint of all Dynatrace-monitored hosts across your entire hybrid and multicloud environment in a single interface. The app translates utilization metrics, including CPU, memory, disk, and network I/O, into their CO2 equivalent (CO2e). Dynatrace engineers developed the app using guidance from the Sustainable Digital Infrastructure Alliance (SDIA), expanding on formulas from Cloud Carbon Footprint.

With Carbon Impact, you can explore the sources of your IT carbon footprint, regardless of where your workloads run. The app automatically identifies opportunities to reduce carbon emissions, capturing the details needed to analyze and make informed decisions. It also generates granular reports and host-level details to provide a focus for carbon reduction initiatives.

The Carbon Impact dashboard: Your IT carbon footprint overview

The Carbon footprint summary reports total CO2e emissions for the selected and preceding timeframes for quick interval-based comparisons.

Optimization summaries report idle instances and under-utilized instances. The app derives thresholds that influence these calculations from Google Compute Engine recommendations, although you have full configuration control to adjust these thresholds to your needs.

The table breaks down emissions by data center, listing your cloud and on-premises instances. All your carbon footprint data is in one place, and the measurement methodology is consistent across all your data centers.

Two charts complete the dashboard. The first shows accumulated carbon footprint and energy consumption over time. As cloud providers decarbonize their data centers, or as you move your workloads to more carbon-efficient locations, the slope of the carbon emission trajectory should change to illustrate the impact. The second chart allows you to compare carbon emissions over time with a business metric of your choice derived from any of your configured business events.

Instances: Where should you focus your attention?

The Instances view details energy and CO2e consumption per host instance. Filters help narrow your focus. For example, you could view underutilized instances in a specific AWS data center or top CO2e emitters within a specific host group. Because Carbon Impact automatically connects to Dynatrace Smartscape® topology modeling, it’s easy to drill into host and process details. Or you can open Notebooks for ad hoc analysis, arming your teams with the insights needed to evaluate opportunities to reduce carbon emissions.

Carbon Impact and Dynatrace AppEngine

Carbon Impact tracks, reports, and helps you reduce the carbon footprint of your cloud and on-premises infrastructure. It also provides data center, host, process, and application details to help you understand where to focus your efforts. Moreover, it provides recommendations to help you get started quickly. Significantly, Carbon Impact supports multiple audiences, including compliance teams as a source for reporting, business teams that need investment guidance, and operations teams that need to act to reduce their carbon footprint.

Carbon Impact is a purpose-built app we created using the new Dynatrace AppEngine. It addresses a well-defined business need, bringing custom analytics to Dynatrace data stored in Grail. Carbon Impact is available now on the Dynatrace Hub. To learn more, please contact your account team.

The post From observability to sustainability: Reduce your IT carbon footprint with Dynatrace Carbon Impact appeared first on Dynatrace news.

How do you get more value from petabytes of exponentially exploding, increasingly heterogeneous data? The short answer: The three pillars of observability—logs, metrics, and traces—concentrated in a data lakehouse.

As observability and security data converge in modern multicloud environments, there’s more data than ever to orchestrate and analyze. The goal is to turn more data into insights so the whole organization can make data-driven decisions and automate processes.

Dynatrace Chief Technology Officer Bernd Greifeneder and his colleagues discuss how organizations struggle with this problem and how Dynatrace is meeting the moment.

Grail data lakehouse delivers massively parallel processing for answers at scale

Modern cloud-native computing is constantly upping the ante on data volume, variety, and velocity. As teams try to gain insight into this data deluge, they have to balance the need for speed, data fidelity, and scale with capacity constraints and cost. To solve this problem, Dynatrace launched Grail, its causational data lakehouse, in 2022. Grail combines the big-data storage of a data warehouse with the analytical flexibility of a data lake.

“With Grail, we have reinvented analytics for converged observability and security data,” Greifeneder says. With a data lakehouse approach, Dynatrace captures all the data in one place while retaining its context and semantic details. These details include all the relationships among data from different sources. This unified approach enables Grail to vault past the limitations of traditional databases.

From this central platform, teams can use the Dynatrace query language (DQL) to build flexible queries that reveal the stories hidden in their data. “This means you can ask any question anytime, all the unknown unknowns,” Greifeneder says.

Grail is schemaless, indexless, and lossless. With no schema, you don’t have to think of all the questions you want to ask of your data ahead of time. And with no indexes, teams can access data faster without having to maintain a massive index that quickly falls out of date. Because the data is lossless, teams from across the organization can query the data with its full details in place. This completeness meets the needs of IT professionals and business analysts alike.

“Grail can store and process 1,000 petabytes per day,” Greifeneder explains. And without the encumbrances of traditional databases, Grail performs fast. “In most cases, especially with more complex queries, Grail gives you answers at five to 100 times more speed than any other database you can use right now.”

Three pillars of observability converge with context and business data at the massive scale of Grail

The Dynatrace Software Intelligence Platform has always brought the full value of logs, metrics, and traces together with context and user data for AI-powered analysis. Now, that same full-spectrum value is available at the massive scale of the Dynatrace Grail data lakehouse. “You’re getting all the architectural benefits of Grail—the petabytes, the cardinality—with this implementation,” Greifeneder says, including the three pillars of observability: logs, metrics, and traces in context.

Logs on Grail

Log data is foundational for any IT analytics. Logs on Grail, included in the 2022 release, enables an endless variety of log-based use cases. For example, these include verifying app deployments, isolating faults coming from a single IP address, identifying root causes of traffic spikes, or investigating malicious user activity.

The Business Events capability enables business analysts to get the real-time insights and broad context they need to answer questions their business intelligence tools can’t.

But logs are just one pillar of the observability triumvirate.

Metrics on Grail

“Metrics are probably the best understood data type in observability,” says Guido Deinhammer, CPO of infrastructure monitoring at Dynatrace. Simply put, metrics are the counts and measures that teams calculate or aggregate over time as the basis for analysis.

But the process of aggregating data in traditional environments results in losing fidelity. These are crucial details that can help analysts understand data’s relevance in different scenarios, Deinhammer says.

In addition, the scale of relationships among data in microservices environments is crippling conventional databases. “Today, you can have a microservice environment with hundreds of thousands of ports talking to 100,000 other ports, each of which is recycled every couple of minutes,” Deinhammer explains. “This creates such high cardinality that it blows up existing metrics databases.”

With its massively parallel architecture, Grail can ingest and process high cardinality data at the scale of modern cloud architectures without losing fidelity. And with automatic analysis from Dynatrace Davis AI, Deinhammer continues, “if there is a problem, you get answers at a level of detail that allows you to automate remediation well before your end users notice.”

Traces on Grail

Traces are another area where cloud-native technologies have complicated application and microservices monitoring. Distributed traces are the path a transaction takes as it touches applications, services, and infrastructure from beginning to end.

Open source solutions are also making tracing harder. “OpenTelemetry, for example, is a way to get metrics, logs, and traces into an observability platform like Dynatrace,” explains Florian Ortner, CPO of applications and microservices at Dynatrace. “With OpenTelemetry, you have spans that provide links and attributes. But the sheer number can be extreme—up to 10 million spans per second.”

Another big trend is Kubernetes and container-based computing, both in the cloud and on premises. “Kubernetes makes spans longer,” Ortner explains. “Microservices get smaller, and you have more hops and additional tiers, such as service meshes.”

As a centralized data platform, Dynatrace makes it possible to keep ever-lengthening spans intact. With Grail, teams can track increasingly long and complex distributed traces with full fidelity and context at a massive scale.

Smartscape on Grail: Context at scale

Without a centralized view of what’s happening in a distributed multicloud environment, teams lack situational awareness.

Smartscape interactive topology mapping visualizes the dynamic relationships among all application components across every tier. It gives teams visibility into the dependencies throughout their environment. “With Smartscape on Grail with DQL, you can run queries across multiple data types throughout your entire topology,” Greifeneder says. This access gives teams an unprecedented ability to answer more complex questions than they could before.

Smartscape with DQL also adds new possibilities for automation. “If a team builds a new service or changes an API, how do you know it’s not breaking a dependency you didn’t know about?” Greifeneder asks. “Now, you can ask Grail this question and embed it into your automation so you can have a much smarter and more intelligent rollout. There are so many opportunities. Grail and DQL will give you new superpowers.”

Unleashing a new era of collaborative analytics on Dynatrace Grail

Having the full spectrum of observability data and the AI-driven value of the Dynatrace platform running on the unified Grail data lakehouse unlocks a whole new era of collaborative analytics for users across the organization. To make that collaboration even easier, Greifeneder and his colleagues announced several new UI and analytics capabilities, including the following:

• an enhanced UI that makes it easier to scan machine data and drill down on individual data points without having to switch contexts;
• new dashboarding capabilities that draw data from everywhere and enable analysts to fine-tune data views using markdown, custom code, and variables; and
• a new Notebooks feature that enables cross-functional teams to create custom analytics for all kinds of use cases.

For more about the new Notebooks feature, see “Dynatrace makes it easy for observability, security, and business data analysis.”

Learn more about the announcements at Perform 2023 in the Perform 2023 Guide: Organizations mine efficiencies with automation, causal AI.

The post Data lakehouse innovations advance the three pillars of observability for more collaborative analytics appeared first on Dynatrace news.

Powered by Grail and the Dynatrace AutomationEngine, Site Reliability Guardian helps DevOps platform teams make better-informed release decisions by utilizing all the contextual observability and application security insights of the Dynatrace platform. The app also enables SREs to set and automate service-level objectives (SLOs) for critical services. Site Reliability Guardian further simplifies automation requirements by using entity and ownership information, alerting the right teams when regressions are detected. Dynatrace users can now unlock answer-driven automation to safely and securely release new features to customers while ensuring production SLOs are intact.

Streamline development and delivery processes

Nowadays, digital transformation strategies are executed by almost every organization across all industries. In the context of application development, digital transformation aims to streamline development and delivery processes to ensure quick delivery to the market with guaranteed high quality and security standards.

To achieve this, many organizations are adopting DevOps practices to provide developers with a delivery platform to release their applications and services autonomously and independently. While this empowers teams to frequently deliver new features, the overall business, security, and quality objectives must be maintained. This is where Site Reliability Engineering (SRE) practices are applied. SREs use Service-Level Indicators (SLI) to see the complete picture of service availability, latency, performance, and capacity across various systems, especially revenue-critical systems. Consequently, Service-Level objectives (SLO) are defined to enact countermeasures before the business is impacted.

Given the momentum of DevOps and SRE, digital transformation goals can be achieved when automation enables organizations to apply best practices rapidly and to keep pace with the scale of the organization and applications.

Automation is key for effective collaboration

While DevOps engineers rely on observability and automation to determine whether they can promote a new release into production, SREs look for answers when a new release has a positive or negative impact on application performance, affects user experience, or threatens other SLOs. SREs face ever more challenging situations as environment complexity increases, applications scale up, and organizations grow:

• Growing dependency graphs result in blind spots and the inability to correlate performance metrics with user experience.
• Siloed teams and multiple tools make it difficult to align on a single version of the truth for overall system health.

Other observability solutions don’t provide the required automation capabilities that would allow:

• Automating and speeding up the SLO validation process and quickly reacting to regressions detected in application topology.
• Informing the right people with the answers they need to implement targeted countermeasures.

Automate the validation of key objectives

Dynatrace evolves Cloud Automation release validation (powered by Keptn) into Site Reliability Guardian and natively enriches the Dynatrace platform by automating change impact analysis. For each service, a guardian can automatically validate service-level objectives, “golden signals,” or security vulnerabilities before and after any deployment or configuration changes. It detects regressions and deviations from previously observed behavior, including latency, traffic, error rates, saturation, security coverage, vulnerability risk levels, and memory consumption. Thus, Site Reliability Guardian supports DevOps and SREs in speeding up release delivery and improving release quality.

• DevOps achieve safer and more secure releases by applying a gating mechanism that identifies release issues quickly and prevents poor-quality code from being promoted to production.
• SREs leverage on-demand reliability validation by comparing observability data like service-level objectives against a dedicated release version from the past or as part of progressive delivery.

Get started

Get started with Site Reliability Guardian by selecting the components that you want to guard automatically. Relevant objectives, known as the four golden signals, are suggested by default:

• Latency: the time it takes to service a request
• Traffic: a measure of how much demand is placed on your system
• Errors: the rate of failed requests
• Saturation: a measure of a system’s remaining capacity, emphasizing the resources that are most constrained

The four golden signals are recommended for end-user-facing services because they have proven to be a good starting point for diving into SRE practices. Additionally, you can easily use any previously defined metrics and SLOs from your environments.

Automation workflow runs validations and notifies responsible team members

With a solid set of objectives to validate, automatically generated workflows take care of fully automated validation for you. Whether triggered by a test result or a new release deployment, detected events work as a trigger to check the defined objectives and derive an overall status automatically. Based on the results, further actions are taken, like sending targeted notifications to the service and application owners. This is all available out-of-the-box with the default workflow template provided by Site Reliability Guardian.

In addition, the workflow can be easily extended to any custom demands, for example, integration with tools that support your software product lifecycle. This includes executing tests, running Dynatrace Synthetic checks, or creating tickets. In this way, Dynatrace supports an ecosystem of tool integrations that unlock the full potential of DevOps and SRE with cloud-native delivery orchestration and remediation automation use cases.

What’s next?

Site Reliability Guardian is built for ensuring and maintaining production reliability and resilience. Further enhancements will focus on seamlessly integrating the solution into your existing pipelines to help you shift left and block poor-quality releases before they reach production. Davis will also assist Site Reliability Guardian in recommending relevant objectives and baselines for comparison.

• For an early preview of this new functionality, please request Dynatrace platform preview access, which allows you to install the Site Reliability Guardian via Dynatrace Hub.
• The Dynatrace Site Reliability Guardian will be available within the next 90 days. In the meantime, watch out for upcoming “Observability Clinics” and “Ask me anything” sessions covering the main topics of this blog post. You can either see a list of the next webinars on our website or follow us on LinkedIn to stay up-to-date with upcoming announcements and activities.

The post Automated Change Impact Analysis with Site Reliability Guardian appeared first on Dynatrace news.

Cloud environments—including multicloud, hybrid, and cloud-native ecosystems—offer unmatched agility, scalability, and cost-effectiveness, though they also present new challenges and complexities that are impossible to manage manually. As cloud environments become increasingly ubiquitous, so does the need for effective and efficient management.

Dynatrace integrates observability and security monitoring while leveraging causal AI to deliver answers and intelligent automation from data at an enormous scale. The Dynatrace platform provides real-time situational awareness of your cloud ecosystem to give you the confidence that your organization is operating as it should. With answer-driven automation powered by the new Dynatrace AutomationEngine, we’re releasing a solution that further extends our platform, putting answers into action, and empowering organizations to overcome cloud management challenges and unlock the full potential of their cloud strategy while reducing manual effort.

Context is king

Many modern enterprises suffer because their top technical talent is consumed with manual toil. According to the Global CIO Report 2022, 71% of CIOs say the explosion of data produced by cloud-native technology stacks is beyond human abilities to manage manually. Building effective and reliable systems is only possible with automation, which, in the past, proved difficult due to the following issues:

• Complexity of systems: The complexity of modern systems makes it difficult to gather all the necessary information to automate decision-making.
• Lack of standardization: The lack of standardization across tools and technologies makes it difficult to handle automation decisions and actions across different systems.
• Lack of data: In many cases, available data may not be enough to make informed decisions due to data silos, distribution, or negligence.
• Incorrect action on unreliable data: Automation systems without context-rich observability and security data often fail in pinpointing the best remediation actions to take, resulting in unwanted or incorrect reactions, as well as false-positive and false-negative alerts.
• Evolving threats: New security threats and vulnerabilities are constantly emerging. This makes it difficult for pure rule-based automation systems to adapt to new threats and vulnerabilities in real time.
• Primitive safety nets: Having no real-time automation feedback loop results in additional toil and escalation actions.

With the introduction of AutomationEngine, the Dynatrace platform has overcome these automation challenges

Dynatrace enables full observability by supporting open-source tools, as well as Dynatrace OneAgent and other data-ingest methods in enriching the classic observability pillars of metrics, traces, and logs with topology, user behavior, security, metadata, and network data. Dynatrace Davis® AI integrates all this context-rich data to provide answers within complex, multi-level systems and environments.

Extending the platform with Dynatrace AutomationEngine unlocks the power to combine observability, security, and business data with causal AI to easily automate BizDevSecOps workflows at enterprise scale.

Automate what matters

Extending the Dynatrace platform natively with AutomationEngine leads to causal-AI-powered, answer-driven automation. This helps organizations to achieve proactive prevention, efficient operation, faster remediation, strong protection, and business acceleration. Use cases include but are not limited to:

• Targeted notification and collaboration: connect the right teams and equip them with answers in real-time
• Closed-loop remediation: automatically remediate problems and vulnerabilities
• Quality and security gating: block vulnerabilities and regressions before they reach production
• Continuous vulnerability assessment: detect zero-day vulnerabilities
• Predictive automation: take maintenance measures before risking a potential failure
• Progressive delivery orchestration: orchestrate releases based on user and system feedback
• Intelligent infrastructure and application orchestration: auto-scale and resize smarter load distributions
• Preventive security configurations: block malicious IPs based on known attack patterns

Dynatrace Workflows, powered by AutomationEngine, puts automation use cases at your fingertips. Every automated workflow consists of easy configuration, extensive trigger options, out-of-the-box actions and integrations, and unprecedented extensibility by leveraging webhooks, JavaScript code, and application actions powered by AppEngine.

Easy workflow creation and maintenance

Workflow creation and maintenance at scale

Another way to ensure always up-to-date workflows and automation at scale, teams can use the automation-as-code approach available with Dynatrace’ own configuration as code solution named Monaco. By shifting your automation configuration left, Monaco moves automation definition closer to not only your source code, but also your testing, observability, security, and remediation capabilities. Automation-as-code allows for powerful and concise automation across the full software development lifecycle. Another big advantage of automation-as-code is the scale at which automation is enabled. By following a programmatical approach, developers, DevOps and SREs benefit from a single source of truth and streamlined collaboration. This helps to eliminate outdated manual runbooks, which are inevitably fail.

Proactive automation

You can configure a variety of event triggers  to initiate custom workflows based on your needs:

• On-demand trigger: manual execution of an otherwise tedious chain of tasks, for example, audit automation
• API-based trigger: direct calls embedded into existing toolchains, for example, pipeline integration
• Schedule-based trigger: scheduled execution of recurring tasks, for example, reporting
• Problem, security, or Davis event-based trigger: Davis causal-AI triggered actions that reduce mean time to repair (MTTR), for example, problem remediation
• Dynatrace Query Language (DQL) event-filter trigger: results of event filters applied to the observability, security, and business data in Dynatrace to trigger actions based on certain event types, for example, deployments of a specific service

Boundless automation

Every workflow consists of several tasks that can be executed or skipped based on the conditions of previous tasks, retried based on error logic, or looped to iterate over lists. To enable this, workflows are aware of variables and context and can execute DQL queries in real-time. Besides executing actions within the Dynatrace platform itself, Dynatrace AutomationEngine allows the safe triggering of actions to both cloud services and—by leveraging Dynatrace EdgeConnect—on-premises services. Dynatrace provides an ever-growing ecosystem of deep, out-of-the-box, third-party integrations that connect automation with collaboration, ITSM, AIOps, DevOps, and security tools.

For truly limitless automation capabilities, Dynatrace Workflows can issue HTTP requests to any API and execute custom Javascript code. By leveraging Dynatrace AppEngine, any application can publish universally-available actions for all workflows, thereby extending custom logic to automated workflows. This automation context, embedded into every app, unlocks even more automation use cases.

Dynatrace AutomationEngine leverages platform-wide event triggers and timezone-aware schedules with dynamic recurrence rules. It allows custom flow logic and parallel task execution with automatic retry, and can be configured-as-code, all to accommodate the complexities of global enterprises.

What’s next?

Interested and want to learn more? The new Dynatrace user experience, including the newly developed AutomationEngine, will be available within 90 days. In the meantime, watch out for upcoming Dynatrace Observability Clinics and live “Ask me anything” sessions covering the main topics of this blog post. You can see a list of the next webinars on our website or follow us on LinkedIn to stay up-to-date with upcoming announcements and activities.

The AutomationEngine actions and integrations ecosystem are being continuously developed and extended. If you’re looking for a certain third-party integration for automation that is currently missing, let us know by posting to the Dynatrace Community.

The post AutomationEngine tames cloud complexity with answer-driven automation appeared first on Dynatrace news.