Johannes Bräuer | Dynatrace news https://www.dynatrace.com/news/blog/author/johannes-brauer/ The tech industry is moving fast and our customers are as well. Stay up-to-date with the latest trends, best practices, thought leadership, and our solution's biweekly feature releases. Tue, 23 Sep 2025 17:45:05 +0000 en hourly 1 Observability throughout the software development lifecycle increases delivery performance https://www.dynatrace.com/news/blog/observability-throughout-the-software-development-lifecycle/ https://www.dynatrace.com/news/blog/observability-throughout-the-software-development-lifecycle/#respond Fri, 04 Oct 2024 15:50:05 +0000 https://www.dynatrace.com/news/?p=66010 Insights into Azure DevOps pipelines

Delivering software efficiently and reliably is a cornerstone of many organizations' business success today. Dynatrace fulfills a crucial role by aiding platform engineers with insights into the delivery processes to reveal performance bottlenecks, track pipeline improvements, and optimize resource consumption. To unlock pipeline observability for streamlining your software delivery processes, Dynatrace drives the standardization of software development lifecycle events as the foundation for analytics and automation use cases.

The post Observability throughout the software development lifecycle increases delivery performance appeared first on Dynatrace news.

]]>
Insights into Azure DevOps pipelines

Today, development teams suffer from a lack of automation for time-consuming tasks, the absence of standardization due to an overabundance of tool options, and insufficiently mature DevSecOps processes. This leads to frustrating bottlenecks for developers attempting to build and deliver software. For multinational enterprises, the cost of all that toil, complexity, and frustration multiplies exponentially due to the size of those organizations. In such contexts, platform engineering offers a compelling solution to enable business competitiveness in a manner that significantly enhances the developer experience. A central element of platform engineering teams is a robust Internal Developer Platform (IDP), which encompasses a set of tools, services, and infrastructure that enables developers to build, test, and deploy software applications.

Treating an Internal Developer Platform (IDP) as a product is an emerging paradigm within platform engineering communities. Similar to the observability desired for a request being processed by your digital services, it’s necessary to comprehend the metrics, traces, logs, and events associated with a code change from development through to production. The end-to-end observability of the engineering process becomes essential when building software under product safety legislation, such as medical devices. It’s nearly as crucial as the final product itself when high-frequency releases are critical to the business.

This blog post focuses on pipeline observability as a method for monitoring the software delivery capabilities of an organization’s IDP. This process begins when the developer merges a code change and ends when it is running in a production environment. In other words, pipeline observability encompasses Continuous Integration (CI) and includes Continuous Delivery (CD), as depicted in the following image.

Pipelines observability in DevSecOps circle

Unlocked use cases

Gaining insights into your pipelines and applying the power of Dynatrace analytics and automation unlocks numerous use cases:

  • Make data-driven improvements: Invest in those software delivery capabilities that provide the most significant payoff. To make the right decision, pipeline KPIs can reveal areas for improvement, protect your long-term investment in development, and present stakeholders with the benefits of sustained funding.
  • Automate delivery processes: Ideally, an improvement entails introducing automation to eliminate manual tasks, foster collaboration, or speed up processes.
  • Ensure pipeline compliance/governance: By obtaining an end-to-end view of the pipeline activities that a code change progresses through, you can identify blind spots that require monitoring and ensure compliance across the entire organization.

Software Development Lifecycle (SDLC) events

The Software Development Lifecycle (SDLC) is similar to the DevSecOps loop as it outlines the various phases a product idea undergoes before it is released into production. These phases must be aligned with security best practices, as discussed in A Beginner`s Guide to DevOps. While the lifecycle starts with a ticket specifying a new product idea, an actual code change often triggers various automated tasks kicking off the next phase. These automation tasks process the code change to build a deployable and create new artifacts accompanying the change throughout the delivery process.

In Dynatrace, examples of such tasks and resulting artifacts include the following, visualized in the below diagram:

  • Plan: A ticket for enhancing a platform service is created in Jira.
  • Code: The branch for the new feature in a GitHub repository is merged into the main branch.
  • Build: GitHub builds the code changes and creates a deployable image.
  • Test: Playwright executes end-to-end tests.
  • Release:
    • The image is published on Harbor, and the Helm Chart is updated.
    • BlackDuck performs a security and vulnerability check, returning a scan result.
  • Deploy:
    • ArgoCD deploys the Helm Chart in a development environment.
    • Synthetic HTTP monitors are executed in the hardening stage.
    • An automation updates a GitOps repository to roll out the artifact to production
  • Operate: FlagD switches a feature flag from OFF to ON.

Events in DevSecOps and pipeline observability

Given these examples, which are a subset of many others, crucial information about a software change and the related component is generated before monitoring begins. Consequently, Dynatrace defines the semantics of essential data points and stores them in a normalized manner. For instance, the Git commit, ticket ID, artifact version, release version, deployment stage, etc., are data points that require special attention. When the semantics of this metadata are well-defined, you can build insightful analytics and robust automation.

The specification defines the new event type SDLC_EVENT, which covers the event category for pipeline and task events. A pipeline can be the parent of multiple tasks to group the resulting events logically. A task event, however, represents the state of a single lifecycle activity like a build, test, deployment, scan, etc. Dynatrace Documentation maintains a list of events, which will grow as we unlock new use cases.

Defining an event standard succeeds through adoption within the ecosystem. This is why we’re aligned with open source initiatives and compatible with the CI/CD specification in OpenTelemetry driven by the CD Foundation. Consequently, your efforts into OpenTelemetry for pipeline observability will be natively supported by Dynatrace.

Generic SDLC event ingest endpoint

Dynatrace introduced a generic SDLC event ingest endpoint that enables data ingestion directly from any third-party CI/CD tool. While this endpoint accepts any JSON payload and adds default properties of the SDLC specification,  OpenPipeline™ is leveraged to process vendor-specific data and extract those properties defined by the specification. GitHub and ArgoCD are the first two tools to receive an OpenPipeline configuration.

The endpoint, together with OpenPipeline, opens a range of use cases, such as:

  • Pipeline anomaly visualization on custom dashboards tailored within Dynatrace.
  • In-depth analysis of delivery tasks using tools like Notebooks.
  • Automated release validation using Dynatrace Site Reliability Guardian.
  • Contextualization of third-party CI/CD activities with Dynatrace monitored entities.

Stay tuned

Currently, the API allows for the configuration of an event processing pipeline. Support for managing configuration in the OpenPipline app is a high priority. This will be accompanied by the above-mentioned GitHub and ArgoCD configurations and others to follow.

In some cases, a CI/CD tool’s log line contains important data points that, on their own, eliminate the need to additionally send an event to Dynatrace. Therefore, we’re working on a log processor to automatically extract relevant metadata and generate SDLC events from log lines.

Lastly, we’re working on a ready-made dashboard for the DORA metrics based on GitHub and ArgoCD metadata.

What’s next

  • Explore: See the analytics sample in our documentation, which is executable in the playground environment.
  • Adopt: Identify your most critical pipelines and make them visible. In other words, leverage the available event ingest endpoint and send pipeline activities to Dynatrace. In many cases, this is possible by leveraging webhook capabilities.
  • Fine-tune: Perform data normalization by creating an OpenPipeline configuration that extracts properties according to the SDLC specification.

The post Observability throughout the software development lifecycle increases delivery performance appeared first on Dynatrace news.

]]>
https://www.dynatrace.com/news/blog/observability-throughout-the-software-development-lifecycle/feed/ 0
Auto-adaptive thresholds for AI-driven quality gating https://www.dynatrace.com/news/blog/auto-adaptive-thresholds-for-ai-driven-quality-gating/ https://www.dynatrace.com/news/blog/auto-adaptive-thresholds-for-ai-driven-quality-gating/#respond Tue, 04 Jun 2024 17:12:50 +0000 https://www.dynatrace.com/news/?p=64258 auto-adaptive thresholds

The Site Reliability Guardian automates the validation process for new software releases or changes. These validations involve setting specific performance, availability, or security objectives that must be met. For example, response time or failure rate thresholds can be used to define the desired state, warning range, or when an objective is violated. However, it can be difficult or even impossible to set these targets upfront for a new software component because it's unclear how the component will behave or what circumstances it will face.

The post Auto-adaptive thresholds for AI-driven quality gating appeared first on Dynatrace news.

]]>
auto-adaptive thresholds

The latest release of the Site Reliability Guardian incorporates assistance from Dynatrace Davis® AI to automatically derive appropriate threshold targets and adjust them over time to protect your quality improvements. This process, known as auto-adaptive thresholding, eliminates the need to define a static threshold upfront. Instead, it derives the suitable thresholds from previous validation results.

Build an umbrella for Development and Operations

In modern software engineering, the discipline of platform engineering delivers DevSecOps practices to developers to bridge the gaps between development, security, and operations and enhance the developer experience. A key element in platform engineering is the establishment of fast feedback cycles regarding the quality and security measures of new software releases. To provide automated feedback for developers, the concept of quality gates for static code analysis in continuous integration is widely adopted throughout the industry. However, this perspective differs in the continuous deployment practices of various organizations, where the feedback is either delayed or not returned to the developer.

While receiving no feedback on the quality or security of new features leaves developers uncertain about feature performance, delayed feedback also increases a developer’s cognitive load. The developer must pause their current engineering work to address the reported issue and consider the code changes they worked on a few days or weeks prior.

To reduce developers’ cognitive load by providing timely information, platform engineers must create tools that allow validations to be run in the early phases of development, with direct and fast feedback loops. Ideally, this should be a self-service offering that enables individual adoption by teams. While platform engineers can build and prepare the necessary infrastructure and templates for self-adoption, developers must still provide some customization. For example, the team must establish specific thresholds for desired service performance behavior. Setting these thresholds upfront can be challenging because the team might not know how a service will behave in its environment.

How we define auto-adaptive thresholds at Dynatrace

This blog post explores how Dynatrace leveraged the Site Reliability Guardian to establish a fast feedback loop for Davis AI model improvements. The conducted validations avoided regression within the models, and the outcomes were immediately fed back to the data science team when deviations were detected. While the data science team appreciated the quick insights and validations of their improvements, they initially struggled with the setup. Consequently, this blog post highlights the new capability of the Site Reliability Guardian to define auto-adaptive thresholds that tackle the challenge of configuring static thresholds and protect your quality and security investments with relative comparisons to previous validations.

Fast feedback cycles on model improvements

While the Site Reliability Guardian was originally designed to validate new software releases, Dynatrace has internally extended its application area to include validation of models for Davis AI.

The Dynatrace data science team continuously improves the machine learning models used by Davis AI, for example, by adding new features to forecasting or refining mathematical calculations. A single change can influence multiple models, as features are often used across several models. To ensure that model changes don’t lead to regressions, the data scientists set up Site Reliability Guardian, which is automatically triggered whenever a change is made in the codebase via CI/CD pipelines.

A series of models are continuously trained on Dynatrace tenants to effectively set objectives. The training times and other quality metrics, such as the RMSE (Root Mean Squared Error), SMAPE (Scaled Mean Absolute Percentage Error), and coverage probability, are monitored using Dynatrace. Our data scientists utilize metrics and events to store these quality metrics. However, other data formats, like logs, can also be employed. The quality metrics can then be easily queried using DQL and utilized for the objectives of a guardian. Validations are automatically triggered when a change is committed to the code base via the Dynatrace API. This helps data scientists quickly respond to recently introduced regressions. For instance, if an objective is violated, they’re immediately notified, for example, through a Slack channel.

Validation history
Figure 1. Validation history

One difficulty encountered when setting up objectives in the guardians was selecting an appropriate threshold for the quality metrics, as this is typically heavily dependent on the data.

Leverage Davis AI to quickly start validating

To address the challenge of defining a static threshold for an objective, the Site Reliability Guardian enables switching objective thresholds to auto-adaptive mode, as depicted in the screenshot below.

Activating an auto-adaptive threshold for the response time objective
Figure 2. Activating an auto-adaptive threshold for the response time objective

Davis AI controls auto-adaptive thresholds. It analyzes the next five validations to derive this objective’s proper warning and failure thresholds. Once the learning phase is complete, all subsequent validation results are fed into Davis AI to fine-tune the thresholds based on changed behavior.

Considering previous validation results, the latest validation is always relative to the past, protecting quality and security improvements. If, for example, recent performance improvements change a service’s response time from 200 ms to 175 ms, the auto-adaptive threshold is adjusted to reflect the new behavior. Nevertheless, the Site Reliability Guardian detects sudden behavior changes by reporting a warning or failure if response time returns to 200 ms or above.

Learning phase

Unless an objective has been validated five times, it’s still in the learning phase. During this phase, the measured values are informative, allowing observation of the objective’s development without affecting deployment or delivery processes. The Site Reliability Guardian denotes the learning phase of an objective with a loading symbol on the heatmap and in the objective details.

Representation of the learning phase of an auto-adaptive threshold
Figure 3. Representation of the learning phase of an auto-adaptive threshold

The warning and failure thresholds will be automatically set if sufficient validations are available to establish a solid baseline for an objective’s auto-adaptive thresholds. Consequently, the next objective validation will impact the overall validation result.

Auto-adaptive thresholds as code

To enhance the developer’s experience in adopting the Site Reliability Guardian in a self-service manner, the configuration for a guardian and its workflow can be provided in a configuration-as-code fashion. This enables the management of the configuration within the service’s code repository. Incorporating the new capability of auto-adaptive thresholds into configuration-as-code is as simple as adding the autoAdaptiveThresholdEnabled flag to an objective.

Configuration-as-code example for activating an auto-adaptive threshold
Figure 4. Configuration-as-code example for activating an auto-adaptive threshold

Before concluding, we wish to announce the change of the Site Reliability Guardian icon from purple to shiny golden. This new appearance enhances the icon’s geometry while preserving its core values. Therefore, the icon continues to feature the infinity loop as a symbol for the DevSecOps loop and the fast forward sign to expedite delivery while ensuring quality and security standards.

The new and shiny appearance of Site Reliability Guardian

Evolution of the Site Reliability Guardian icon
Figure 5. Evolution of the Site Reliability Guardian icon

What’s next

The new auto-adaptive thresholds capability is now available in Site Reliability Guardian. Open the app and switch from static to auto-adaptive thresholds for those objectives where Davis AI should derive the thresholds for you. For full details, see Dynatrace Documentation.

If you haven’t used Site Reliability Guardian yet, try it out in the Dynatrace Playground or watch the latest app spotlight recording.

The post Auto-adaptive thresholds for AI-driven quality gating appeared first on Dynatrace news.

]]>
https://www.dynatrace.com/news/blog/auto-adaptive-thresholds-for-ai-driven-quality-gating/feed/ 0
Ensure safe and secure releases at scale by providing Golden Paths https://www.dynatrace.com/news/blog/ensure-safe-and-secure-releases-at-scale-by-providing-golden-paths/ https://www.dynatrace.com/news/blog/ensure-safe-and-secure-releases-at-scale-by-providing-golden-paths/#respond Tue, 14 Nov 2023 16:17:41 +0000 https://www.dynatrace.com/news/?p=60743 Unlocking the Power of Quality Gating by enabling Self-Service

Golden Paths for rapid product development Modern software development aims to streamline development and delivery processes to ensure fast releases to the market without violating quality and security standards. DevOps practices have been established in the last decade to accomplish this goal and deal with the dynamics of modern, cloud-native software architectures. To bring these […]

The post Ensure safe and secure releases at scale by providing Golden Paths appeared first on Dynatrace news.

]]>
Unlocking the Power of Quality Gating by enabling Self-Service

Golden Paths for rapid product development

Modern software development aims to streamline development and delivery processes to ensure fast releases to the market without violating quality and security standards. DevOps practices have been established in the last decade to accomplish this goal and deal with the dynamics of modern, cloud-native software architectures. To bring these practices to life within an organization at scale, the discipline of platform engineering has gained popularity. From a high-level point of view, platform engineering aims to:

  • Reduce the cognitive load on development teams.
  • Improve reliability and resiliency of products that rely on platform capabilities.
  • Accelerate product development and delivery by reusing and sharing platform tools and knowledge.
  • Reduce risk of security, regulatory, and functional issues in products and services.
  • Enable cost-effective and productive use of services.

While it takes multiple capabilities to achieve these goals, implementing “Golden Path” templates is a fundamental ingredient. The Cloud Native Computing Foundation defines a Golden Path as a “templated composition of well-integrated code and capabilities for rapid project development.” Simply put, a Golden Path is a self-service template for common tasks that allows for autonomy while providing guardrails that safeguard production environments.

Imagine that instead of development teams fending for themselves amidst a sea of tools and infrastructure, well-defined and enterprise-wide templates are provided for the development of all new product services. Such a template should contain a get-started tutorial, sample source-code framework, policy guardrails, CI/CD pipeline, infrastructure-as-code templates, and reference documentation. This approach helps you quickly integrate best practices within your organization and provides cloneable artifacts for rapid product development.

No developer is left in the dark to fend for themselves, Golden Paths light the way.

Ensure governance across your organization

While Golden Paths are key to bringing DevOps best practices to development teams, distributing them is challenging. This challenge is partially addressed by internal development platforms (IDP), which have been adopted by many, but not all, organizations.

Consequently, Dynatrace provides templates in the context they are needed—with or without the internal use of an IDP. This ensures governance across your organization with the proper templates in the right place. At the same time, this does not restrict you from managing them in your IDP, as explained below.

Leverage opinionated, self-service, and optional templates

The latest enhancements of the Site Reliability Guardian have introduced opinionated Golden Path templates. For now, they concentrate on Kubernetes, host, and security objectives but they will grow to cover even more reliability, performance, and security aspects. These templates enable development teams to instantiate a guardian for automating release validation in a self-service manner. Along the journey, monitored entities can be selected to provide the context to fetch the right data from Dynatrace Grail™.

Two-step approach to creating a guardian using a template.
A two-step approach to creating a guardian using a template.

After completing this two-step process, a ready-to-use guardian is created. It’s possible to add new objectives if needed or to tailor the objectives and their thresholds.

Fully functional and pre-configured guardian for a Kubernetes workload.
Fully functional and pre-configured guardian for a Kubernetes workload.

Allow for flexibility

Custom query variables are available to fine-tune guardian objectives and maintain flexibility in fetching data from Grail. An example of such a variable is the version number of a service. In many cases, you want to retrieve the logs, metrics, or traces of a particular version, which is unknown upfront. Consequently, the custom variable version can be defined in the DQL query, which retrieves its value before executing.

Custom query variables can be defined by starting with the $ sign followed by the variable name in the DQL query editor of a guardian objective. An overlay component then helps you add this variable and define the default value in the context of the guardian.

Leverage variables to parameterize the query of an objective.
Leverage variables to parameterize the query of an objective.

While a variable will fall back to its default if no value is provided, there are three ways of ingesting the value:

  1. When triggering a guardian validation from within the UI by selecting the Validate button, set a value for the configured variables using the Set variables option.
  2. The Site Reliability Guardian workflow action allows you to set a variable value. In this case, it is possible to consume data from the triggering event or a previous workflow action using Jinja expressions.
  3. If a workflow is used to automate the validation process, the workflow trigger event can provide automatically mapped properties to variables. Therefore, the properties must be within the execution_context property, as shown below. Given this example, the version number 0.1.0 is set where the variable $version is defined.

Code snippet to set variables

Since variables provide important context for validation, you can view validation-result values over time. The following screenshots depict the version number associated with the last 10 validation results.

Site Reliability Guardian Carts validation history in Dynatrace screenshot

Integrate with existing internal developer platforms

If you have an internal developer platform that serves as a central product engineering hub, you need Golden Path templates incorporated there so that you can scale out the platform for use by different teams. With Dynatrace, this can be achieved by extracting an instantiated template using Dynatrace Configuration as Code or cloning it from our public repository. This declarative format of a template, consisting of a guardian and workflow configuration, as depicted by the sample below, can then be added to the template repository managed and distributed by an IDP.

Dynatrace Configuration as Code sample

What’s next?

Site Reliability Guardian is built to ensure and maintain reliability and resiliency of products by leveraging automated release validations. Therefore, Golden Path templates in a self-service manner are offered for platform engineers to enable development teams to get started easily. The next enhancements of the Site Reliability Guardian will bring Davis® AI even closer to the Site Reliability Guardian by recommending relevant objectives and baselines for comparison.

  • The new functionality has already been released and is available for your use. If you’re using the Site Reliability Guardian already, update to the latest version. Otherwise, navigate to Dynatrace Hub and install it from there.
  • If the Site Reliability Guardian is new to you and you’re curious about it, go to Dynatrace Discovery and explore its capabilities in a playground environment.

We’d love to hear your feedback about Site Reliability GuardianLet us know your thoughts or share your ideas about further improvements.

The post Ensure safe and secure releases at scale by providing Golden Paths appeared first on Dynatrace news.

]]>
https://www.dynatrace.com/news/blog/ensure-safe-and-secure-releases-at-scale-by-providing-golden-paths/feed/ 0
How Dynatrace boosts its production resilience with Site Reliability Guardian https://www.dynatrace.com/news/blog/how-dynatrace-boosts-production-resilience-with-site-reliability-guardian/ https://www.dynatrace.com/news/blog/how-dynatrace-boosts-production-resilience-with-site-reliability-guardian/#respond Wed, 17 May 2023 17:55:18 +0000 https://www.dynatrace.com/news/?p=57700 Site Reliability Guardian, CrowdStrike outage

In this blog post, we share insights into how Dynatrace automates software delivery and quality validation tasks using the Dynatrace® Site Reliability Guardian. Announced at Perform 2023, the Site Reliability Guardian exemplifies our commitment to DevOps and SRE best practices that ensure reliable, secure, and high-quality releases.

The post How Dynatrace boosts its production resilience with Site Reliability Guardian appeared first on Dynatrace news.

]]>
Site Reliability Guardian, CrowdStrike outage

To ensure high standards, it’s essential that your organization establish automated validations in an early phase of the software development process—ideally when code is written. The Dynatrace Site Reliability Guardian is designed for this practice; it allows development teams to define quality objectives in their code, which is validated throughout the delivery process before the code reaches production.

While quality standards and validations should ideally be implemented throughout the software development lifecycle, we see teams adopting the Site Reliability Guardian at various stages, from development to production environments. Our discussions with teams who use Site Reliability Guardian in production reveal that starting with post-deployment validation provides quick insights into the status quo and helps identify areas that need improvement via pre-production validations. This approach, known as “shift-left,” is preferred by organizations that derive insights from their production environments and plan improvements from there.

How Dynatrace uses Site Reliability Guardian

In each of these Dynatrace examples, insight is made in a production-like environment. Validation tasks are then extended left to cover performance testing and release validation in a pre-production environment. These examples can help you define your starting point for establishing DevOps and SRE best practices in your organization.

Ensure expected production behavior

One Dynatrace team is responsible for the demo applications we use to demonstrate Dynatrace capabilities. We use monitored demo applications to deliver constant load and a defined set of business transactions. Without demo applications, it would not be possible for our sales teams in the field to show off the full power of Dynatrace. To ensure these objectives, the team leverages the Site Reliability Guardian to check for a potential decrease in business transactions whenever a new application release is deployed.

Two guardians, which are managed by the team’s Site Reliability Guardian and contain different sets of objectives, are used in this setup. While the first guardian validates the traffic, the second guardian checks the business transactions generated during the observation period. The functionality is implemented via an automated workflow.

The Workflows screenshot below shows that a task is triggered by a change event related to the application, execution of the guardians, and final aggregation of the results. Ultimately, the result is shared via a Slack notification to report on the current business behavior.

Dynatrace Workflows screenshot - task triggered by change event

Resilient applications with chaos testing in pre-production

Another Dynatrace team uses a guardian as a safeguard during chaos testing. More precisely, this team uses AWS Fault Injection Simulator (FIS) to run fault injection to improve the application’s performance and resiliency. These experiments are accompanied by well-defined objectives formulated as Dynatrace service-level objectives (SLOs). The guardian is used to bring the experiment of the fault injection simulator and the SLOs together. Hence, it validates the status of the SLOs for the experiment period to derive potential regressions that might have been introduced in a newer release.

While this team uses Fault Injection Simulator (FIS) to run their chaos testing experiments, the Site Reliability Guardian is not limited to this testing practice. It fits well into the entire performance testing domain where reliability, performance, and security indicators must be continuously validated.

Release validation on the four golden signals in pre-production

Another Dynatrace team adopted Site Reliability Guardian with an early version of our Traces in Grail capability for a classic approach to release validation within its delivery pipeline. In other words, they use it as a quality gate that detects bad releases before they reach production. They first explore relevant objectives in their production environment. Based on those insights, they implemented automated validation tasks, and shifted left in their software delivery pipeline.

In this case, the four golden signals (latency, traffic, errors, and saturation) are derived from span attributes and DQL metric queries via Dynatrace Grail™. The queries are depicted below (sensitive data has been removed).

Latency – Response time (below 50 ms) derived from spans

fetch spans
| filter application.name == "APPLICATION-NAME"
| filter application.environment == "hardening"
| filter dt.entity.service_method.name == "getEffectivePolicies"
| filter span.kind == "SERVER"
| summarize toDouble(countIf(duration <= 50ms)) / count()

Traffic – Number of requests derived from log lines

fetch logs
| filter log.source == "projects/APPLICATION-NAME/logs/requests"
| parse content,"JSON:request"
| fieldsAdd httpRequest = request[httpRequest]
| summarize count = count()

Error rate – Number of HTTP 5xx errors derived from spans

fetch spans
| filter application.name == "APPLICATION-NAME"
| filter application.environment == "hardening"
| filter dt.entity.service_method.name == "getEffectivePolicies"
| filter span.kind == "SERVER"
| summarize response_500_error_rate = toDouble(countIf(http.status_code >= 500 AND http.status.code < 600))/count()

Saturation – Memory requested by a Kubernetes workload

timeseries avg(dt.kubernetes.workload.requests_memory),
filter: in(dt.kubernetes.workload.name, "WORKLOAD-NAME")

Please note that “Traces in Grail” are currently in private preview, and the DQL syntax could be subject to change. 

What’s next?

Dynatrace strives to enable DevOps and SREs to ship software that works perfectly. Accordingly, we provide the Site Reliability Guardian as one way to establish reliability, security, and quality practices early in your software development lifecycle.

Stay tuned for an upcoming blog post that shows how you can shift your DevOps and SRE practices even further left by declaring your standards in code—leveraging the Dynatrace configuration-as-code approach—and validating them via Site Reliability Guardian as part of your delivery process.

Check out the Site Reliability Guardian for yourself by installing it using the Dynatrace Hub, then share your feedback with us and let us know how you intend to use it. Head over to the Dynatrace Community to see our plans for additional features. We’d love to hear your suggestions and ideas.

The post How Dynatrace boosts its production resilience with Site Reliability Guardian appeared first on Dynatrace news.

]]>
https://www.dynatrace.com/news/blog/how-dynatrace-boosts-production-resilience-with-site-reliability-guardian/feed/ 0
Automated Change Impact Analysis with Site Reliability Guardian https://www.dynatrace.com/news/blog/site-reliability-guardian/ https://www.dynatrace.com/news/blog/site-reliability-guardian/#respond Thu, 16 Feb 2023 05:00:32 +0000 https://www.dynatrace.com/news/?p=56098 SLOs graphic

The Dynatrace® Site Reliability Guardian simplifies the adoption of DevOps and SRE best practices to ensure reliable, secure, and high-quality releases.

The post Automated Change Impact Analysis with Site Reliability Guardian appeared first on Dynatrace news.

]]>
SLOs graphic

Powered by Grail and the Dynatrace AutomationEngine, Site Reliability Guardian helps DevOps platform teams make better-informed release decisions by utilizing all the contextual observability and application security insights of the Dynatrace platform. The app also enables SREs to set and automate service-level objectives (SLOs) for critical services. Site Reliability Guardian further simplifies automation requirements by using entity and ownership information, alerting the right teams when regressions are detected. Dynatrace users can now unlock answer-driven automation to safely and securely release new features to customers while ensuring production SLOs are intact.

Streamline development and delivery processes

Nowadays, digital transformation strategies are executed by almost every organization across all industries. In the context of application development, digital transformation aims to streamline development and delivery processes to ensure quick delivery to the market with guaranteed high quality and security standards.

To achieve this, many organizations are adopting DevOps practices to provide developers with a delivery platform to release their applications and services autonomously and independently. While this empowers teams to frequently deliver new features, the overall business, security, and quality objectives must be maintained. This is where Site Reliability Engineering (SRE) practices are applied. SREs use Service-Level Indicators (SLI) to see the complete picture of service availability, latency, performance, and capacity across various systems, especially revenue-critical systems. Consequently, Service-Level objectives (SLO) are defined to enact countermeasures before the business is impacted.

Given the momentum of DevOps and SRE, digital transformation goals can be achieved when automation enables organizations to apply best practices rapidly and to keep pace with the scale of the organization and applications.

SLOs graphic

Automation is key for effective collaboration

While DevOps engineers rely on observability and automation to determine whether they can promote a new release into production, SREs look for answers when a new release has a positive or negative impact on application performance, affects user experience, or threatens other SLOs. SREs face ever more challenging situations as environment complexity increases, applications scale up, and organizations grow:

  • Growing dependency graphs result in blind spots and the inability to correlate performance metrics with user experience.
  • Siloed teams and multiple tools make it difficult to align on a single version of the truth for overall system health.

Other observability solutions don’t provide the required automation capabilities that would allow:

  • Automating and speeding up the SLO validation process and quickly reacting to regressions detected in application topology.
  • Informing the right people with the answers they need to implement targeted countermeasures.

Automate the validation of key objectives

Dynatrace evolves Cloud Automation release validation (powered by Keptn) into Site Reliability Guardian and natively enriches the Dynatrace platform by automating change impact analysis. For each service, a guardian can automatically validate service-level objectives, “golden signals,” or security vulnerabilities before and after any deployment or configuration changes. It detects regressions and deviations from previously observed behavior, including latency, traffic, error rates, saturation, security coverage, vulnerability risk levels, and memory consumption. Thus, Site Reliability Guardian supports DevOps and SREs in speeding up release delivery and improving release quality.

  • DevOps achieve safer and more secure releases by applying a gating mechanism that identifies release issues quickly and prevents poor-quality code from being promoted to production.
  • SREs leverage on-demand reliability validation by comparing observability data like service-level objectives against a dedicated release version from the past or as part of progressive delivery.

Get started

Get started with Site Reliability Guardian by selecting the components that you want to guard automatically. Relevant objectives, known as the four golden signals, are suggested by default:

  • Latency: the time it takes to service a request
  • Traffic: a measure of how much demand is placed on your system
  • Errors: the rate of failed requests
  • Saturation: a measure of a system’s remaining capacity, emphasizing the resources that are most constrained

The four golden signals are recommended for end-user-facing services because they have proven to be a good starting point for diving into SRE practices. Additionally, you can easily use any previously defined metrics and SLOs from your environments.

Site Reliability Guardian showing a failed validation
Site Reliability Guardian showing a failed validation due to an increase of error logs for a cart service.

Automation workflow runs validations and notifies responsible team members

With a solid set of objectives to validate, automatically generated workflows take care of fully automated validation for you. Whether triggered by a test result or a new release deployment, detected events work as a trigger to check the defined objectives and derive an overall status automatically. Based on the results, further actions are taken, like sending targeted notifications to the service and application owners. This is all available out-of-the-box with the default workflow template provided by Site Reliability Guardian.

Workflow leveraging Site Reliability Guardian
Workflow leveraging Site Reliability Guardian to make release decisions.

In addition, the workflow can be easily extended to any custom demands, for example, integration with tools that support your software product lifecycle. This includes executing tests, running Dynatrace Synthetic checks, or creating tickets. In this way, Dynatrace supports an ecosystem of tool integrations that unlock the full potential of DevOps and SRE with cloud-native delivery orchestration and remediation automation use cases.

What’s next?

Site Reliability Guardian is built for ensuring and maintaining production reliability and resilience. Further enhancements will focus on seamlessly integrating the solution into your existing pipelines to help you shift left and block poor-quality releases before they reach production. Davis will also assist Site Reliability Guardian in recommending relevant objectives and baselines for comparison.

  • For an early preview of this new functionality, please request Dynatrace platform preview access, which allows you to install the Site Reliability Guardian via Dynatrace Hub.
  • The Dynatrace Site Reliability Guardian will be available within the next 90 days. In the meantime, watch out for upcoming “Observability Clinics” and “Ask me anything” sessions covering the main topics of this blog post. You can either see a list of the next webinars on our website or follow us on LinkedIn to stay up-to-date with upcoming announcements and activities.

The post Automated Change Impact Analysis with Site Reliability Guardian appeared first on Dynatrace news.

]]>
https://www.dynatrace.com/news/blog/site-reliability-guardian/feed/ 0