The business observability gap in cloud-based transactions

Business observability, at its best, delivers real-time insights into business health, reporting, and interpretation of the metrics and KPIs that business and IT leaders need to make decisive and effective decisions.

But how can you access this data when cloud-based solutions deliver your critical services? One of the more common examples is payment gateways, or payment service providers (PSPs)—hosted services that verify account details, check for available funds, and transfer money into your business merchant account.

There’s a wealth of business data in your systems’ transaction requests to these PSPs and other SaaS solutions. Without visibility into this data, you’re left with an incomplete and delayed understanding of transaction success, sales data, and business health. This is particularly problematic since the transaction success or failure status is often embedded within the response body.

Here are some of the challenges our customers have identified:

• Delays in resolving issues. Failed transactions are only discovered in scheduled reports or when customers complain.
• Reconciliation discrepancies. Inconsistencies between internal records and PSP reports lead to frustrating, time-consuming, and error-prone manual work.
• Inability to understand short-term trends. Identifying failure patterns for specific providers or transaction types is nearly impossible without timely and detailed transaction data.
• Compliance risks. Lack of granular detail makes auditing and demonstrating compliance with payment regulations more challenging.

Business events from outgoing web requests

Dynatrace business events are in a special class designed to support even the most demanding business use cases. Whether derived from OneAgent, log files, RUM sessions, or APIs, business events deliver real-time lossless access to critical business data—wherever it lies. In particular, OneAgent’s unique capability to extract business data from in-flight application payloads—without writing any code—has proven invaluable to hundreds of our customers.

OneAgent now supports capturing business events from outgoing web requests that are specifically designed to illuminate these blind spots. You can now transform critical business information within your outgoing web requests and their corresponding responses into structured business events, which can then be analyzed, correlated, and acted upon within Dynatrace.

Use case: Business events and payment gateways

Here’s how it works, and what it means for our payment gateway use case.

Granular configuration for targeted capture

• Define endpoint-specific rules to capture data from requests to specific URLs or IP addresses (for example, api.stripe.com/payments or paypal.com/checkout).
• Define method-specific rules to capture data from relevant HTTP methods like POST or PUT that are used to initiate transactions.

Deep insight into request and response payloads

Support for business-event capture from outgoing web requests adds compelling new value and expands on current OneAgent capabilities for extracting business data from in-flight application payloads.

• Extract data from the request body. Imagine you’re sending payment requests that include customer account numbers, transaction amounts, and transaction identifiers. You can now define rules to extract these key pieces of information directly from your JSON or XML request bodies. This is crucial for linking your internal transaction IDs to the business data sent to the provider.
• Extract data from the response body. Potentially greater value lies in the response body. Often, PSPs return transaction status messages (for example, transactionStatus:success, authorizationResult:approved, or responseCode:1954) within the response body. Extracting these status messages allows you to report, alert, and act on failures or anomalies as they occur.

Transform raw data into meaningful business insights

Business data increases in value when enriched with business and IT attributes. For example, you could map extracted fields to meaningful attributes of a “Payment Transaction” business event. These attributes might include payment_provider_id, transaction_amount, and customer_id, extracted from the request body, and payment_status extracted from the response body.

These events are automatically enriched with Dynatrace observability data, such as the request’s originating host, service, user journey, or geographical source, facilitating business and IT collaboration.

Real-time visibility and proactive alerts

• Real-time business transaction monitoring. Know each PSP transaction’s success, failure, and value as it happens, not hours later.
• Custom dashboards. Dashboards make it easy to visualize different PSPs’ success rates and revenue, exposing bottlenecks and highlighting business and failure trends.
• Proactive alerting. Set up alerts based on real-time payment status messages. Trigger alerts on failed transactions over a certain amount, or when the success rate SLA degrades.

Enhanced traceability and root cause analysis

• End-to-end transaction tracing. Trace the entire lifecycle of a payment, from the user initiating the purchase through your internal services to the outgoing request to the PSP and the response status indicating success or failure.
• Accelerated root cause analysis. Quickly determine the source of payment transaction failures, bottlenecks, and anomalies to speed resolution and improve business outcomes.

Payment gateway example: From opaque to transparent

Here’s what this could mean for your organization:

1. A customer attempts a major purchase. Your e-commerce system sends the request to your PSP, but the transaction is declined.
2. OneAgent captures relevant business data from the outgoing request, including customer ID and transaction amount attributes. Dynatrace also captures the relevant response fields, including the status:declined message.
3. A “Payment Failure” business event is automatically generated in Dynatrace. This event triggers an alert to notify your operations and customer support teams.
4. Your customer support team can now proactively contact the customer while your operations team works to resolve the issue, knowing exactly why the transaction failed and which provider was involved.

This is possible without any code changes. Extracting business data from outgoing requests and responses turns opaque customer interactions into transparent, actionable business insights, empowering you to improve customer experience, optimize payment workflows, and reduce operational risk.

Initial support for outgoing web request capture is available for Java; see Supported Technologies for details. Additional technology support will be available soon.

The post Extend business observability: Extract business events from outgoing web requests appeared first on Dynatrace news.

Dynatrace Business Flow simplifies business process observability, connecting top-level process KPIs with detailed flow analytics. The app tracks the progress of every process instance, reporting individual and aggregated process performance, throughput, exceptions, and business outcomes.

With this update, Davis® AI can track and alert on KPI threshold violations to assure end-to-end process efficiency and reliability.

Anomalies are inevitable

Despite the deep IT observability you might have deployed, many of your business processes will likely encounter failures, intermittent delays, or ongoing inefficiencies. These issues can impact a wide range of business outcomes, from a direct reduction in revenue to a decrease in Net Promoter Score (NPS), from idle production lines to departmental budget overruns. Each business unit relies on a collection of processes, and each process has metrics and KPIs that can be affected by delays, exceptions, or failures. Examples include:

• A stalled connection between two services delays the process of validating food orders, resulting in dissatisfied customers, idle delivery drivers, and canceled orders.
• A failure in a third-party API breaks a loan validation process, reducing the number of applications approved and directly impacting the company’s revenue.
• A delay in warehouse order processing results in late deliveries, impacting customer experience.

Business process observability

Real-time business process observability can minimize the impact of these and other incidents by ensuring anomalies are detected quickly and include the IT context required for rapid—even automated—remediation. Business process observability is a new approach to ensuring process efficiency and reliability, quickly detecting and remediating anomalies, and identifying optimization opportunities. Are detected business exceptions specific to one host in a cluster? Which services does the credit check API depend on? Is there a fund transfer bottleneck at the source or destination bank?

Business process observability becomes increasingly important as new regulatory frameworks like the Digital Operational Resilience Act (DORA) impose strict operational resiliency requirements on financial and other institutions.

Dynatrace Business Flow

Business Flow is a Dynatrace® App that delivers on the promise of business process observability, connecting top-level process KPIs with supporting step- and instance-level analytics. Some of the questions Business Flow answers include:

• What is the average end-to-end process delay? How does this change over time?
• Are the business goals achieved? (For example, average loan amount, service response time, or close rate.)
• What is the business exception rate? Is there a trend?
• Which process steps are most problematic? Which steps take longer than normal?
• Were optimizations successful in their goal of improving performance or reducing exceptions?
• How does one process path differ from another regarding exceptions and throughput?
• What is the current status of a specific customer order?

Business Flow: KPI alerts

The latest version of Business Flow introduces a significant enhancement to support anomaly detection and alerting. App users can configure alerts for any of the four reported business process KPIs, leveraging Davis AI Anomaly Detection for continuous KPI analysis.

Alerting can be configured for each of the app’s four business process KPIs:

• Conversions or fulfillments. How many process instances—claims, deliveries, payments, etc—have successfully completed the final step?
• Errors or business exceptions. Distinct from IT errors, these are process-specific exceptions, usually defined in the application and reported within the application payload.
• Average duration. This is the cycle time of process instances, the elapsed time from start to finish.
• Custom business KPI. This can be any user-defined metric extracted from process instances. Examples include revenue, order quantity, loan amount, and approval rate.

Configure anomaly detection

Use the time series charts from the KPIs over time view to configure Davis AI Anomaly Detection for any of the four key business process KPIs. Each KPI graph includes a Create alert button.

To create an alert for a KPI, select Create alert to open the Davis anomaly detection configuration page. Choose the analyzer type: auto-adaptive, seasonal, or static threshold.

The configuration window is from the Davis AI Anomaly Detection app, adapted for use with Business Flow KPIs. Since Business Flow automatically defines the DQL for the KPIs, the anomaly detection configuration is quite simple.

There are three Davis AI Analyzers to choose from:

• Static thresholds are fixed and don’t change over time
• Auto-adaptative thresholds are calculated by Dynatrace automatically and adapt dynamically to your data’s behavior
• Seasonal baseline thresholds adapt to observed seasonal baselines and are charted with a confidence band

Alerts can be triggered for any of the three Davis Analyzers whenever the metric falls above, below, or outside the defined thresholds. Dynatrace Documentation provides configuration guidance for Davis AI Anomaly Detection.

The actor—or simulated Dynatrace end-user operated by the Davis AI Anomaly Detector in automated workflows—can be a service user or an interactive user with privileges to run the anomaly detector and execute the DQL queries for the time series chart. For more details, see service users.

Once configured, the new Davis AI Anomaly Detector evaluates a certain metric each minute, creating problem events when alert conditions are met. The time series charts add alert annotations to facilitate further analytics in such cases.

What’s next

For more information about alerting on business process KPIs, please visit Dynatrace Documentation.

You can also try out the new Business Flow alerting feature in the Dynatrace Playground.

Note that anomaly detection currently supports process flows of less than 60 minutes. Support for longer-running processes will be added in a future release.

The post Business process observability: Alerting on process KPIs appeared first on Dynatrace news.

Globally, organizations are increasingly prioritizing sustainability. Board-level mandates and executive-level sponsorship have become the norm. Motivations include regulatory pressures, favorable investment potential, cost savings, competitive advantage, talent acquisition, and moral imperative. While a dynamic political climate can disrupt how these drivers are prioritized, the goal remains important, and momentum remains strong.

IT carbon footprint is top of mind for observability professionals in many industries. The explosion of AI models shines a new spotlight on the issue, with a recent study showing that using AI to generate an image takes as much energy as a full smartphone charge. For IT, a natural focus should be reducing carbon emissions from electricity consumption from on-premises, hybrid, and multicloud computing.

The challenge along the path

Well-understood within IT are the coarse reduction levers used to reduce emissions; shifting workloads to the cloud and choosing green energy sources are two prime examples. To continue down the carbon reduction path, IT leaders must drive carbon optimization initiatives into the hands of IT operations teams, arming them with the tools needed to support analytics and optimization.

Cloud service providers (CSPs) share carbon footprint data with their customers, but the focus of these tools is on reporting and trending, effectively targeting sustainability officers and business leaders. Unfortunately, they lack the detailed IT context practitioners need to identify carbon reduction opportunities and analyze optimization solutions. While stand-alone tools attempt to complement CSP solutions, they face similar challenges:

• Data granularity and accuracy: Tools struggle with providing detailed and accurate data on carbon emissions. This is partly due to the complexity of instrumenting and analyzing emissions across diverse cloud and on-premises infrastructures.
• Real-time monitoring: The periodic reports from cloud service providers lack real-time monitoring and actionable insights, limiting IT teams’ ability to make immediate adjustments to reduce carbon footprints.
• Integration with existing systems and processes: Integration with existing IT infrastructure, observability solutions, and workflows often requires significant investment and customization.
• Quantifying impact: Quantifying the effectiveness of various emission reduction methods can be challenging, making it difficult for IT leaders to prioritize actions and justify sustainability investments.

The Dynatrace carbon optimization solution

Introduced in 2023, Carbon Impact helps practitioners identify and evaluate meaningful opportunities to optimize their cloud and on-premises infrastructures. The Carbon Impact app directly supports our customers’ sustainability efforts through granular real-time emissions reporting and analytics, translating host utilization metrics into their CO2 equivalent (CO2e). By leveraging existing OneAgent® instrumentation, customers can get started in minutes with no new instrumentation hurdles.

Today, Carbon Impact has a new name: Cost & Carbon Optimization. The name change reflects the app’s expansion into cloud cost analytics, driven by the understanding that optimization approaches for both cost and carbon mostly overlap. You’ll be able to read more about our approach to cloud cost optimization in an upcoming blog post.

Industry certification for Dynatrace Cost & Carbon Optimization

To enhance the trust our customers and partners have in our approach, we commissioned the Sustainable Digital Infrastructure Alliance (SDIA) to test and certify the Cost & Carbon Optimization app. The certification focuses on accuracy and transparency in calculating greenhouse gas (GHG) emissions for AWS, Azure, GCP, and on-premises host instances.

The certification results are now publicly available.

“The SDIA has certified Dynatrace Cost & Carbon Optimization app, ensuring its accuracy and adherence to the environmental management principles outlined in ISO 14004.

“Dynatrace Cost & Carbon Optimization is a reliable estimation system to calculate the operational GHG emissions of IT infrastructure both in cloud and on-premises environments. The calculations and methodology used are in line with the best available scientific approach, as well as with relevant reporting requirements. Further, Dynatrace meets the requirements on transparency, allowing customers to have insight into the way these metrics are calculated, as well as recreating the measurements themselves for independent verification.”

Actions resulting from the evaluation

The certification process surfaced a few recommendations for improving the app. These are the outcomes:

• We replaced GCP’s emissions estimations with more accurate data from Ember, a global not-for-profit clean energy think tank.
• We implemented a wasted energy metric in the app to enhance practitioner actionability.
• We are updating product documentation to include underlying static assumptions.

Transparency breeds confidence

In the spirit of transparency, we’re publishing details of the data sources and assumptions the app makes. Some highlights:

• Data conversion: To derive CO2e and wasted energy metrics, we convert OneAgent data using external sources:
  • Energy emission data is sourced from the European Energy Agency and the Cloud Carbon Footprint tool to determine emission factors specific to cloud data center locations.
  • Thermal design power (TDP) values are derived from AMD and Intel to calculate CPU power consumption.
  • Power usage effectiveness (PUE) is derived from data provided by the cloud providers and data center operators.
• Static assumptions: To complete the calculations, we apply these assumptions:
  • Memory power calculations assume a static draw value of 3 W for each 8 GB memory module.
  • Network traffic power calculations rely on static power estimations for both public and private networks. These estimates are converted using the emission factor for the data center location. Static assumptions are:
    • Local network traffic uses 0.12 W per GB.
    • Public network traffic uses 1.0 W per GB.
  • CPU calculations apply these assumptions:
    • A virtual CPU (vCPU) on any cloud host equals one thread of a physical CPU core, with two threads per core.
    • A CPU operating at 100% utilization consumes power equal to its TDP.
  • Storage calculations assume that one terabyte consumes 1.2 Wh.
  • Cloud storage is replicated twice, which doubles the energy consumption per terabyte.
  • Annual country-level emission factors from the European Environmental Agency are used as input to the calculations.

Connecting carbon reduction to cloud cost optimization through the FinOps Framework

As we merge cost and carbon analytics into a single solution, we expect to reduce redundant or conflicting optimization efforts. Approaches to optimizing IT carbon emissions overlap significantly with those applied to cloud cost management (CCM). Optimizing resource utilization and reducing waste benefit each of these goals. The FinOps Foundation includes sustainability in its framework, stating, “FinOps and cloud sustainability mutually support each other through a similar approach to conscientious and responsible technology usage that enables workload efficiency. If you’re doing one of these – you’re amplifying the other.”

The post Dynatrace Cost & Carbon Optimization certified for accuracy and transparency appeared first on Dynatrace news.

Business processes support virtually all aspects of an organization’s operations. They’re often categorized by their function; core processes directly create customer value, support processes increase departmental efficiency, and management processes drive strategic goals and compliance. Sometimes overlooked is a fourth category we might call “long-tail processes;” these are the ad hoc or custom workflows that develop in response to gaps between systems, applications, departments, or workflows. Regardless of their role, every business process is designed to improve business outcomes.

Most organizations have hundreds of business processes across these four categories, supported by IT systems through a mix of on-premises, cloud, and SaaS solutions. Business processes are valuable assets, and the lack of visibility is alarming, representing both risk and opportunity.

The whole is other than the sum of its parts

Processes are typically quite complex, involving multiple disparate systems, external interfaces, and human interactions. Despite the deep IT observability you may have deployed, you still can’t infer process health from system status; problems occur even when the underlying infrastructure is healthy. Many of your business processes likely encounter failures, intermittent delays, or ongoing inefficiencies, yet your observability solution likely remains oblivious to these issues.

Given the business-critical nature of many of these processes, it’s clear that the risk from unobserved processes is substantial, tangible, and an unfortunate shared experience.

Is business process management the answer?

Cost and complexity are two primary factors that limit the adoption of business process management and mining solutions. License fees are only the tip of the cost iceberg; it’s not uncommon for a process mining implementation to take many months and cost over $100K per process. Complex data integrations require significant skilled work and coordination from multiple teams and require ongoing maintenance. These factors often limit deployment to a few mission-critical processes, usually managed by a business process management (BPM) center of excellence. This centralized approach can compel organizations to prioritize process monitoring and optimization initiatives to just a few mission-critical processes while neglecting those with less obvious—though significant—impact on business outcomes.

At its best, business process management embraces collaboration between business teams responsible for optimizing business outcomes and IT teams responsible for system observability. But even the best BPM solutions lack the IT context to support actionable process analytics; this is the opportunity for observability platforms.

Enter business process observability

Business process observability determines process health and performance through data gathered at each step, leveraging appropriate levels of system monitoring or interfaces. Log files and APIs are the most common business data sources, and software agents may offer a simpler no-code option.

• Transaction data (such as purchase confirmations, loan approvals, or service requests) is used to calculate inter-step timings and capture the step’s outcome (for example, success, failure, or conditional branch).
• Transaction metadata (such as a product SKU, loan amount, or service address) enriches process insights and increases analytic granularity.
• Transaction context (such as the responsible host, service, or system process) connects each step to its supporting IT infrastructure for troubleshooting and optimization.
• Correlation IDs (such as a transaction ID, customer number, or order number) are used to reassemble discrete transactions into a sequenced end-to-end flow to support detailed process analytics.
• Process health is determined through calculated metrics and process-specific KPIs such as throughput, revenue, cycle time, and exception rate.

To summarize, business process observability:

• Treats the process as a business asset or entity with unique discernable flows.
• Determines process health via data collected at each step, reported as process-specific business KPIs.
• Includes end-to-end and step-specific details for every process instance.

Process observability benefits include enhanced operational efficiency, reduced costs, increased productivity, compliance visibility, and improved customer satisfaction. These benefits come from robust process analytics, often augmented by AI. They’re initially realized reactively and, as organizations mature, proactively, in two key categories:

• Remediation: When process KPIs degrade, process analytics uncover the root cause to inform or automate service recovery.
• Optimization: Whether an ad-hoc initiative or a continuous improvement function, process analytics provide the data-driven foundation for process optimization.

The Dynatrace advantage

Few observability platforms offer business process observability. At best, they group metrics from isolated process steps into a linear process-like view, ignoring the complexity of real-world process flows and lacking end-to-end flow details to support effective analytics and optimization.

Dynatrace treats your business processes as true business assets, connecting business KPIs with end-to-end process analytics to meet the collaboration and optimization needs of business and IT teams. Dynatrace makes it easy to get started and cost-effective to implement broadly.

Dynatrace business process observability leverages key platform capabilities, including:

Business events, which provide easy access to business data, are automatically enriched with IT context. Business events can come from:

• OneAgent – a unique capability offering configurable no-code access to in-flight application payload.
• Log files – using OpenPipeline to extract and transform business data while reducing log management and storage overhead.
• API – to ingest data from relevant business systems.
• RUM – for high-precision user journey analytics.

Business Flow is a Dynatrace® App that simplifies business process configuration, reporting, and analytics.

• Easy configuration of process flows
• Automatic KPI reporting
• Granular flow details

Simplified access to business data coupled with a purpose-built app makes it easy to get started with business process observability. It’s not uncommon for Dynatrace customers to implement Business Flow in just a few weeks, often without any development effort, with only incremental subscription costs. To learn first-hand about some of their successes, check out a few of these Business Analytics customer breakouts from our recent Perform user conference. You can also try it yourself in our Dynatrace Playground (registration is required).

Wherever you might be on your transformation journey, business processes form the framework that drives better business outcomes. This is an opportune time to expand the value IT delivers to your business teams.

The post Business process observability: An IT solution to a business challenge appeared first on Dynatrace news.

There’s a goldmine of business data traversing your IT systems, yet most of it remains untapped. To unlock business value, the data must be:

• Accessible from anywhere. Data has value only when you can access it, no matter where it lies.
• Easy to access. Simplicity accelerates time-to-value and reduces implementation and maintenance costs.
• Real time. Agile business decisions rely on fresh data.
• Precise. Accuracy provides the confidence needed for business automation.
• Contextualized. Metadata enrichment improves collaboration and increases analytic value.

The Dynatrace® platform continues to increase the value of your data—broadening and simplifying real-time access, enriching context, and delivering insightful, AI-augmented analytics. Our Business Observability solution is a prominent beneficiary of this commitment.

Business events: Delivering the best data

It’s been two years since we introduced business events, a special class of events designed to support even the most demanding business use cases. Since then, many of our customers have embraced the opportunity to explore and adopt new business observability use cases. Most of these leverage the unique capability of Dynatrace OneAgent® to extract business data from in-flight application payloads—without writing any code. Other data sources, including APIs and log files—are used to expand access, often to external or proprietary systems. Enhancing access to business data from log files is an important priority, and OpenPipeline makes this a reality.

Dynatrace OpenPipeline is a new stream processing technology that ingests and contextualizes data from any source. You can now use OpenPipeline to extract business events from log files, complementing OneAgent as a primary source of business data. This is especially important when legacy or proprietary systems don’t meet OneAgent’s environmental prerequisites; in these cases, log files are usually the preferred source of business data.

In fact, it’s likely that some of your critical business systems already write business data to log files. For years, logs have been the dominant approach many observability vendors have taken to report business metrics on dashboards. You might still be using one of these tools despite the drawbacks, which include a lack of IT context, constraints on data privacy and data retention, and, as the only convenient source of business data, a limiting lack of breadth. OpenPipeline makes migrating these use cases to Dynatrace easy, helping you overcome these limitations to realize greater value.

Use case categories

It’s helpful to categorize the nearly unlimited use cases covered by Business Observability. Two of the most important categories are:

• Business reporting, analytics, and automation. Track business metrics, key performance indicators (KPIs), and service level objectives (SLOs)—automatically and in context with IT infrastructure and services—to promote collaboration between business and IT teams.
• Business process monitoring and optimization. Monitor and optimize business processes with real-time visibility into process KPIs and detailed analytics for each step to improve customer satisfaction, increase operational efficiency, and reduce cost.

Most of the use cases in these two broad categories benefit from the flexibility that comes from multiple available sources of business data. For example:

• An airline’s reporting and analytics dashboard includes data showing flights, passengers, available seats, passenger load, revenue per passenger, flight crew staffing, arrival delays, and customer satisfaction metrics. The data may come from a mix of systems, including a departure control system (DCS), an airline reservation system (ARS), a legacy inventory control system, and a SaaS-based Voice of the Customer (VoC) solution.
• A financial institution’s loan origination business process includes a series of process milestones, including loan application, credit scoring, application review, contract generation, CRM, and loan funding. All of these steps are critical components of the process, likely to be implemented using different systems. Furthermore, to understand process health, individual steps must be viewed in the context of the entire process. For this, we use Business Flow to track, analyze, and optimize complex business processes, treating the process as an observable IT and business asset.

Use OpenPipeline to extract business events from logs

Using OpenPipeline, you can ingest log data into Dynatrace from a wide range of sources, including OneAgent, Extensions, the Log ingest API, and OpenTelemetry. The pipeline includes a configurable business event processor as part of the data extraction stage; this processor is used to extract and convert relevant business data into business events.

There are many benefits to extracting business events from log files. These include:

• Improved data privacy. Sensitive business data is separated from IT observability data.
• Improved data management. Fine-grained permission and retention policies can be tailored to individual business use cases.
• Reduced storage and query overhead for business use cases. Business events are a small, often negligible subset of log data.
• Simplified and enhanced analytics efficiency. Business events from log files are unified with business events from OneAgent, RUM, and API.

Extracting business events from logs: Configuring OpenPipeline

The OpenPipeline app guides you through the three configuration steps required to extract business events from log files:

1. Identify the source of the log file.
2. Define OpenPipeline’s dynamic routing matching criteria used to route the log lines of interest to a second pipeline.
   
3. Create the target pipeline. Use the Data extraction tab to define the matching condition that creates the business event. You need to include static or dynamic Event type and Event provider fields.
   

Within the target pipeline, you can also define processing rules, extract metrics, set the security context, and define retention periods. Log data is then processed accordingly, stored in Dynatrace Grail™ causational data lakehouse, and available for your Business Observability use cases.

The value is in the data

OpenPipeline broadens Dynatrace’s unified access to the best data to deliver the best value. Now’s the time to see how it can benefit your organization.

For more details about OpenPipeline read this Dynatrace OpenPipeline blog post.

The post OpenPipeline: Simplify access to critical business data appeared first on Dynatrace news.

The business process observability challenge

Increasingly dynamic business conditions demand business agility; reacting to a supply chain disruption and optimizing order fulfillment are simple but illustrative examples. Business agility requires real-time visibility into process health and performance, measured by business Key Performance Indicators (KPIs) that are shared between business stakeholders and the supporting IT operations teams. However, business processes can be inefficient, broken, or violate Service Level Objectives (SLOs) even when the underlying system’s health is good; a process is greater than the sum of its parts. A business process only becomes observable when a) it is defined as a unique entity or asset and b) process-specific business KPIs are used to gauge health.

Business Process Management (BPM) solutions might seem like the answer, but they too often suffer from legacy technical constraints. Many are platform-centric, unable to expand beyond proprietary borders to embrace more common heterogeneous environments. They can be expensive to implement and maintain, rely on fragile data pipelines, and require highly skilled data analysts to ensure ongoing relevance. BPM solutions lack IT context, making them ill-suited for collaboration between business and IT operations teams.

Most business processes are not monitored. Why?

First and foremost, it’s a data problem. Locked away in disparate systems, difficult to access, and with different formats, business data can be days or weeks old. As a result, it takes significant effort to build and maintain these often fragile data pipelines. Undue reliance on log files as the primary business data source adds development overhead for implementation and maintenance, further limiting agility.

If you can collect the relevant data (and that’s a big if), the problem shifts to analytics. Business processes can be quite complex, often including conditional branches and loops; many business process monitoring initiatives are abandoned or simplified after attempting to map the process flow. Connecting data from different systems, stitching process steps together, calculating delays between steps, alerting on business exceptions and technical issues, and tracking SLOs are just some of the requirements for an effective analytics solution.

As a result, most business processes remain unmonitored or under-monitored, leaving business leaders and IT operations teams in the dark. Business health and IT health remain disconnected in separate silos, limiting opportunities for effective collaboration. The resulting business process blind spots delay response to business disruptions, leading to dissatisfied customers, BizOps friction, and inefficient IT resource allocation.

Business events: Addressing the data challenge

Dynatrace business events address the data challenge by making it easy to access real-time business data. Business events can come from anywhere—OneAgent^®, log files, Real User Monitoring (RUM) sessions, or external systems through an API. They deliver the real-time precision needed for confident data-driven business decisions. Uniquely, OneAgent can capture business data from in-flight application payloads, eliminating the need for application changes (for example, to write business data to a log file). Regardless of the source, business events are unified in Grail^® and are automatically enriched with Smartscape^® topology context, connecting business data directly to the supporting IT infrastructure.

Business Flow: Addressing the analytics challenge

Dynatrace addresses the analytics challenge with Business Flow (available on Dynatrace Hub), which was built using Dynatrace AppEngine. Initially released in April 2023, Business Flow masks analytics complexity through simple business process configuration and an intuitive interface. With Business Flow, you can:

• Connect business events from any source into an end-to-end process view.
• Report end-to-end process delays and measure delays between each step.
• Identify drops at each step.
• Report business exceptions at each step.
• Drill into the details of any step or process flow.
• Report process KPIs, including completed flows, average flow completion time, business exceptions, and a customizable business KPI.

Business Flow

The newest release of Business Flow introduces significant enhancements to cover more complex business processes and increase the depth of analysis.

To reduce the complexity of business process monitoring, it’s good practice to evaluate whether some process steps can be abstracted by monitoring only key milestones. For cases where process abstraction is not desirable, Business Flow adds two new features:

• Process branches – Business Flow now supports process branching. A branch can be conditional—only active for certain flows—as in the case of a document flagged for manual auditing. Branches can also represent alternate paths, one of which must be followed. A step may have up to five branches.
• Increased number of process steps – Business Flow now supports up to 20 steps in a single flow. Note that each branch is counted as a step when calculating the total number of steps in a process.

Business events are automatically connected into an end-to-end flow using a correlation ID such as order_number. Using consistent labels at each process step across all systems is good practice. For cases where the labels differ between systems (order_number at one step and order_id at another), Business Flow now supports step-specific local correlation IDs, overriding the global ID configured for the process.

New detailed flow views enhance process analysis. At each step, you can view a list of unique flows that a) pass through that step, b) are dropped at that step, or c) have been classified as in-flight (not yet complete). Each unique flow can be examined in its entirety to view the timestamps and attributes of the associated business events.

Greater business process complexity increases the potential for configuration errors. Business Flow now alerts on two conditions:

• Missing correlation ID – When a business event lacks a correlation ID, Business Flow can’t connect it to a unique end-to-end flow. Some use cases might benefit from isolated step metrics, but these are rare.
• Out-of-sequence flows – Business Flow generates an alert when an individual business flow skips a step. This condition can occur if a conditional process branch exists but has not been configured in Business Flow. A skipped step may also indicate a process anomaly worthy of investigation.

Business processes are the heart of modern organizations. IT operations and business teams benefit from a shared view of process health, using business KPIs as primary health indicators. Business Flow makes monitoring, analyzing, and optimizing complex business processes easier than ever.

The post Business Flow: Why IT operations teams should monitor business processes appeared first on Dynatrace news.

As global warming advances, growing IT carbon footprints are pushing energy-efficient computing to the top of many organizations’ priority lists. Energy efficiency is a key reason why organizations are migrating workloads from energy-intensive on-premises environments to more efficient cloud platforms. But while moving workloads to the cloud brings overall carbon emissions down, the cloud computing carbon footprint itself is growing.

“The cloud now has a greater carbon footprint than the airline industry,” wrote anthropologist Steven Gonzalez Monserrate in a 2022 article from MIT. “A single data center can consume the equivalent electricity of 50,000 homes.” The growing adoption of innovations like generative AI, based on large-language models (LLMs), will only increase demand for cloud computing. This adoption will further impact carbon emissions. Research from 2020 suggests that training a single LLM generates around 300,000 kg of carbon dioxide emissions—equal to 125 round-trip flights from New York to London.

Does that mean the answer is to slow the growth of AI or cloud technologies more broadly? Given the benefits of these innovations, organizations can’t afford to pull back on their efforts to build AI and shift more workloads to the cloud. However, organizations can turn to innovative solutions to improve their energy efficiency by mitigating their cloud computing carbon footprint.

How Dynatrace tracks and mitigates its own IT carbon footprint

Like many tech companies, Dynatrace is experiencing increased demand for its SaaS-based Dynatrace platform, which we host on cloud infrastructure. As we onboard more customers, the platform requires more infrastructure, leading to increased carbon emissions. At the same time, many existing customers are migrating from Dynatrace Managed, our on-premises solution, to our SaaS offering. These migrations add to the Dynatrace cloud computing carbon footprint as we onboard more customers’ observability and security workloads. However, since moving on-premises workloads to the cloud can lower the overall carbon footprint by 80% or more, the result is a net reduction in carbon emissions.

Nonetheless, to help mitigate climate change, it’s critically important for organizations to measure, monitor, and reduce their IT carbon footprints. Certainly, this is true for us. We also recognize that many of our customers have the same need. Many cloud service providers offer tools that measure a subscriber’s cloud computing carbon footprint when using their service. But they don’t measure the carbon footprint of the many apps and infrastructure resources running across that subscriber’s multicloud environments. They also can’t assess the IT carbon footprint of a subscriber’s on-premises apps and infrastructure. That’s why we developed Carbon Impact.

The Carbon Impact app assesses carbon emissions and energy consumption from all monitored hosts. It also provides organizations with actionable guidance for how to reduce their overall IT carbon footprint. Developed using guidance from the Sustainable Digital Infrastructure Alliance (SDIA) and expanding on formulas from the open source project Cloud Carbon Footprint, Carbon Impact measures and reports the IT carbon footprint of all Dynatrace-monitored hosts across an organization’s entire hybrid and multicloud environment in a single interface.

Assessing our baseline cloud computing carbon footprint

Using Carbon Impact, we can assess our baseline carbon footprint with accuracy and granularity that’s nearly impossible to glean from other sources. The app’s advanced algorithms and real-time data analytics translate utilization metrics into their CO2 equivalent (CO2e). These metrics include CPU, memory, disk, and network I/O. This analysis provides us with a holistic view of our multicloud environment’s carbon emissions and identifies major emissions sources. As a result, this baseline measurement has become an important component of our sustainability strategy. It increases our awareness across IT and business stakeholders as we use these insights to build action plans to reduce our emissions and track the results of those efforts.

Tracking cloud computing carbon footprint by host

The ‘Hosts’ view details energy and CO2e consumption per host with filters to help narrow the focus to high-impact areas. For example, Dynatrace has been able to view underutilized instances in a specific AWS data center along with top CO2e emitters within a specific host group.

Optimizing host idling and scaling to reduce IT carbon footprint

Carbon Impact automatically reports idle and under-utilized instances as targets for optimization. Because Carbon Impact is integrated with Dynatrace Smartscape® topology modeling, it’s easy to drill into host and process details. Or open a Notebook for ad hoc analysis, giving us insights so we can safely scale down or retire underutilized instances. Using these recommendations, we focused our reduction goals on instances with the highest potential impact, shifting workloads and resizing instances where appropriate.

Helping organizations track their IT carbon footprint to forge a greener future

At Dynatrace, we’re committed to measuring and reducing our own greenhouse gas emissions. By extension, we want to enable our customers to do the same. The Dynatrace unified observability and security platform makes this possible. Carbon Impact is an example of our contribution to making IT more energy-efficient and sustainable for everyone, even as AI is fueling the data explosion.

We built Carbon Impact using Dynatrace AppEngine, which customers and partners can also use to create additional custom, compliant, and intelligent data-driven apps. The AppEngine uses an easy, low-code approach to unlock the wealth of insights available in modern cloud ecosystems, including revealing where organizations consume their energy.

”As Dynatrace looks to the future, considering our environmental impact is more important than ever,” says Thomas Reisenbichler, VP of Site Reliability Engineering at Dynatrace. “We’re confident that Dynatrace will be able to optimize our cloud infrastructure carbon emissions by leveraging the Dynatrace platform for proactive management and orchestration, utilizing our Carbon Impact app to unlock greater optimization potential, and using green coding initiatives to improve performance.”

Using Carbon Impact, we can now implement efficiency measures driven by the app’s benchmarks and recommendations. Because it facilitates ongoing monitoring and tracks progress toward our sustainability goals, we can adjust our strategy to reduce our IT carbon footprint. By integrating sustainability into our growth strategy—and our product offering—Dynatrace is deepening its commitment to responsible business practices, transparency, and a greener future.

The post IT carbon footprint: Dynatrace Carbon Impact and Optimization app helps organizations measure cloud computing carbon footprint appeared first on Dynatrace news.

Digital experience is often considered the most important customer-facing aspect of digital commerce. This is typically the first thing that comes to mind for IT professionals working in the retail industry when evaluating holiday readiness. While digital experience has many facets, transaction speed usually ranks among the most important. Almost two decades ago, a Google experiment showed that fast-loading transactions are more important to customers than content quality—even small increases in transaction delay result in substantially more abandoned sessions. That lesson remains important. (Though the three-second rule for page load time is often misinterpreted).

CEOs of hybrid retailers prioritize e-commerce growth over in-store shopping, investing heavily in their online storefronts. IT teams spend months preparing for the peak traffic they anticipate will arrive with holiday shopping. However, this is a dynamic target; shopping behaviors are increasingly unpredictable, customer expectations continue to rise, and fierce competition makes cultivating loyalty more challenging than ever. These challenges can be summarized by this quote, paraphrased here from Adobe’s 2021 Digital Trends report: “Your customers are digital, unpredictable, and easy to lose.”

From first to lasting impressions

But there’s more to digital experience than speed. Digital experience, measured by fast, frictionless user journeys, paints an incomplete picture, tracking just the beginning of the customer relationship. What happens after the conversion creates a lasting impression with a larger impact on loyalty and your business.

Let’s shift our focus to the backend systems and business processes, the behind-the-scenes heroes of end-to-end customer experience. These retail-business processes must work together efficiently to orchestrate customer satisfaction:

• Inventory management ensures you can anticipate and meet dynamic customer demand.
• Order processing workflow is triggered by customer orders.
• Order fulfillment is the packaging and delivery of orders to customers.

From a customer perspective, the nuances of these business processes are uninteresting as long as they work. Increasingly, however, order fulfillment is a differentiating customer-facing aspect of the end-to-end customer journey, often with digital touchpoints woven into the experience. The fulfillment clock starts ticking the moment a customer purchases your product. Yet fulfillment is often an area over which retailers have little visibility or control.

Customers value real-time visibility into order status and delivery tracking. However, these fulfillment processes are often strained under the pressure of increased online shopping, next-day delivery expectations, and environment-friendly choices. Flexible delivery options, including “buy online, pick up in store” (BOPIS), curbside pickup, self-service lockers, and gig economy delivery require even greater real-time coordination to commit to competitive and narrowing delivery windows. Decentralized last-mile delivery strategies such as micro-fulfillment centers complicate inventory management and order fulfillment oversight.

Technology to the rescue?

Solutions such as inventory management, order management, and delivery optimization can introduce new challenges:

System integration. To effectively leverage multiple systems to manage orders, inventory, and logistics, retailers must invest in often complex integrations. Unsynchronized and siloed data prevents real-time decision-making and business automation.

Multi-channel logistics. Most retailers work with multiple carriers to handle deliveries, resulting in disparate tracking systems. Aggregating tracking information and presenting it to customers in a uniform way can be a challenge.

Real-time updates: Customers expect real-time visibility into fulfillment milestones beyond order confirmation, including packing, shipping, and delivery notifications. Self-service tracking information, preferred by most customers, becomes especially difficult when there are delays or disruptions.

Embracing business observability

Successful retailers benefit from real-time insights into business processes across all milestones. While each system and service provider might adhere to SLOs, the end-to-end health of the process is greater than the sum of its parts. How can you discover optimization opportunities, patterns behind recurring disruptions, or the root cause of an anomaly? The answer lies in the context—connecting business process KPIs to system performance becomes the starting point for real-time business/IT collaboration and automated remediation. The resulting agility supports targeted responses to process disruptions, anomalies, and bottlenecks as they happen, not when daily or weekly reports are produced, not when your call center is inundated, not when your Net Promoter Score (NPS) plummets. To accomplish this transformation, IT teams need to expand their observability scope to include business KPIs.

How Dynatrace can help

Recent platform innovations have made monitoring end-to-end business processes such as order fulfillment easier. Consider these requirements for effective business observability.

• Business data must be accurate to instill the confidence to make business decisions.
• Business data can come from many sources, including OneAgent, RUM, external business systems, and log files.
• Business data must be easy to access without modifying code to reduce the burden on development and maintenance resources.
• Business data must remain granular over long retention periods to support long-running business processes and “needle in the haystack” queries.
• Business data must be unified, regardless of the source or data type.
• Business data must be easily queried to answer unanticipated questions without upfront indexing.

Business events deliver real-time business observability to business and IT teams with the precision and context to support data-driven decisions and improve business outcomes. Business events extract critical business data from your IT systems with lossless precision and can illuminate dark data quickly and easily, wherever that data exists.

Order fulfillment process example

Retail order fulfillment is a good example of business process monitoring, a use case enabled by these innovations. Fulfillment processes vary between retailers, with subprocesses that might introduce branches and loops. It’s a good practice to identify process milestones as a starting point; these should be relatively consistent. For example:

1. Purchase confirmation
2. Order picked from the warehouse
3. Shipping label created
4. Order accepted by the delivery agent
5. Delivery confirmation
6. Survey completed

Once you’ve defined the list of milestones, identify where to capture the data.

• Purchase confirmation: E-commerce platform (via OneAgent)
• Order picked: Warehouse management system (via OneAgent)
• Shipping label created: Warehouse management system (via OneAgent)
• Order scanned by delivery agent: Agent logistics system (via API)
• Delivery confirmation: Agent logistics system (via API)
• Survey: VoC solution (via API or database query)

The Business Flow app, developed using Dynatrace® AppEngine, makes it easy to configure business process milestones for an end-to-end view of process throughput, delays, and anomalies.

The post Customer expectations for retail: Beyond digital experience appeared first on Dynatrace news.

There are many definitions of environmental sustainability, most of which converge on a common theme. Collectively and individually, we have a responsibility to act to protect global ecosystems and support health and wellbeing, now and in the future. Sustainability seeks a balance between human activity and the natural world.

As a result, environmental sustainability is one of the three pillars of environmental, social, and governance (ESG) initiatives. ESG embodies a set of criteria that guides an organization’s strategy, measures the organization’s impact, and informs potential investors across these three pillars. Specifically, the environmental criteria focus on an organization’s utilization of natural resources and the resulting impact on the environment. McKinsey summarizes the importance of this focus. “Every company uses energy and resources; every company affects and is affected by the environment,” the article states. ESG metrics are increasingly important to investors as they evaluate risk. In turn, these metrics are increasingly important to organizations because they measure and disclose their performance.

Balancing act or tipping point?

What motivates us to act? The answer, to varying degrees, is favorable investment potential, regulatory requirements, competitive advantage, cost savings, and moral imperative. And the time to act is now. In fact, in an article for the Wall Street Journal, Gartner analyst Stephen White anticipates that organizations will rapidly adopt performance metrics tied to energy consumption. “Sustainability is increasingly becoming a board-level issue with broad companywide mandates flowing down from chief executives to tech leaders,” White says.

Regulatory mandates are another driver that organizations can’t ignore. As a result, the Securities and Exchange Commission (SEC) in the US and the European Parliament in the EU are adopting stricter reporting rules that will apply to a larger number of companies. Globally, the International Sustainability Standards Board (ISSB), established at the UN Climate Change Conference in Glasgow, has developed requirements for climate-related disclosure. Sustainability reporting informs investors about sustainability risks and helps focus a company’s sustainability actions.

Your IT carbon footprint

Corporate carbon footprint calculations consist of multiple facets, including transportation, waste management, fuel, and electricity consumption. To that end, measuring, understanding, and reporting results are important precursors to intelligent reduction actions.

From an IT perspective, the sustainability focus is on carbon emissions from electricity consumption, specifically related to on-premises, hybrid, and multicloud computing. Some interesting facts:

• Moving a workload to the cloud can reduce its carbon footprint by up to 96%.
• Cloud computing has a greater carbon footprint than the airline industry.
• Average cloud server idle time exceeds 70%.

Yet most organizations don’t have the tools to measure, much less reduce, their IT carbon footprint.

Measuring your carbon footprint: from intention to action

Cloud providers offer tools to measure carbon emissions from the use of their cloud services. But these tools don’t support multicloud environments, and they don’t account for the footprint of on-premises services. More importantly, these tools are fundamentally backward-looking. They lack both the time and dimensional granularity required to derive carbon-emission analytics and optimization insights. Environmental sustainability emphasizes our collective responsibility to take action. For this, we need real-time intelligence and analytics, not just historical reports.

As executive mandates reach technology leadership teams, your organization’s carbon reduction goals will become more tangible. As a result, you’ll need to shift from intent to action, from passive reporting to active reduction initiatives.

Introducing Carbon Impact

The Carbon Impact app from Dynatrace measures and reports the carbon footprint of all Dynatrace-monitored hosts across your entire hybrid and multicloud environment in a single interface. The app translates utilization metrics, including CPU, memory, disk, and network I/O, into their CO2 equivalent (CO2e). Dynatrace engineers developed the app using guidance from the Sustainable Digital Infrastructure Alliance (SDIA), expanding on formulas from Cloud Carbon Footprint.

With Carbon Impact, you can explore the sources of your IT carbon footprint, regardless of where your workloads run. The app automatically identifies opportunities to reduce carbon emissions, capturing the details needed to analyze and make informed decisions. It also generates granular reports and host-level details to provide a focus for carbon reduction initiatives.

The Carbon Impact dashboard: Your IT carbon footprint overview

The Carbon footprint summary reports total CO2e emissions for the selected and preceding timeframes for quick interval-based comparisons.

Optimization summaries report idle instances and under-utilized instances. The app derives thresholds that influence these calculations from Google Compute Engine recommendations, although you have full configuration control to adjust these thresholds to your needs.

The table breaks down emissions by data center, listing your cloud and on-premises instances. All your carbon footprint data is in one place, and the measurement methodology is consistent across all your data centers.

Two charts complete the dashboard. The first shows accumulated carbon footprint and energy consumption over time. As cloud providers decarbonize their data centers, or as you move your workloads to more carbon-efficient locations, the slope of the carbon emission trajectory should change to illustrate the impact. The second chart allows you to compare carbon emissions over time with a business metric of your choice derived from any of your configured business events.

Instances: Where should you focus your attention?

The Instances view details energy and CO2e consumption per host instance. Filters help narrow your focus. For example, you could view underutilized instances in a specific AWS data center or top CO2e emitters within a specific host group. Because Carbon Impact automatically connects to Dynatrace Smartscape® topology modeling, it’s easy to drill into host and process details. Or you can open Notebooks for ad hoc analysis, arming your teams with the insights needed to evaluate opportunities to reduce carbon emissions.

Carbon Impact and Dynatrace AppEngine

Carbon Impact tracks, reports, and helps you reduce the carbon footprint of your cloud and on-premises infrastructure. It also provides data center, host, process, and application details to help you understand where to focus your efforts. Moreover, it provides recommendations to help you get started quickly. Significantly, Carbon Impact supports multiple audiences, including compliance teams as a source for reporting, business teams that need investment guidance, and operations teams that need to act to reduce their carbon footprint.

Carbon Impact is a purpose-built app we created using the new Dynatrace AppEngine. It addresses a well-defined business need, bringing custom analytics to Dynatrace data stored in Grail. Carbon Impact is available now on the Dynatrace Hub. To learn more, please contact your account team.

The post From observability to sustainability: Reduce your IT carbon footprint with Dynatrace Carbon Impact appeared first on Dynatrace news.

Business events are a special class of events, new to Business Observability; together with Grail, our data lakehouse, they provide the precision and advanced analytics capabilities required by your most important business use cases. Business events deliver real-time business data from anywhere and empower business and IT teams to analyze, discover, and automate to improve business outcomes.

Let’s look at the unique enabling characteristics of business events through two use case examples, then see how easy it’s to get started.

What are business events?

Business events are prioritized over metric events and observability data to deliver lossless precision. It’s an important distinction; many business use cases won’t tolerate sampling or other resource-conservation approaches common to operational monitoring. Inventory counts, revenue reports, and service activation flows are just a few examples that demand precision.

Business events capture business data from anywhere; they can come from OneAgent, from real-user monitoring (RUM) sessions, from log files, or from external tools and data sources. Dynatrace automatically enriches business events with topology metadata before storing them in Grail. From there, you use the Dynatrace platform to analyze, discover, and automate—without worrying about where the data is or where it comes from.

This diagram shows a few examples of business events. Note that most of these have a clear business—and not IT—focus, but that’s an arbitrary distinction; business events can include any data important to your use case.

Analytics without boundaries

There are other characteristics important for business use cases, delivered through Grail and Dynatrace Query Language (DQL). These include the following highlights:

• Long-term cost-effective storage to support seasonal trending and forecasting
• Unaggregated granularity to support “needle in a haystack” queries
• Instant indexless storage to support unanticipated retrospective questions
• Smartscape context to support drill-down analytics and promote collaboration

Let’s take a closer look at two use cases that are uniquely enabled by business events and Grail.

Use case 1: real-time business intelligence

Business intelligence tools have earned a reputation for being inflexible, lacking the context and real-time insights needed to understand and improve business processes and customer experience. They are generally backward-looking, relying on data stored in accounting and other business systems for a historical perspective. How can business teams gain the same real-time observability and analytics that IT teams enjoy?

Business events are the answer; with real-time precision and easy access to in-flight business data, Business Observability offers a compelling and agile complement to business intelligence tools. Consider the following business revenue use case:

A retail chain wants to boost the average checkout transaction value by encouraging customers to add additional items to their purchase. To track this goal, regional sales managers need real-time visibility into the number of items per transaction and the transaction value, segmented by store. But the company’s point-of-sale system limits daily reports to sales volume, with additional detail available only in monthly reports.

With a few simple configuration steps (outlined below), OneAgent begins capturing key sales data as business events, sending them to Grail in real time. Immediately, the sales managers can track transaction data for each store, focusing on the stores with the lowest items to transaction ratio.

Once they’ve identified which stores are at risk, the managers want additional granularity to analyze a store’s performance by duty manager and sales associate.

These questions—and others like them—can easily be answered retrospectively through ad hoc queries, with no need for additional configuration.

Use case 2: from digital experience to customer experience

Your digital teams focus on conversion funnels, creating web experiences that encourage them to become customers. We categorize these user journeys as digital experience, and our customers enjoy best-in-class digital experience monitoring.

But the conversion—from visitor to customer—is just the start of the end-to-end customer experience. What happens after the conversion? You can deliver a perfect digital experience and still disappoint your customer. Whether you’re providing gated access to a video, activating a service, or delivering food to your customer’s doorstep, there are critical business processes that must work together to fulfill the expectations you’ve set.

A pizza chain wants to improve customer satisfaction by tracking delivery times from each shop. They already monitor digital experience using RUM for web and mobile users, but lose visibility into the order fulfillment process once the order is received. Using the order ID as the unique identifier, they can easily connect each conversion to the process milestones that follow, including the handoff to the delivery agent; dashboards summarize each shop’s performance. They also decide to add a follow-up survey after each delivery; since the survey responses are also tagged with the order ID, they can learn how delivery delays impact survey results through ad hoc queries.

Deliveries with survey responses ≤ 3.0

Get started with OneAgent for business events

OneAgent is a great starting point for using business events. Deep payload inspection extracts up to 64KB of business data from in-flight application requests and responses, and configuration is simple and doesn’t require code changes.

The Business Analytics documentation covers configuration steps in detail; here we’ll provide an overview.

Configuration overview

Dynatrace business events loosely follow a subset of the Cloud Events specification for describing event data; if you’re familiar with the specification, you’ll recognize the similarity as you configure your business events. For OneAgent, there are just a few mandatory fields to complete your capture rule and start using business events. Refining the rule with specific data fields and transforming the data via processing rules will add to the value you get from the business data you’re capturing.

Configuration example

Consider the pizza shop example above, with this snippet from the JSON file. We want OneAgent to capture the data as a business event for reporting and analysis.

Step 1: Create a capture rule with trigger conditions, event metadata, and data fields.

• The trigger defines the condition that, when met, will cause OneAgent to capture the business event. We’ll use the request path that contains the value /easypizza/purchase.
• The event metadata fields (provider, type, and category) add context, and can be a fixed value or extracted directly from the event itself. We’ll use easypizza.com as the event source, and com.easypizza.purchase as the event type. The event category field is optional.
• Event data fields are used to extract attribute/value pairs from the event. We’ll capture orderId, quantity, and unitPrice.

Step 2: Add processing rules with matcher DQL, data fields, and processor definition.

Note that there is no total amount field in the sample JSON – just a unit price and quantity. We can use a processing rule to add this new field.

• The matcher uses DQL to filter event data for processing. We’ll use matchesValue(action, “purchase”)
• The data fields will be used in the processor definition; we’ll need unitPrice and quantity to derive totalAmount.
• Processing commands define how to transform the matching data; we’ll use FIELDS_ADD(totalAmount: quantity*unitPrice)

The processing rule will add the totalAmount field to the business event stored in Grail.

Step 3: Define the retention period with matcher DQL and bucket assignment.

Business events are assigned to buckets that have pre-defined retention periods.

• The matcher uses DQL to filter business events for bucket assignment. We’ll use matchesValue(event.provider, "www.easypizza.com"). All business events from this event provider will be assigned to the selected bucket.
• Choose from three bucket options: 35 days, 1 year, or 3 years.

Automatic enrichment

Dynatrace automatically enriches business events with important topology and application information. For OneAgent, these include host, process group, and trace, and provide the context necessary to analyze anomalies and foster collaboration. Intermittent order update failures? Drill down to see the health of the host or process group or open the distributed trace to discover bottlenecks or downstream failures.

The post The value of business events: How IT can increase business agility appeared first on Dynatrace news.

Following the launch of Dynatrace® Grail for Log Management and Analytics, Dynatrace is excited to announce a major update to our Business Analytics solution. Business events powered by our new Grail™ data lakehouse and by other Dynatrace platform technologies ensures the real-time precision that business and IT teams need to make data-driven decisions and improve business outcomes. Business events deliver the industry’s broadest, deepest, and easiest access to your critical business data. Powered by Grail, you can now unify, store, and instantly analyze massive volumes of business data from anywhere, automatically enriched with the IT context needed to unlock precise AI-powered answers and automation.

The need for real-time business observability

Business leaders benefit from in-the-moment business insights, frequently articulating the need for real-time visibility into business data to support agile business decisions. In fact, business teams are often envious of the real-time IT observability that operations teams enjoy. That’s because their existing business intelligence tools often lack the broad context, ease of data access, and real-time insights that are needed to understand and improve customer experience and complex business processes. Business intelligence tools are generally backward-looking, relying on accounting, CRM, ERP, and other business systems to surface anomalies, report results and trends, or mine historical data. To highlight the problem, a recent report from Dimensional Research indicates that 86% of the data used by business analysts is out of date, often by two months or more. While last week’s or last month’s data might be acceptable for historical reporting, it doesn’t help teams respond to dynamic business conditions or react to process anomalies.

The business observability gap

Over the past few years, observability vendors have sought to address some of these analytics challenges, opportunistically introducing real-time reporting for a limited set of business metrics. Using existing APM agent and log monitoring capabilities made it reasonably easy to access certain business metrics and metadata to add to IT dashboards. But these solutions have largely fallen short of business and IT users’ needs. Since they rely on capabilities designed for IT monitoring, they inherit a series of architectural design constraints that limit their usefulness. Top among these are sampling-induced imprecision, limited access to critical business data embedded in application payloads, and lack of context.

Business events, Grail, and OneAgent

Business events, Grail, and OneAgent work together to overcome these limitations, prioritizing business data separately from observability data to deliver precise, real-time business insights with deep visibility into application payloads. This is an entirely new approach to monitoring, investigating, and acting on business metrics. It represents a significant enhancement to our previous Business Analytics capabilities, which emphasized the value and simplicity of business data captured from real user sessions. Business events can be captured by Dynatrace OneAgent, sent from web or mobile real user monitoring RUM, from external sources via public API, or (coming soon) extracted from log files. Business events make it easy for teams to spot business anomalies, long-term trends, and process optimization opportunities, answering even the most challenging business questions with accuracy and clarity. Since business events are automatically enriched with IT context, business and IT teams can collaborate effectively to achieve shared business goals.

Unlike business analytics offerings from other observability vendors, Business Observability easily supports a wide range of important business use cases that demand auditable accuracy. From order fulfillment and bill payments to service activation flows and customer onboarding milestones, business events deliver on the promise of real-time business observability. Business and IT teams can monitor and optimize complex business processes that might span weeks or months, uncovering anomalies before they affect business outcomes. Deep access to in-flight application payloads makes it simple to capture critical business data without code changes. Grail unifies business and IT data at a massive scale for powerful exploratory analytics to answer even unforeseen questions as they arise.

Here’s a quick look at how business events and Grail work together:

1. Business event sources include OneAgent, web and mobile RUM sessions, external business tools via a public API, and (coming soon) log files.
2. Business events are processed through the ingest pipeline using the Dynatrace Query Language (DQL) to transform and enrich the data and set retention periods.
3. Grail unifies business and IT observability data from all sources, leveraging the Smartscape® dependency model to automatically build causational context.
4. Dynatrace Query Language provides exploratory analytics through ad hoc queries without the need to index. Explore interactively with the Log and event explorer, pin your queries to dashboard tiles, or execute queries via the API.

Unlock the value of your business data

With business events from Dynatrace, business and IT teams enhance collaboration and improve agility by leveraging real-time insights and operational analytics for informed business decisions.

Grail, DQL, and business events are available now in select AWS regions and will be available on Azure in early 2024. To learn more or to get started, contact your Dynatrace representative.

What’s next for Business Observability?

There’s much more in the works, using Grail as a game-changing enabling technology. Next to come will be business events from logs, complementing existing OneAgent, RUM, and external sources.

The post Dynatrace enhances Business Analytics with business events powered by Grail appeared first on Dynatrace news.

Digital customers can be unpredictable and easy to lose to the competition–especially if frustrated with digital experience. Leveraging business analytics tools helps ensure their experience is zero-friction–a critical facet of business success. These tools can increase business observability by showing when and why digital experience, and more broadly IT, impacts business outcomes. They can also highlight where you should optimize to improve business outcomes.

Focusing on what matters most–while de-prioritizing less-impactful background issues– enhances agility and frees IT resources for optimization and innovation projects, providing multiple business analytics benefits to an organization.

How do business analytics tools work?

IT teams have traditionally relied on internal metrics to estimate business impact. Two common examples are the number of users affected by a problem or pages with the lowest Apdex scores. And while these may be good starting points, they don’t tell the whole story. More important is how these metrics impact business outcomes: progression through a funnel, conversion rates and value, fulfillment SLOs, and even net promoter scores (NPS). Monitoring business KPIs is necessary to make the connection between IT metrics and business outcomes, and business analytics tools can provide these critical IT business metrics.

Business analytics begins with choosing the business KPIs or tracking goals needed for a specific use case, then determining where you can capture the supporting metrics. In most cases, these tracking goals are already well-understood. Transactional metrics and context, behavioral data, and audience segments may be codified in your website’s data layer, as these are often used by marketing and web analytics solutions. Many business metrics may be captured through backend service call traces as transactions are processed. Business processes and events may be written to system logs or tracked and stored by an ERP solution, exposed via an application programming interface (API).

Break down IT and business silos

While analytics are one challenge, there remains another: silos. Much has been written about breaking down IT silos through cultural shifts and technology alignment. Complementing domain specialists with generalists is one such cultural shift reflective of the dynamic and interconnected nature of modern monitoring platforms. Another is viewing IT services as an iterative product rather than a once-and-done project. On the technology alignment front, the shift has been clearer, from multiple-point solutions to end-to-end observability platforms.

The same principles apply to what is perhaps the last great silo–that between business teams and IT operations. The challenges contributing to this silo include the following:

• The need for a high-level focus. It takes a generalist’s mindset to connect infrastructure, application, user experience, and security insights to business goals. (Imagine asking a highly specialized IoT architect to interpret why sales have suddenly slumped in Florida.)
• The ability to measure outcomes. Viewing IT services as an iterative product implies a focus on measuring outcomes, with ongoing refinements and updates that rely heavily on effective communication informed by a shared view of success metrics.
• Observability that extends into business metrics. Observability solutions must embrace – not just accommodate – business metrics as equal citizens to IT metrics, supporting thresholds, alerting, AI-driven root cause analysis, and automated remediation.

Improve customer experience beyond digital channels

When IT views service quality through the lens of business outcomes, there’s increased emphasis on the benefits of good customer experience. After all, business is increasingly conducted online where customer experience is dominated by digital experience.

For many applications, business KPIs are measured through conversion or goal completion metrics; do users successfully complete the desired task or action? To get there, users must navigate through a series of steps called the conversion funnel or user journey. Each step in the journey is an opportunity–to delight them and coax them closer to the goal or to disappoint them and see them abandon.

Of course, customer experience goes beyond just online transactions. What happens after an order is placed, a reservation is booked, or an application form is submitted? Fulfillment processes can be complex, involving multiple systems and manual steps. Tracking these with business analytics tools adds an end-to-end customer-centric perspective, which is key to ensuring that customer expectations are met. It’s through this outcome-oriented perspective that IT can move beyond digital experience to consider customer experience, aligning with business goals to better meet customer expectations.

Key use cases for business analytics tools

While there are an almost unlimited number of business analytics use cases, most will fit into one of these categories:

Business impact use cases evaluate business outcomes in the context of IT metrics. Through real-time visibility into business KPIs and supporting IT metrics, teams can correct problems before they impact business results. Questions you can answer may include, how does user experience impact conversions? At what point does degraded experience result in a drop in goal achievement?

Conversion analytics use cases complement popular conversion rate optimization (CRO) efforts with important perspectives on performance and usability. By optimizing conversion funnels, teams can reduce abandonments and increase conversion rates by answering questions like, which steps in a user journey result in significant abandonment? What are the common characteristics leading to abandonment?

Release validation use cases help ensure new releases meet their desired business goals. Monitoring these business goals for blue-green deployments, A/B testing, and canary releases provides timely business KPI feedback to app owners and product managers, complementing important technical metrics. This helps to answer questions such as, how does a new feature impact goal achievement? Should the feature be released or rolled back?

Experience reporting use cases focus on SLOs critical to successful business outcomes. These SLOs are often segmented by key audiences for increased granularity and business relevance. For example, are we meeting our response time SLO for each broker, subsidiary, or partner? How do SLO violations impact partner orders?

Business process monitoring use cases extend transaction monitoring across complex processes spanning multiple services. Through business process monitoring, cross-functional teams gain insight into important workflows, uncovering bottlenecks and improving efficiency to understand things like, how long does it take to deliver a custom order? Which steps are the most common bottlenecks?

Business observability is a shared responsibility

As a natural extension of full-stack observability, business analytics is fast becoming critical to the success of agile businesses. As a complement to existing business intelligence solutions, business analytics improves business outcomes through the real-time democratization of business data with IT context. If it’s true that IT exists to support business goals, the two teams must share responsibility as well as accountability for the well-being of critical business processes. And that responsibility demands effective business observability.

Business analytics with Dynatrace

The Dynatrace Software Intelligence Platform provides an automatic and intelligent approach to business analytics metrics and use cases. Business Observability makes it easy to capture business KPIs and connect them to IT metrics, bringing IT performance in context with business goals and outcomes to align IT and business teams through shared context. With application performance, digital experience, and business data all in one platform powered by AI and automation, Dynatrace uniquely delivers business observability to drive better business outcomes.

The post Business analytics tools: Use cases and best practices for IT business analytics appeared first on Dynatrace news.

We introduced Digital Business Analytics in part one as a way for our customers to tie business metrics to application performance and user experience, delivering unified insights into how these metrics influence business milestones and KPIs.

Traditional APM-only solutions care about performance and errors, capturing only the metadata relevant to IT-centric analyses; transaction IDs, error codes, geography, and usernames are examples. For these tools, metadata without performance relevance is considered uninteresting and ignored. Digital Business Analytics extends the Dynatrace Software Intelligence platform to mine the data Dynatrace already observes, creating new opportunities for improving business outcomes.

Dynatrace has delivered automated business impact analysis for years; when a problem occurs, Davis® reports—in real time—how many users are affected and how severe the problem is. But how does this impact business outcomes?

On the other hand, traditional Business Intelligence (BI) tools report when revenue declines or engagement drops but can’t answer if or by how much IT might be contributing.

Disparate tools, different data sources, and diverging interests have made connecting the two all but impossible. As companies intend to prefix Biz- to DevOps, tying together the two perspectives represent a critical step, one which Digital Business Analytics addresses directly.

A quick peek behind the curtains

Dynatrace tracks requests—end-to-end—for each monitored application. Common properties associated with each request have long enabled flexible filtering and analysis; for example, you could evaluate performance and errors associated only with mobile Safari users in South America. Digital Business Analytics leverages this filtering paradigm by allowing you to add existing tags or properties from your application’s payload to the metadata that Dynatrace associates with each user action or session. These properties can be used to measure business objectives and KPIs (such as enrollment or average product revenue) and to define customer segments (such as status level or age group).

These session and user action properties can be defined based on your application’s user interface; important attributes such as loyalty status, search term, product ID, or purchase amount might be extracted from CSS selectors, JavaScript variables, meta tags, or cookies. They can also be derived from server-side request attributes—for example, from a Java method argument.

If you’re already using Dynatrace Real User Monitoring (RUM), you’re ready to go! And if you’re also using full-stack monitoring, you can gain even deeper business insights by leveraging server-side request attributes.

But first, a little preparation

Digital Business Analytics thrives on the right data. Once you have this, Dynatrace makes filtering, multi-dimensional analysis, and creating custom dashboards easy. So your first step is to identify what’s important to the business.

Business objectives can, of course, vary widely; how you measure them will depend on the objective’s characteristics.

• Some are behavioral in nature: engage with a new product feature, increase the average session duration or retention rate, respond to a campaign.
• They may also be defined as independent KPIs: # of orders, average order value, conversion rate.
• When the goal is defined as a user-specific action—purchase a product, request a quote, register for a webinar—it may involve multiple steps. In these cases, it can be helpful to view each step as a milestone, tracking the progress of your users to identify where—and why—drop-off occurs.

Dynatrace makes it easy to capture relevant metadata from each milestone, whether a URL parameter, cookie, or specific tags or field values. Long a RUM best practice, these milestones are configured as conversion goals, leveraging common marketing terminology.

The table below outlines three different business goals: a doctor’s appointment, a data entry task, and a product purchase. While the goals may be different, the paths are quite similar.

Transitions from one step to the next—milestones, or conversions—are important to completing the objective. They can also be influenced by the user’s experience. Defining these as key requests allows for fine-grained analysis of how the experience of segmented user populations impacts their progression through the funnel.

Whether your business objectives are defined as single or multi-step user actions, combining them with relevant audience segmentation will enable valuable BizDevOps analysis and reporting.

Audience segmentation. Gross numbers aren’t enough; it’s important to segment your user populations for finer-grained analysis. You’ll want to identify audience characteristics that will help you understand how one segment may differ from others. Dynatrace already provides technical segmentation criteria, including browser and operating system; you will want to consider adding your own custom segmentation categories. Some examples:

• User status: new, returning, or an associated loyalty level.
• Location: their country and city.
• Demographics: age, gender, or income.
• Technical: production vs. canary release, or A/B testing.

Armed with an understanding of business goals and segmentation themes, you’re ready to experience how easy it is to report on business KPIs and user experience by customer segment.

Four steps to your first Digital Business Analytics dashboard

1. Define session and user action properties

These are used for audience segmentation as well as to extract important business metrics where appropriate. You let Dynatrace know where to find this information by defining session and user action properties. Since these properties are associated with a defined application, you add them by editing the configuration of your application.

If the information is to be extracted from the server instead of the client UI, you should first define the appropriate server-side request attributes, then apply them as user session/action properties.

2. Verify the new properties

Once you’ve defined some properties, you should see them populate on relevant user sessions.

At this point, you can use the new properties as filters, just like other automatically created properties and tags. Through interactive analysis, you can now analyze business objectives for each of your defined audience segments.

3. Define your queries using USQL

Next, you’ll want to become familiar—if you’re not already—with the Dynatrace User Session Query Language (USQL). With USQL, you can run powerful queries, apply flexible user segmentation, and aggregate metrics from captured data, referencing the properties you’ve defined. The SQL-like interface incorporates context-sensitive auto-completion, making even complex queries easy to define. Using USQL, it’s easy to evaluate how business health—measured by KPIs such as revenue, conversions, session count or duration, and feature engagement—is influenced by user experience for key audience segments. And with USQL, you can see the results immediately, easily refining your queries as you go. (For some great USQL examples, see Christian Gustavsson’s blog.)

Let’s consider a simple scenario. We want to track the revenue from users with an experience score of Frustrated. (Your company considers this revenue to be “at risk,” knowing that frustrated users more frequently abandon their shopping carts.) The query below references the income session property that you’ve defined as the business KPI and the Dynatrace-calculated user experience score as the audience segment.

Click the Run query button to test your query:

4. Pin your results.

Once you’ve tested and are satisfied with the query, you can pin the result tile to a dashboard. Here’s the tile I created from the query example above:

At this point, you’d continue to build out your dashboard by creating additional tiles using business-relevant segmentation criteria, and/or additional business KPIs.

Finally, sharing dashboards has never been easier. Our dashboard kiosk mode is ideal for reporting status to managers and executives, as well as for Network Operations Center screens.

In part three we’ll show you how you can automate your business requirements into dashboards.

The post Digital Business Analytics: Let’s get started! appeared first on Dynatrace news.