In part 1 of this blog series, we explored the concept of business observability, its significance, and how real-time visibility aids in making informed decisions. In part 2, we’ll show you how to retrieve business data from a database, analyze that data using dashboards and ad hoc queries, and then use a Davis analyzer to predict metric behavior and detect behavioral anomalies.

Dataflow overview

Dynatrace ActiveGate extensions allow you to extend Dynatrace monitoring to any remote technology that exposes an interface. Dynatrace users typically use extensions to pull technical monitoring data, such as device metrics, into Dynatrace.

However, as we highlighted previously, business data can be significantly more complex than simple metrics. To accommodate this complexity, we created a new Dynatrace extension.

Create an extension to query complex business data

Creating an ActiveGate extension with the Dynatrace extension framework is easy; there’s a tutorial on using the ActiveGate Extension SDK that guides you through making an extension to monitor a demo application bundled with the SDK.

Similar to the tutorial extension, we created an extension that performs queries against databases. Notably, the SQL query is not limited to specific columns or data with specific metric values (int or float). Instead, the data can be of any type, including string, Boolean, timestamp, or duration.

There are three high-level steps to set up the database business-event stream.

1. Create and upload the extension that connects to the database and extracts business data in any form.
2. Configure the extension with the appropriate database credentials, query names, Dynatrace endpoint, and tokens necessary to send the business data to Grail.
3. Once the data is received in Grail, you can explore, manipulate, and analyze the data, utilizing advanced techniques such as filtering, grouping and aggregation, calculations and transformations, time windowing, and much more. Further, you can set alerts based on predefined or auto-adaptive thresholds.

Step-by-step: Set up a custom MySQL database extension

Now we’ll show you step-by-step how to create a custom MySQL database extension for querying and pushing business data to the Dynatrace business events endpoint.

Create and upload the extension

1. Download the extension ZIP file. Don’t rename the file. This is a sample extension that connects to a MySQL database and pushes business events to Dynatrace.
2. Unzip the ZIP file to the plugin deployment directory of your ActiveGate host (found at /opt/dynatrace/remotepluginmodule/plugindeployment/).
3. In the Dynatrace menu, go to Settings > Monitored technologies > Custom extensions and select Upload Extension.
4. Upload the ZIP file.
5. Once uploaded, extract the ZIP file at the same location.
6. Configure the information needed to query business observability data from the target database.
   There are three configuration sections, as shown below in the Dynatrace web UI.

Configuration details

Database configuration

• Endpoint name: Any label to identify this connection. This is used for identification purposes.
• SQL IP/Hostname: The database IP or hostname.
• SQL Username: Username of the user who has permission to login on the SQL server remotely and access the database.
• SQL Password: Password for the username.
• SQL DB: The database name.

Bizevents API and token configuration

• Endpoint to Push Bizevents: Bizevents API that will receive the business data.
  • Replace tenantid with your tenant ID
• Client ID to generate token: Client ID used to generate OAuth token. To generate client-id, refer to our OAuth documentation.
• Client secret to generate token: Client secret for token generation.

Define your SQL Queries

• Queryname 01: Unique name to identify the query to ensure data identification and retention within Dynatrace.
• Query 01: SQL query to retrieve data.
• Interval 01: Frequency in minutes for executing the configured query.
• Add multiple queries (depending on the requirement) with the above config for each query.

Define the retention period with matcher DQL and bucket assignment

Data stored in Grail can be preserved for extended periods, up to 10 years. To achieve this, we’ll create a Grail bucket specifically designed to retain data for a duration of 10 years (3,657 days).

Here is a JSON response from an API that successfully created a bucket capable of storing data for a period of up to 10 years.

After obtaining a bucket with a suitable retention period, it’s time to create a DQL matching rule that effectively filters events and directs them to the appropriate Grail bucket. This ensures that the data is retained for the correct duration while restricting access to users who are authorized for that specific bucket.

Analyze the data in real-time using Dashboards or collaborate with colleagues using Notebooks

In the screen recording provided below, we begin by examining the business data ingested into Grail using a notebook. This initial overview provides a broad perspective of the ingested data. However, real insights emerge when we delve deeper and analyze specific events over time. As you follow along in the video, you’ll notice the ability to determine the day of the week for each transaction and visualize the data in a user-friendly bar chart.

The video below showcases a business dashboard that effectively visualizes important events, including pending withdrawals and deposits from the past hour, transaction amounts throughout the week, transaction queue status from the previous hour, and the overall transaction status.

Enhance data insights with real-time ad hoc queries

While predefined dashboards can offer comprehensive overviews, they don’t always anticipate and meet the needs of business analysts. Dynatrace Query Language (DQL) is a powerful tool for exploring your data and discovering patterns, identifying anomalies and outliers, creating statistical modeling, and more based on data stored in Dynatrace Grail. Now we’ll use a Dynatrace Notebook to execute our DQL queries.

In the below query, we’re specifically searching for pending deposit transactions greater than $8,000 that occurred between 10:00:04 AM and 12:00:00 AM on August 21, 2023. The query for pending deposit transactions within a specific time frame is useful for real-time analysis, issue investigation, performance assessment, impact assessment, and compliance/auditing purposes.

Proactive alerting for accumulating business transactions: Mitigating business impact

To ensure timely action and address potential bottlenecks, we can set up alerts that notify you when pending transactions accumulate within a short period. These alerts serve as early business warnings, allowing you to take necessary measures to prevent disruptions and minimize delays in transaction processing.

In the above recording, we demonstrate an alert specifically designed to notify when there is a significant increase in pending transactions. This alert serves as a valuable tool in maintaining operational efficiency, ensuring business continuity, and delivering optimal customer experiences.

Forecast business data Using a Davis analyzer

In the context of monitoring business-related data such as sales, orders, payments, withdrawals, deposits, and pending transactions, Dynatrace Davis analyzers offer valuable forecast analysis capabilities. Davis analyzers offer a broad range of general-purpose artificial intelligence and machine learning (AI/ML) functionality, such as learning and predicting time series, detecting anomalies, or identifying metric behavior changes within time series.

By utilizing a Davis analyzer, organizations can predict future trends and patterns in their payment and transaction data. This forecast analysis helps businesses anticipate customer behavior, plan for fluctuations in transaction volumes, and optimize their operations accordingly.

For example, by applying forecast analysis to payment data, businesses can identify potential cash flow issues or predict periods of high transaction activity. This type of insight enables you to proactively manage liquidity, ensure sufficient funds are available, and make informed decisions about resource allocation.

Conclusion

By combining proactive alerts and leveraging AI-powered insights, we can effectively manage pending transactions, optimize processes, and ensure smooth operations.

To address the business need for extracting business data from databases, we demonstrated using a custom database extension to bring the data into Dynatrace. This integration allows seamless connectivity to a variety of databases, enabling the real-time retrieval and storage of business data.

By leveraging the powerful combination of business, security, and observability, organizations gain immediate access to their critical business data without any delays or data staleness. The real-time nature of the data extraction ensures that decision-makers have up-to-date information at their fingertips, empowering them to make timely and informed decisions.

Furthermore, we showcased the flexibility and versatility of the Dynatrace platform in exploring and analyzing the extracted data. By seamlessly integrating the data into Notebooks and Dashboards, organizations can gain comprehensive insights into trends, patterns, and key performance indicators relevant to their business. This empowers data analysts and business users to delve deep into the data, uncover valuable insights, and derive actionable intelligence.

Additionally, we demonstrated the power of custom alerts in Dynatrace. By defining specific thresholds for key business KPIs, the platform can proactively monitor data and generate alerts whenever a breach or potential issue is detected. This proactive alerting capability ensures that stakeholders are promptly notified of any anomalies or deviations, enabling them to take immediate corrective actions and mitigate risks. More advanced use cases integrate with automation workflows to automate recovery actions.

Through seamless database connectivity, real-time data retrieval, exploratory capabilities, proactive alerting, and automation, organizations can enhance their overall operational efficiency, customer satisfaction, and business performance. The integration of the Dynatrace observability platform with the custom database extension provides organizations with a solution to extract, analyze, and act upon their at-rest business data, driving success in a rapidly evolving business landscape.

The post Extend business observability: Extract business events from online databases (Part 2) appeared first on Dynatrace news.

Business leaders benefit from in-the-moment business insights. They frequently articulate the need for real-time visibility into business data to support agile business decisions. But existing business intelligence (BI) tools often lack the broad context, ease of data access, and real-time insights needed to understand and improve customer experience and complex business processes.

The key challenges include:

• Business data is often difficult to access, resulting in fragile data pipelines.
• Data is not delivered in real time; it’s often delayed by weeks or longer.
• Business data often lacks IT context, which prevents effective BizOps collaboration.

Dynatrace business events address these systemic problems, delivering real-time business observability to business and IT teams with the precision and context required to support data-driven decisions and improve business outcomes.

Dynatrace business events provide precise, real-time business metrics that support fine-grained business decisions and auditable business reporting. They offer lossless access to hard-to-reach business data embedded in in-flight application payloads, ensuring that valuable information is not missed. Additionally, Dynatrace business events enable organizations to explore and analyze large, long-term data sets without pre-indexing, which allows for flexible and comprehensive data analysis.

Extend business observability to data at rest

In our past blog post about business agility, we looked at a retail sales use case example to investigate potential causes of underperforming store locations. We also looked at a pizza chain example, connecting each customer order to the fulfillment process milestones that followed, including the handoff to the delivery agent.

In both examples, we used Dynatrace OneAgent® deep payload inspection to capture business data in motion. There are also many cases where business data—transactional, inventory, or financial—is at rest or in use, stored in a database. For comprehensive business observability, you need access to this data in real time. This can be accomplished using Dynatrace extensions. Dynatrace extensions can easily query data from various databases and store the results in Grail™, the Dynatrace data lakehouse. Once the data is in Grail, it can be transformed, queried, reported to dashboards, and more.

Business data is more than metrics

Dynatrace Extensions enable the expansion of Dynatrace monitoring to encompass any technology that provides an interface. For instance, the SQL datasource facilitates universal database queries across commonly used databases, subsequently transmitting the results to Dynatrace in the form of metrics or logs.

However, in the real world, business-related data isn’t limited to metrics. Business data should be viewed through a different lens, storing it separately while preserving the unique characteristics that enable business observability:

• Certain business data, such as product names, customer details, sentiments, order dates, payment methods, and more, are not simple metrics. Instead, they can consist of various data types: strings, integers, float, timestamps, and combinations of values.
• Such business observability can’t reside in traditional databases or data warehouses and thus needs to be in a data lakehouse that can unify and contextually analyze observability, security, and business.
• Metrics lack the contextual information to automatically trigger actions such as targeted outreach to impacted customers or automations to remediate process anomalies. Business events, however, capture specific occurrences or actions, allowing organizations to understand triggers, respond promptly, and foster collaboration among teams for improved customer experiences and business outcomes.

To get past the basic metric limitations, we created a custom extension to extract business data from existing databases and store it in Grail. Here’s a peek at the approach:

Business observability

Business observability refers to gaining insights into a business’s operation, performance, and behavior in real time. It involves collecting and analyzing data from various sources within an organization, such as IT systems, applications, customer interactions, and business processes, to gain a comprehensive view of how the business is functioning. An effective business observability solution should make it easy to ingest business data from any source, including databases.

Similar to the concept of observability in IT systems and applications, business observability focuses on capturing data at different layers of the business and making it easily accessible and understandable for analysis and decision-making. It goes beyond traditional business intelligence by providing real-time, granular, and contextual data that enables organizations to identify patterns, trends, anomalies, and correlations across different business dimensions.

Illustrating the value of business observability

Business observability helps you understand and evaluate the performance and effectiveness of systems in achieving their intended business goals. While observing individual requests is essential for performance engineering purposes, taking a business lens perspective provides deeper insights into the actual value delivered by the underlying system.

For example, consider an e-commerce website aiming to maximize sales. By implementing business observability, you can analyze conversion rates, sales patterns, and order fulfillment times. This enables you to identify bottlenecks, optimize user experiences, and make data-driven decisions to improve sales performance.

Similarly, in the case of a ride-sharing app, business observability allows you to monitor metrics like ride acceptance rates, driver and rider satisfaction, and average wait times. By analyzing these business-oriented indicators, you can optimize an app’s algorithms, allocate resources effectively, and enhance the overall experience for both riders and drivers.

For an insurance provider, business observability provides insights into key metrics such as policy sign-ups, claim processing times, and customer satisfaction levels. By closely monitoring these business-focused metrics, you can identify areas for improvement, streamline processes, and deliver better service to your customers.

Business observability not only ensures that systems perform well technically, it also ensures that systems are aligned with their intended business objectives. By gaining visibility into the business value delivered by these systems, you can make informed decisions, optimize performance, and ultimately achieve your business goals more effectively.

The post Extend business observability: Extract business events from online databases (Part 1) appeared first on Dynatrace news.

Digital experience is often considered the most important customer-facing aspect of digital commerce. This is typically the first thing that comes to mind for IT professionals working in the retail industry when evaluating holiday readiness. While digital experience has many facets, transaction speed usually ranks among the most important. Almost two decades ago, a Google experiment showed that fast-loading transactions are more important to customers than content quality—even small increases in transaction delay result in substantially more abandoned sessions. That lesson remains important. (Though the three-second rule for page load time is often misinterpreted).

CEOs of hybrid retailers prioritize e-commerce growth over in-store shopping, investing heavily in their online storefronts. IT teams spend months preparing for the peak traffic they anticipate will arrive with holiday shopping. However, this is a dynamic target; shopping behaviors are increasingly unpredictable, customer expectations continue to rise, and fierce competition makes cultivating loyalty more challenging than ever. These challenges can be summarized by this quote, paraphrased here from Adobe’s 2021 Digital Trends report: “Your customers are digital, unpredictable, and easy to lose.”

From first to lasting impressions

But there’s more to digital experience than speed. Digital experience, measured by fast, frictionless user journeys, paints an incomplete picture, tracking just the beginning of the customer relationship. What happens after the conversion creates a lasting impression with a larger impact on loyalty and your business.

Let’s shift our focus to the backend systems and business processes, the behind-the-scenes heroes of end-to-end customer experience. These retail-business processes must work together efficiently to orchestrate customer satisfaction:

• Inventory management ensures you can anticipate and meet dynamic customer demand.
• Order processing workflow is triggered by customer orders.
• Order fulfillment is the packaging and delivery of orders to customers.

From a customer perspective, the nuances of these business processes are uninteresting as long as they work. Increasingly, however, order fulfillment is a differentiating customer-facing aspect of the end-to-end customer journey, often with digital touchpoints woven into the experience. The fulfillment clock starts ticking the moment a customer purchases your product. Yet fulfillment is often an area over which retailers have little visibility or control.

Customers value real-time visibility into order status and delivery tracking. However, these fulfillment processes are often strained under the pressure of increased online shopping, next-day delivery expectations, and environment-friendly choices. Flexible delivery options, including “buy online, pick up in store” (BOPIS), curbside pickup, self-service lockers, and gig economy delivery require even greater real-time coordination to commit to competitive and narrowing delivery windows. Decentralized last-mile delivery strategies such as micro-fulfillment centers complicate inventory management and order fulfillment oversight.

Technology to the rescue?

Solutions such as inventory management, order management, and delivery optimization can introduce new challenges:

System integration. To effectively leverage multiple systems to manage orders, inventory, and logistics, retailers must invest in often complex integrations. Unsynchronized and siloed data prevents real-time decision-making and business automation.

Multi-channel logistics. Most retailers work with multiple carriers to handle deliveries, resulting in disparate tracking systems. Aggregating tracking information and presenting it to customers in a uniform way can be a challenge.

Real-time updates: Customers expect real-time visibility into fulfillment milestones beyond order confirmation, including packing, shipping, and delivery notifications. Self-service tracking information, preferred by most customers, becomes especially difficult when there are delays or disruptions.

Embracing business observability

Successful retailers benefit from real-time insights into business processes across all milestones. While each system and service provider might adhere to SLOs, the end-to-end health of the process is greater than the sum of its parts. How can you discover optimization opportunities, patterns behind recurring disruptions, or the root cause of an anomaly? The answer lies in the context—connecting business process KPIs to system performance becomes the starting point for real-time business/IT collaboration and automated remediation. The resulting agility supports targeted responses to process disruptions, anomalies, and bottlenecks as they happen, not when daily or weekly reports are produced, not when your call center is inundated, not when your Net Promoter Score (NPS) plummets. To accomplish this transformation, IT teams need to expand their observability scope to include business KPIs.

How Dynatrace can help

Recent platform innovations have made monitoring end-to-end business processes such as order fulfillment easier. Consider these requirements for effective business observability.

• Business data must be accurate to instill the confidence to make business decisions.
• Business data can come from many sources, including OneAgent, RUM, external business systems, and log files.
• Business data must be easy to access without modifying code to reduce the burden on development and maintenance resources.
• Business data must remain granular over long retention periods to support long-running business processes and “needle in the haystack” queries.
• Business data must be unified, regardless of the source or data type.
• Business data must be easily queried to answer unanticipated questions without upfront indexing.

Business events deliver real-time business observability to business and IT teams with the precision and context to support data-driven decisions and improve business outcomes. Business events extract critical business data from your IT systems with lossless precision and can illuminate dark data quickly and easily, wherever that data exists.

Order fulfillment process example

Retail order fulfillment is a good example of business process monitoring, a use case enabled by these innovations. Fulfillment processes vary between retailers, with subprocesses that might introduce branches and loops. It’s a good practice to identify process milestones as a starting point; these should be relatively consistent. For example:

1. Purchase confirmation
2. Order picked from the warehouse
3. Shipping label created
4. Order accepted by the delivery agent
5. Delivery confirmation
6. Survey completed

Once you’ve defined the list of milestones, identify where to capture the data.

• Purchase confirmation: E-commerce platform (via OneAgent)
• Order picked: Warehouse management system (via OneAgent)
• Shipping label created: Warehouse management system (via OneAgent)
• Order scanned by delivery agent: Agent logistics system (via API)
• Delivery confirmation: Agent logistics system (via API)
• Survey: VoC solution (via API or database query)

The Business Flow app, developed using Dynatrace® AppEngine, makes it easy to configure business process milestones for an end-to-end view of process throughput, delays, and anomalies.

The post Customer expectations for retail: Beyond digital experience appeared first on Dynatrace news.

More development teams across enterprises are adopting new mobile UI frameworks, namely SwiftUI, .NET MAUI, and Android’s latest toolkit, Jetpack Compose. While these frameworks use a declarative syntax to simplify the codebase and expedite development lifecycles, they also introduce new challenges in monitoring the user experience of mobile apps. As a front-runner for auto-instrumentation in the observability space, Dynatrace has rolled out support for these technologies to expand its coverage for mobile applications. With mobile RUM, developers and their teams who have migrated their codebase to these new UI frameworks can take advantage of the benefits of the Dynatrace platform, which enable them to quickly identify errors and facilitate their immediate resolution. This allows developers to focus more of their efforts on innovation and delivering the best user experience to their customers.

High consumer expectations for mobile

In today’s digital world, mobile apps have become an essential part of our daily lives. We use mobile apps to communicate, entertain us, conduct business, shop, and much more—on the go, anytime and anywhere. As people typically spend 4.8 hours a day on their mobile devices, it’s natural that they demand a flawless experience. Among all digital channels, mobile has the lowest tolerance for bad experiences. If an app is slow, slightly buggy, or doesn’t fulfill the user’s needs promptly, it gets deleted. In many cases, an alternative app is installed. Overall, 52% of users feel frustrated by their experiences with mobile apps.

New development frameworks from the key players

Apple, Google, and Microsoft, among others, are heavily invested in development tools and frameworks. Backed by strong communities, these tools are continuously enhanced and have redefined how apps are built. Incorporating the latest advancements in technology allows companies to develop and deliver more reliable applications, resulting in smoother user experiences. To ensure consistent progress in app development, it’s crucial to stay updated and integrate these innovations into your development process.

With the introduction of Jetpack Compose and SwiftUI, the development of mobile apps has become more accessible, efficient, and streamlined. These frameworks are based on declarative syntax, which allows developers to build native UI for Android and iOS, respectively, with ease and speed. As a result, modern mobile apps increasingly use Jetpack Compose and SwiftUI to deliver excellent user experiences. According to Google’s recent announcements, approximately 23% of the top 1,000 Android apps are built with Jetpack Compose, and these numbers doubled from last year.

The importance of observability for mobile app development

An essential aspect of improving the user experience on mobile is observability. In order to make informed decisions, business owners, UX designers, and developers require data—specifically, they rely on insights from their data on a large scale, ideally in a single location. Providers of mobile apps must be able to identify issues that users are experiencing quickly so that they can address them. They also need to assess and optimize the performance of both their apps and the back end services that support them. Finally, by gaining complete visibility into user behavior, app providers can improve user journeys and achieve better outcomes for their businesses.
A key aspect of observability is the monitoring agent that a mobile app is instrumented with. This is important because the data collected—including both its breadth, depth, and semantics—play a significant role in determining the value that can be derived from automated or ad hoc analysis.

Mobile observability challenges

Mobile agents are included and shipped with their respective mobile apps, making the selection of the right agent and its configuration crucial. Frequent changes necessitate the publication of new app versions, and successful updates are dependent on users manually updating their devices. Therefore, anything that is not monitored accurately or is monitored incorrectly leads to lengthy cycles of roll-out and adoption until accurate data and answers can be obtained.

Furthermore, companies often require time from mobile app development teams to add dependencies to their agents and configure them. However, the primary goal of these teams is to develop new features and improve existing ones, which means that monitoring user experience is often a secondary concern. Therefore, it’s crucial to minimize the time and complexity required to set up a mobile agent.

Modern development frameworks pose unique challenges when it comes to monitoring. Jetpack Compose and SwiftUI, in particular, allow developers to create UI components using declarative programming. With this approach, a developer describes the desired end result and lets the framework figure out how to update the user interface. This approach simplifies development but introduces new challenges in detecting and relating user interactions with corresponding functions and context. Recognizing on which screen a particular interaction happened and providing human-readable names automatically is challenging. However, such automation is key to ensuring that developers spend less time instrumenting their apps (and that Dynatrace users can more easily make informed decisions based on analyzed data).

Dynatrace extends auto-instrumentation capabilities

The Dynatrace platform offers the best observability coupled with the least required effort for monitoring your mobile channels. Dynatrace boasts industry-leading auto-instrumentation and auto-capture of Real User Monitoring data, regardless of whether it’s for troubleshooting, performance monitoring, or optimizing user experience and business outcomes. With auto-instrumentation, Dynatrace tackles the aforementioned challenges, offering a high level of visibility into the user experience of your mobile app with no manual effort.

Based on feedback from our development community and customers, we have expanded our auto-instrumentation support to include the following technologies:

• Jetpack Compose
• SwiftUI
• .NET MAUI

Dynatrace OneAgent^® now automatically captures user interactions, related web requests, crashes, and other vital app lifecycle information when these technologies are in use (detailed functional scope varies based on the technology in use). Hence, it has never been easier to monitor the user experience of modern mobile apps.

Jetpack Compose support

Starting with the Android Gradle plugin version 8.263+, Dynatrace OneAgent offers auto-instrumentation for Jetpack Compose UI components.

OneAgent creates user actions based on the UI components that trigger these actions and automatically combines the user action data with other monitoring data, such as web request information and crashes. Currently, to get this feature set activated, developers need to manually enable Jetpack Compose auto-instrumentation. For the Android Gradle plugin version 8.271+, the instrumentation will be enabled by default. At the moment we auto-capture various types of user interactions, including

• Clickables
• Toggleables
• Swipeables
• Sliders

For complete details about Jetpack Compose UI components and user action support, go to User action monitoring for Jetpack Compose documentation.

As mentioned, analyzing user experience and identifying the specific elements with which the user interacts can be more challenging with a declarative UI. This is especially true when it comes to providing proper names and context. With the addition of Jetpack Compose support, we have developed a new method for detecting user action names. Dynatrace now evaluates four properties to generate meaningful user action names. The property value that has the most intuitive meaning is used as the name. A special sensor captures semantic information and evaluates the information from the merged semantics tree. This evaluation occurs in the following order:

1. SemanticsPropertyReceiver.dtActionName
2. SemanticsPropertyReceiver.contentDescription
3. SemanticsPropertyReceiver.text
4. Class name

With this approach, companies that use semantic properties appropriately also benefit from a better understanding of user journeys in Dynatrace user sessions.

Another key addition that helps make sense of user interactions is the auto-capture of additional metadata based on the UI component used. Metadata is stored as key-value pairs and is accessible through the Dynatrace web UI. With this additional context—for example, location in code, initial and transition states, interaction types, and more—Dynatrace makes sense of the user journey and the technical components in use. So, not only UX but also developers can retrieve the information they need to optimize user experience.

Based on each UI component, Dynatrace also automatically captures relevant metadata, including:

• The function name to make it easier to find the code behind each interaction
• The user action type, to make it easier to understand the user journey in the User session view
• And a state (from/to) to make it easier to understand the evolving state of each UI component

For full details, see Captured component metadata documentation.

SwiftUI support

Initial support for SwiftUI controls was released with OneAgent for iOS version 8.249+. With OneAgent for iOS version 8.265+, the auto-instrumentation capabilities of SwiftUI were significantly expanded to include:

• New controls and views
• SwiftUI methods(for example, onTapGesture)
• Lifecycle monitoring

SwiftUI is a declarative UI framework. Therefore, its instrumentation and monitoring pose additional challenges.

The Dynatrace SwiftUI instrumentor adds additional code to the project source code (*.swift files) during the build process in order to enable the auto-capture of UI elements. After the build process is complete, all changes to the source code are reverted.

Auto-capture support has been expanded. For details regarding which SwiftUI controls and views are supported, see Instrument SwiftUI controls documentation.

At the moment Dynatrace auto-captures various types of user interactions, including

• Buttons
• Pickers
• Sliders
• NavigationLink
• Tabview

In addition, Dynatrace now supports the instrumentation of some SwiftUI methods. When a supported method is closed, Dynatrace collects the method name, the type of view the method was attached to, and the parent view name.

Finally, Dynatrace is now capable of tracking key lifecycle events for SwiftUI views, including:

• Application start
• onAppear (for all SwiftUI views presented from a NavigationLink or a TabView control)
• onAppear (for revisited SwiftUI views presented from a NavigationLink or a TabView control)

.NET MAUI support

Starting with Dynatrace version 1.265+, our dedicated NuGet package helps auto-instrument your .NET MAUI mobile application.

.NET MAUI is a cross-platform framework for creating native mobile apps and more. It’s a combination of open source code and the evolution of Xamarin.Forms that enables developers to use a single code base to cover as much of app logic and UI layout as possible. The repo has been starred by ~19,000 users and receives regular updates and pull requests.

The Dynatrace NuGet package is used to instrument your mobile applications with OneAgent for Android and iOS. The supported feature set is similar to Xamarin.

Auto-instrumentation support is available for:

• User actions
• Lifecycle events
• Web requests
• Crashes

…plus, a wide range of options for manually instrumenting your app.

Conclusion

Regardless of whether you have a Dynatrace SaaS or a Dynatrace Managed deployment, update OneAgent for Mobile today to leverage the latest observability innovations and benefit from immediate visibility and fast value-creation with the unique end-to-end visibility Dynatrace provides!

The post Dynatrace RUM expands coverage for modern mobile UI frameworks appeared first on Dynatrace news.

In Part 1 of this blog post series, we introduced the concept of security policies in preparation for upcoming enhancements to the Dynatrace platform. Security policies give authorized users access to Dynatrace capabilities, such as Grail and its add-ons, exploratory analytics, and Dynatrace AppEngine.

Modern authorization concepts for the Dynatrace platform

With the introduction of Dynatrace Grail™ and AppEngine, we’re changing how access management is provided on the new platform. The new architecture enables more granularity in permission management and provides the dynamics necessary to serve modern access management use cases.
We’ve replaced the role-based permission system with security policies that provide fine-grained control over service access and introduce new dynamic controls for user authorization.
Dynatrace administrators must now set three permission types to ensure that users benefit from the latest platform capabilities.

• Permission to query data stored in Grail
• Permission to run and access Dynatrace® Apps
• Permission to access supporting services

Managed security policies

Managed security policies are used to assign permissions to users and simplify the configuration of access permissions without the hassle of dealing with all the fine-grained permissions in the Dynatrace IAM permission system. The managed policies cover the essential access scenarios that administrators face and can be used to conveniently set up more complex permissions. When there is a need for more customized access control, administrators can create their own policies.

Dynatrace offers a set of managed policies that cover standard use cases for accessing the new capabilities of the Dynatrace platform: AppEngine, AutomationEngine, and Grail. The managed policies always reflect the latest changes in the Dynatrace platform and its underlying permission system and, therefore, can’t be customized.

Security policies for granting platform access

Administrators can find the policies managed by Dynatrace in the policy overview in Account management and bind them to user groups just like any other policy. These policies are of type Dynatrace managed.

Policies can be assigned to groups via the user group details either on the account level, which includes all tenants in that account, or on the individual environment level.

The following section offers an overview of the managed policies available for Grail™ and AppEngine, which you can use to give your users access to these new capabilities.

Grant users basic access to AppEngine

AppEngine user policy

This policy provides basic access to AppEngine to run apps and functions and access the main supporting services (such as the state service, document service, or document sharing).

ALLOW app-engine:apps:run, app-engine:functions:run;
ALLOW automation:workflows:read, automation:rules:read, automation:calendars:read;
ALLOW document:documents:read, document:documents:write, document:documents:delete, document:environment-shares:read, document:environment-shares:write, document:environment-shares:claim, document:environment-shares:delete, document:direct-shares:read, document:direct-shares:write, document:direct-shares:delete;
ALLOW state:app-states:read, state:app-states:write, state:app-states:delete, state:user-app-states:read, state:user-app-states:write,state:user-app-states:delete, app-settings:objects:read;

AppEngine admin policy

This policy provides users full access to AppEngine, with write access to all services.

ALLOW app-engine:functions:run, app-engine:apps:run, app-engine:apps:install, app-engine:apps:delete;
ALLOW automation:workflows:read, automation:rules:read, automation:calendars:read;
ALLOW document:documents:read, document:documents:write, document:documents:delete, document:environment-shares:read, document:environment-shares:write, document:environment-shares:claim, document:environment-shares:delete, document:direct-shares:read, document:direct-shares:write, document:direct-shares:delete;
ALLOW state:app-states:read, state:app-states:write, state:app-states:delete, state:user-app-states:read, state:user-app-states:write, state:user-app-states:delete, app-settings:objects:read, app-settings:objects:write;
ALLOW settings:objects:read, settings:objects:write, settings:schemas:read where settings:schemaId startsWith "app:";

Give additional access to new platform capabilities

AppEngine developer

Add additional permissions that enable a user to develop and install custom apps

ALLOW app-engine:apps:install, app-engine:apps:delete WHERE shared:app-id startsWith “my”;

AutomationEngine access

Allows users to use the Workflows app and automation capabilities

ALLOW app-engine:apps:run WHERE shared:app-id = "dynatrace.automations";
ALLOW automation:workflows:read, automation:workflows:write, automation:workflows:run, automation:rules:read, automation:rules:write, automation:automations:run, automation:calendars:read, automation:calendars:write;

Grant access to data stored in Grail

Storage Default Monitoring Read

Administrators can use this permission to give users access to the tables and data stored in default buckets (default bucket names starts with default_). This policy will be adjusted automatically as new tables are added to Grail in the future.

Be aware that this policy only covers default buckets. Once custom buckets are added to Grail, administrators need to define additional permissions.

ALLOW storage:buckets:read WHERE storage:bucket-name startsWith “default_”;ALLOW storage:events:read, storage:logs:read, storage:metrics:read, storage:entities:read, storage:bizevents:read, storage:spans:read;

Storage Read per table

Each table includes a policy that combines table and bucket access called Storage <tablename> Read. Administrators can use this policy to grant a user group access to a certain table and the assigned buckets.

ALLOW storage:buckets:read WHERE storage:table-name = “logs”;
ALLOW storage:logs:read;

Storage All System Data Read

The Storage All System Data Read policy grants access to Dynatrace internal data such as auditing events and query execution events. Administrators can add this policy to those users that need access to this type of data.

ALLOW storage:buckets:read;ALLOW storage:system:read;ALLOW storage:events:read, storage:logs:read, storage:metrics:read, storage:entities:read, storage:bizevents:read,storage:spans:read;

Storage All Grail Data Read

This policy grants unfiltered access to all data in Grail.

ALLOW storage:buckets:read;
ALLOW storage:system:read;
ALLOW storage:events:read, storage:logs:read, storage:metrics:read, storage:entities:read, storage:bizevents:read,storage:spans:read;

For full details, see bucket and table permissions for Grail documentation.

Auto-migration for environment admins and default user groups

To make the initial setup more convenient, we auto-assigned AppEngine, AutomationEngine, and Grail policies to existing admins in each Dynatrace environment.

Every user group with permission to change settings receives the respective admin policies for AppEngine and Grail. This means every environment admin is automatically assigned the policies AppEngine Admin, AutomationEngine Access, and Storage All Grail Data Read.

Automate policy assignment via IAM APIs

To automate the above-mentioned policy management or to automatically provision users with the proper privileges, please use our IAM APIs, which can be found at api.dynatrace.com as part of the Account management API.

Use the bindings endpoint in the Dynatrace Account Management API to assign managed policies to user groups.

Pro tip: Notebooks migration script

The newly introduced Dynatrace Notebooks allows you to run custom code using AppEngine. We created a script that helps administrators easily migrate existing platform-access permissions for all users—just add and execute the script to any notebook after providing OAuth client credentials for the account that needs to be migrated with the following OAuth scopes assigned:

• account-idm-read
• iam-policies-management

This script crawls the entire account and identifies the user groups with Viewer access to the environments. It then creates the new policy bindings for AppEngine and Grail based on the existing role-based permissions.

Head over to Dynatrace Community to download the script, get support, or ask any related questions.

What’s next

To learn more about how to manage policies within Dynatrace IAM, see Dynatrace Documentation and Part 1 of this blog post series.

Once you’ve started working with policies, we’d love to hear your feedback: let us know your thoughts or share your ideas about how we should continue to enhance authorization settings in Dynatrace.

The post Tailored access management, Part 2: Onboard users to Grail and AppEngine appeared first on Dynatrace news.

There are many definitions of environmental sustainability, most of which converge on a common theme. Collectively and individually, we have a responsibility to act to protect global ecosystems and support health and wellbeing, now and in the future. Sustainability seeks a balance between human activity and the natural world.

As a result, environmental sustainability is one of the three pillars of environmental, social, and governance (ESG) initiatives. ESG embodies a set of criteria that guides an organization’s strategy, measures the organization’s impact, and informs potential investors across these three pillars. Specifically, the environmental criteria focus on an organization’s utilization of natural resources and the resulting impact on the environment. McKinsey summarizes the importance of this focus. “Every company uses energy and resources; every company affects and is affected by the environment,” the article states. ESG metrics are increasingly important to investors as they evaluate risk. In turn, these metrics are increasingly important to organizations because they measure and disclose their performance.

Balancing act or tipping point?

What motivates us to act? The answer, to varying degrees, is favorable investment potential, regulatory requirements, competitive advantage, cost savings, and moral imperative. And the time to act is now. In fact, in an article for the Wall Street Journal, Gartner analyst Stephen White anticipates that organizations will rapidly adopt performance metrics tied to energy consumption. “Sustainability is increasingly becoming a board-level issue with broad companywide mandates flowing down from chief executives to tech leaders,” White says.

Regulatory mandates are another driver that organizations can’t ignore. As a result, the Securities and Exchange Commission (SEC) in the US and the European Parliament in the EU are adopting stricter reporting rules that will apply to a larger number of companies. Globally, the International Sustainability Standards Board (ISSB), established at the UN Climate Change Conference in Glasgow, has developed requirements for climate-related disclosure. Sustainability reporting informs investors about sustainability risks and helps focus a company’s sustainability actions.

Your IT carbon footprint

Corporate carbon footprint calculations consist of multiple facets, including transportation, waste management, fuel, and electricity consumption. To that end, measuring, understanding, and reporting results are important precursors to intelligent reduction actions.

From an IT perspective, the sustainability focus is on carbon emissions from electricity consumption, specifically related to on-premises, hybrid, and multicloud computing. Some interesting facts:

• Moving a workload to the cloud can reduce its carbon footprint by up to 96%.
• Cloud computing has a greater carbon footprint than the airline industry.
• Average cloud server idle time exceeds 70%.

Yet most organizations don’t have the tools to measure, much less reduce, their IT carbon footprint.

Measuring your carbon footprint: from intention to action

Cloud providers offer tools to measure carbon emissions from the use of their cloud services. But these tools don’t support multicloud environments, and they don’t account for the footprint of on-premises services. More importantly, these tools are fundamentally backward-looking. They lack both the time and dimensional granularity required to derive carbon-emission analytics and optimization insights. Environmental sustainability emphasizes our collective responsibility to take action. For this, we need real-time intelligence and analytics, not just historical reports.

As executive mandates reach technology leadership teams, your organization’s carbon reduction goals will become more tangible. As a result, you’ll need to shift from intent to action, from passive reporting to active reduction initiatives.

Introducing Carbon Impact

The Carbon Impact app from Dynatrace measures and reports the carbon footprint of all Dynatrace-monitored hosts across your entire hybrid and multicloud environment in a single interface. The app translates utilization metrics, including CPU, memory, disk, and network I/O, into their CO2 equivalent (CO2e). Dynatrace engineers developed the app using guidance from the Sustainable Digital Infrastructure Alliance (SDIA), expanding on formulas from Cloud Carbon Footprint.

With Carbon Impact, you can explore the sources of your IT carbon footprint, regardless of where your workloads run. The app automatically identifies opportunities to reduce carbon emissions, capturing the details needed to analyze and make informed decisions. It also generates granular reports and host-level details to provide a focus for carbon reduction initiatives.

The Carbon Impact dashboard: Your IT carbon footprint overview

The Carbon footprint summary reports total CO2e emissions for the selected and preceding timeframes for quick interval-based comparisons.

Optimization summaries report idle instances and under-utilized instances. The app derives thresholds that influence these calculations from Google Compute Engine recommendations, although you have full configuration control to adjust these thresholds to your needs.

The table breaks down emissions by data center, listing your cloud and on-premises instances. All your carbon footprint data is in one place, and the measurement methodology is consistent across all your data centers.

Two charts complete the dashboard. The first shows accumulated carbon footprint and energy consumption over time. As cloud providers decarbonize their data centers, or as you move your workloads to more carbon-efficient locations, the slope of the carbon emission trajectory should change to illustrate the impact. The second chart allows you to compare carbon emissions over time with a business metric of your choice derived from any of your configured business events.

Instances: Where should you focus your attention?

The Instances view details energy and CO2e consumption per host instance. Filters help narrow your focus. For example, you could view underutilized instances in a specific AWS data center or top CO2e emitters within a specific host group. Because Carbon Impact automatically connects to Dynatrace Smartscape® topology modeling, it’s easy to drill into host and process details. Or you can open Notebooks for ad hoc analysis, arming your teams with the insights needed to evaluate opportunities to reduce carbon emissions.

Carbon Impact and Dynatrace AppEngine

Carbon Impact tracks, reports, and helps you reduce the carbon footprint of your cloud and on-premises infrastructure. It also provides data center, host, process, and application details to help you understand where to focus your efforts. Moreover, it provides recommendations to help you get started quickly. Significantly, Carbon Impact supports multiple audiences, including compliance teams as a source for reporting, business teams that need investment guidance, and operations teams that need to act to reduce their carbon footprint.

Carbon Impact is a purpose-built app we created using the new Dynatrace AppEngine. It addresses a well-defined business need, bringing custom analytics to Dynatrace data stored in Grail. Carbon Impact is available now on the Dynatrace Hub. To learn more, please contact your account team.

The post From observability to sustainability: Reduce your IT carbon footprint with Dynatrace Carbon Impact appeared first on Dynatrace news.

Powered by Grail and the Dynatrace AutomationEngine, Site Reliability Guardian helps DevOps platform teams make better-informed release decisions by utilizing all the contextual observability and application security insights of the Dynatrace platform. The app also enables SREs to set and automate service-level objectives (SLOs) for critical services. Site Reliability Guardian further simplifies automation requirements by using entity and ownership information, alerting the right teams when regressions are detected. Dynatrace users can now unlock answer-driven automation to safely and securely release new features to customers while ensuring production SLOs are intact.

Streamline development and delivery processes

Nowadays, digital transformation strategies are executed by almost every organization across all industries. In the context of application development, digital transformation aims to streamline development and delivery processes to ensure quick delivery to the market with guaranteed high quality and security standards.

To achieve this, many organizations are adopting DevOps practices to provide developers with a delivery platform to release their applications and services autonomously and independently. While this empowers teams to frequently deliver new features, the overall business, security, and quality objectives must be maintained. This is where Site Reliability Engineering (SRE) practices are applied. SREs use Service-Level Indicators (SLI) to see the complete picture of service availability, latency, performance, and capacity across various systems, especially revenue-critical systems. Consequently, Service-Level objectives (SLO) are defined to enact countermeasures before the business is impacted.

Given the momentum of DevOps and SRE, digital transformation goals can be achieved when automation enables organizations to apply best practices rapidly and to keep pace with the scale of the organization and applications.

Automation is key for effective collaboration

While DevOps engineers rely on observability and automation to determine whether they can promote a new release into production, SREs look for answers when a new release has a positive or negative impact on application performance, affects user experience, or threatens other SLOs. SREs face ever more challenging situations as environment complexity increases, applications scale up, and organizations grow:

• Growing dependency graphs result in blind spots and the inability to correlate performance metrics with user experience.
• Siloed teams and multiple tools make it difficult to align on a single version of the truth for overall system health.

Other observability solutions don’t provide the required automation capabilities that would allow:

• Automating and speeding up the SLO validation process and quickly reacting to regressions detected in application topology.
• Informing the right people with the answers they need to implement targeted countermeasures.

Automate the validation of key objectives

Dynatrace evolves Cloud Automation release validation (powered by Keptn) into Site Reliability Guardian and natively enriches the Dynatrace platform by automating change impact analysis. For each service, a guardian can automatically validate service-level objectives, “golden signals,” or security vulnerabilities before and after any deployment or configuration changes. It detects regressions and deviations from previously observed behavior, including latency, traffic, error rates, saturation, security coverage, vulnerability risk levels, and memory consumption. Thus, Site Reliability Guardian supports DevOps and SREs in speeding up release delivery and improving release quality.

• DevOps achieve safer and more secure releases by applying a gating mechanism that identifies release issues quickly and prevents poor-quality code from being promoted to production.
• SREs leverage on-demand reliability validation by comparing observability data like service-level objectives against a dedicated release version from the past or as part of progressive delivery.

Get started

Get started with Site Reliability Guardian by selecting the components that you want to guard automatically. Relevant objectives, known as the four golden signals, are suggested by default:

• Latency: the time it takes to service a request
• Traffic: a measure of how much demand is placed on your system
• Errors: the rate of failed requests
• Saturation: a measure of a system’s remaining capacity, emphasizing the resources that are most constrained

The four golden signals are recommended for end-user-facing services because they have proven to be a good starting point for diving into SRE practices. Additionally, you can easily use any previously defined metrics and SLOs from your environments.

Automation workflow runs validations and notifies responsible team members

With a solid set of objectives to validate, automatically generated workflows take care of fully automated validation for you. Whether triggered by a test result or a new release deployment, detected events work as a trigger to check the defined objectives and derive an overall status automatically. Based on the results, further actions are taken, like sending targeted notifications to the service and application owners. This is all available out-of-the-box with the default workflow template provided by Site Reliability Guardian.

In addition, the workflow can be easily extended to any custom demands, for example, integration with tools that support your software product lifecycle. This includes executing tests, running Dynatrace Synthetic checks, or creating tickets. In this way, Dynatrace supports an ecosystem of tool integrations that unlock the full potential of DevOps and SRE with cloud-native delivery orchestration and remediation automation use cases.

What’s next?

Site Reliability Guardian is built for ensuring and maintaining production reliability and resilience. Further enhancements will focus on seamlessly integrating the solution into your existing pipelines to help you shift left and block poor-quality releases before they reach production. Davis will also assist Site Reliability Guardian in recommending relevant objectives and baselines for comparison.

• For an early preview of this new functionality, please request Dynatrace platform preview access, which allows you to install the Site Reliability Guardian via Dynatrace Hub.
• The Dynatrace Site Reliability Guardian will be available within the next 90 days. In the meantime, watch out for upcoming “Observability Clinics” and “Ask me anything” sessions covering the main topics of this blog post. You can either see a list of the next webinars on our website or follow us on LinkedIn to stay up-to-date with upcoming announcements and activities.

The post Automated Change Impact Analysis with Site Reliability Guardian appeared first on Dynatrace news.

Many organizations attempt to combine tools, products, and do-it-yourself solutions with custom code to fulfill custom use cases that are specific to their unique business requirements and technology stacks. Not only are these approaches difficult and costly to maintain, they also lack proper security and scalability.

Especially when seamless end-to-end solutions are needed, it’s necessary to add relevant business context to data to unlock the value of insights that are hidden in the vast amount of observability, security, and business data derived from modern clouds, and overcome the challenges of data that’s locked in organizational silos.

AppEngine empowers organizations to tame cloud complexity, innovate faster and more securely, and ensure consistently better business results, thus delivering answers and driving collaboration across teams.

Multiply the value of your data with Dynatrace

In contrast to other custom app development solutions and extensibility frameworks that embed third-party interfaces and self-hosted custom code via iFrame, Dynatrace AppEngine enables you to create custom apps tailored to your specific needs that are integrated into the Dynatrace platform and offer a simplified user experience.

AppEngine is a core technology within the Dynatrace platform. It empowers Dynatrace customers to use the same tools and technologies as Dynatrace engineers to develop new tailored apps. App developers have the same limitless possibilities for creating customized analytics and integrations in any IT environment, whether in the cloud or on-premises. Apps enable organizations to leverage a goldmine of observability, security, and business data—and add further value by addressing any imaginable use case.

Boundless possibilities

You now have unlimited possibilities to address any use case you can imagine. Consider a simple interactive app that helps convey the business impact of new product enhancements by combining IT Ops metrics with business data—or an app that not only calculates the costs of your cloud usage but also automatically optimizes it by leveraging the power of Davis, our causational AI, and the new Dynatrace AutomationEngine. Or imagine a vulnerability notification generated by Dynatrace Application Security triggering a custom security app that initiates a fully automated action flow, informs your SecOps team, and creates and assigns tickets to the right people with all the relevant contextualized information. These and an unlimited array of other use cases are now possible with AppEngine.

We’re also releasing a variety of apps that we developed using AppEngine. These will be built into the Dynatrace platform and address use cases ranging from automation to security and business analytics, and include apps such as Smartscape Health View, Carbon Impact, and Site Reliability Guardian.

Dynatrace Carbon Impact is a perfect example of combining data stored in Dynatrace with external data and custom logic to explore and visualize the sources of your IT carbon footprint, regardless of where your workloads run. It measures and reports carbon emissions for all Dynatrace-monitored hosts across your entire environment, including hybrid and multicloud, in a single interface and translates utilization metrics, including CPU, memory, disk, and network I/O, into carbon emission metrics. The app automatically identifies opportunities to reduce carbon emissions, capturing the details needed for analysis and making informed decisions.

From code to app to code

Apps built with Dynatrace AppEngine can take different forms. From full-blown apps addressing end-to-end use cases, like the Carbon Impact app example shown above, to small “components” that focus on a particular task, such as:

• offering integration with external line-of-business (LOB) applications
• connecting apps with external databases, normalizing data for immediate platform-wide usage
• extending the Dynatrace AutomationEngine with new custom actions
• adding reusable UI components to extend Dashboards and Notebooks with new chart types

Once you write an app, you can use it everywhere within the Dynatrace platform: within Notebooks, as part of a workflow running on AutomationEngine, or as a standalone app.

Logic to data: Understanding the value of data at its source

Many data-related requirements, such as GDPR, governance, data quality, or auditing, must be addressed when writing software. These requirements are mandatory from both a performance and a cost perspective.

Analyzing petabytes of data only works if the logic is very close to where the data is stored and processed. Grail, the Dynatrace causational data lakehouse, is built for managing and querying enormous amounts of data from the entire hybrid and multicloud environment. Utilizing the capabilities of Grail together with Smartscape topology mapping and Davis AI, AppEngine understands the IT, security, and business value of data in context. This contextual understanding enables you to do any analysis you need and helps to answer questions that can’t be answered with disconnected data stored in different silos.

Addressing enterprise needs: Security, scalability, and manageability

Traditionally, building apps and integrations on different enterprise software platforms was tough and required a lot of effort. By using Dynatrace AppEngine, developers can focus their time on adding value by meeting the urgent needs of the business instead of managing integrations and runtime infrastructure or addressing security needs. The apps run in the Dynatrace environment, thus automatically meeting enterprise requirements. This provides benefits including:

• auto-scaling runtime environments
• automated delivery and operations
• governance by Dynatrace Identity and Access Management (IAM)
• built-in security capabilities such as code shielding, secure runtime environment, audit logs, SSO, secret vaults, and more
• safe connection for secure remote invocations using EdgeConnect

And all of this comes with full app-lifecycle support, including extensive developer experience for creating, building, deploying, and managing apps.

Data value at scale: Become a data-driven enterprise

Dynatrace AppEngine enables teams to achieve better business results by offering a secure, serverless, auto-scaling runtime environment that makes it easy to create custom, compliant, data-driven apps, that address the use cases that are most important to them.​

Everything thinkable is now possible. With apps delivering answers and driving collaboration across IT, development, security, and business teams, you can multiply the value of your data.

Dynatrace AppEngine will be available in Q2 2023. In the meantime, look for upcoming webinars sharing more insights on the app offering of the Dynatrace platform and how to build your own custom apps. Visit our webpage or follow us on LinkedIn to stay current with upcoming announcements and activities.

The post Dynatrace AppEngine: Everything thinkable is now possible appeared first on Dynatrace news.

Today, IT and development teams have myriad responsibilities in managing and working with cloud environments. They need to create new products and maintain existing ones to deliver customer value at speed and scale while managing risk. Therefore, modern observability platforms are becoming a must-have tool for these teams.

Teams can now use an observability platform to unify operations, business, and development data to gain automatic, precise answers about cloud environments and user experience. Further, developers can build more agility into their processes with an intuitive, low-code approach to create custom, compliant, and intelligent data-driven apps and automation.

The ability to create custom apps becomes increasingly important as organizations strive to innovate and grow. They want to develop secure, compliant custom apps and integrations—whether in the cloud or on premises. Indeed, according to one report, more than 50% of U.S. organizations have over 100 custom applications in development and testing today.

A modern observability platform like Dynatrace is uniquely positioned to serve as a foundation that enables teams to build low-code, custom apps that are secure, compliant, and based on precise answers. And with Dynatrace AppEngine, IT pros have the tools to build custom apps that serve their organization’s business needs and fit in their multicloud ecosystems.

The convergence of observability and security data

Organizations now realize that better understanding their applications and cloud environments is the key to assessing their health from both performance and security perspectives. Therefore, leaders are embracing a converged security and observability approach by unifying all observability and security data in a single location or source of truth.

As a result, these leaders can ask different questions of the same data. And because the data is complete and in context, the answers and insights are always accurate, resulting in an increased trust of these data-backed insights. As development and security teams work more closely together, they also recognize the importance of converging observability and security data to deliver answers and insights through secure, compliant, enterprise-grade applications.

Dynatrace AppEngine supports these goals and empowers customers and partners with an easy way to create and share custom apps for their IT, development, security, and business teams. With consistent answers and insights, organizations can solve their toughest business challenges.

Unifying data, retaining full data context

The Dynatrace platform now uniquely consolidates observability, security, and business data while retaining full context and dependency mapping with its data lakehouse, Grail. Additionally, its modern architecture delivers cost-effective storage and compute. As a result, teams benefit from low-cost cloud storage that provides access to all data and doesn’t require data rehydration. This frees teams from manual approaches to connecting siloed data—using imprecise machine-learning and correlation-only techniques—and the high total costs of using multiple DIY-style solutions.

AppEngine uses this data and simplifies intelligent app creation and integrations for teams throughout an organization. It provides automatic scalability, runtime application security, secure connections and integrations across hybrid and multicloud ecosystems, and full lifecycle support, including security and quality certifications. As a result, for the first time, any team in an organization can create intelligent apps powered by causal AI and build integrations for use cases and technologies specific to their unique business requirements and technology stacks.

Delivering precise answers and insights to the whole enterprise with secure apps

AppEngine is a core technology in the Dynatrace platform that enables anyone in IT, business, or security to create and share inherently secure, custom apps that leverage causal AI and data and scale well. It’s a more productive and secure approach for teams to meet requirements without undermining an organization’s security posture. By using these secure apps, teams are empowered with powerful mechanisms for collaboration and insight-sharing across domains.

AppEngine is built for enterprise software development needs

The custom applications that teams develop run in the Dynatrace environment and automatically meet enterprise requirements, such as scalability, availability, performance, scale, governance, and full lifecycle support.

AppEngine provides a runtime environment that automatically scales the environment, ensuring the application is highly available and performing healthily. Additionally, AppEngine handles security and compliance by leveraging the capabilities of the Dynatrace platform, including access control, secrets management, audit logs, and app signing. Further, the EdgeConnect capability allows secure connections to on-premises systems. This ensures users do not damage the enterprise’s security posture with insecure rogue applications. And, lastly, it provides full lifecycle support with a rich developer experience to create, deploy, and manage applications easily.

Endless possibilities with custom apps powered by observability, security, and business data

Dynatrace customers and partners can now easily bring their business logic to data and create applications on the Dynatrace platform. Dynatrace tames the underlying complexity of managing data in a cost-efficient, performant way, ensuring the integrations with an organization’s ecosystem solutions are inherently secure and everything runs and scales in an automated way. This allows teams to focus on quickly creating apps and sharing with others to promote collaboration and data-backed decision-making.

The following apps showcase the power of AppEngine and the diversity of use cases it can address:

• Smartscape Health View. Teams can visualize an application’s vital signs, including its security posture. This app also showcases AppEngine’s ability to unlock actionable insights from data through enrichment, visualization, and analytics.
• Site Reliability Guardian. This enables teams to proactively maintain service-level objectives by automating quality and security gates. This also exemplifies how apps created using AppEngine fuel answer-driven automation to optimize cloud operations.
• Carbon Impact. Organizations can understand and reduce the carbon footprint of hybrid and multicloud ecosystems. This also demonstrates how AppEngine can help teams measure and optimize the key performance indicators that matter most for business executives or regulatory requirements.

This newfound agility in packaging and delivering data-driven answers and insights in a secure, compliant way is highly beneficial for customers seeking to use data for automating insights and actions.

Taking your data insights to the next level

At Perform 2023 in Las Vegas, Dynatrace announced a host of platform enhancements to enable IT pros to move from data in context to insights on which they can execute with confidence. The platform has been updated with a variety of capabilities to enable teams to discover data insights more easily:

• expanded Grail data lakehouse to support new data types, such as metrics and distributed traces, and enable exploratory analytics with causation powered by topology and dependency mapping;
• a new user experience for collaborative and exploratory data analytics using Notebooks;
• Dynatrace AutomationEngine for automating BizDevSecOps workflows driven by answers from causal AI; and
• Dynatrace AppEngine with a low-code tool set and application programming interfaces for building and sharing custom, compliant, data-driven apps.

To learn more about Dynatrace AppEngine and how to create low-code applications for exploratory analytics, check out our on-demand Perform sessions.

The post AppEngine empowers organizations to create custom apps for better data insights appeared first on Dynatrace news.

Grail – the foundation of exploratory analytics

Grail can already store and process log and business events. Now we’re adding Smartscape to DQL and two new data sources to Grail: Metrics on Grail and Traces on Grail.

Introducing Metrics on Grail

Despite their many advantages, modern cloud-native architectures can result in scalability and fragmentation challenges. Ensuring observability across these environments requires access to data at a massive scale. The proliferation of metrics can quickly result in a high cardinality challenge, with each service, host, or Kubernetes pod adding its own unique values to the data set.

Grail solves this scalability issue! Metrics on Grail is architected to manage billions of metrics to cope with cardinalities and unique value combinations of 1 trillion potential permutations for timeframes beyond a year. This is only possible because of our no-index approach and massive parallel processing capabilities, which enable Dynatrace to offer extra-long data retention (15+ months) at full granularity that is cost-efficient and fast.

You no longer need to split, distribute, or pre-aggregate your data. Let Grail do the work, and benefit from instant visualization, precise analytics in context, and spot-on predictive analytics.

Introducing Traces on Grail

A distributed trace follows a transaction on its journey through every service, cloud platform, and host in your environment. Having access to traces that span the full hybrid and multicloud stack enables developers to debug their applications in production and understand dependencies in live environments. For more complex cloud-native architectures, adding more services and applications leads to a massive increase in the volume of collected traces.

With Grail, we address these customer challenges by offering the most powerful and future-proof trace analytics solution on the market, which:

• Handles data volumes of hundreds of terabytes a day
• Retains large data volumes for up to 15 months in a highly cost-efficient way
• Ensures that data retains its context by assembling trace spans into PurePath® distributed traces (including additional code and thread profiling data)
• Returns instant query results in real-time using indexless queries

Smartscape for DQL: Context is king

Bill Gates wrote an essay in 1996 entitled “Content is King” in which he described the future of the internet as a marketplace for content. In DevSecOps, content includes applications and services—in addition to information about the environments where they run and the users who use them. Observability and application security use cases rely on data. However, data on its own, without context, doesn’t reveal all its insights. Whereas Bill Gates’ observation is still valid, for the DevSecOps industry today, a more accurate description is “context is king.”

In a traditional monitoring environment, metrics are aggregated data points that lose their context and granularity when data sets are trimmed to make them more manageable. With Dynatrace and Smartscape for DQL, metrics are a completely different game. Whether it’s metrics, logs, events, traces, or any other data type, Dynatrace not only retains the data context but also enables you to analyze data in its semantic context without boundaries.

These capabilities are powered by Smartscape for DQL, a directional graph representing the real-time topology and dependencies of a data architecture. Smartscape unifies the different data types ingested into Dynatrace and retains the full context of this data to enable holistic and precise data analytics.

With the Dynatrace Query Language (DQL), teams can perform these analyses by asking questions that weren’t possible in the past. There are now boundless possibilities, such as identifying users affected by a service outage in a red-alert scenario or doing forensic research on a recent data breach. With DQL, you can easily combine different data types into a single query.

The power of Smartscape is, of course, not limited to manual queries. The same data model fuels Davis, the causational AI engine at the core of the Dynatrace platform. Dynatrace has used Davis for many years and is leveraging its power for root cause analysis, identifying security risks, and many other use cases. Davis doesn’t rely on machine learning or statistical correlations—the models that power most available AIs and try to correlate data points by timestamp analysis, searching for similarities, or processing manual instrumentations. Alternatively, Davis is causal AI that reflects continuously updated topology and dependencies (powered by Dynatrace Smartscape) and understands the precise relationships and dependencies between isolated signals.

Whereas other AIs must guess, Davis knows and eliminates false positives. With the addition of Dynatrace Grail, which ingests, retains, and maintains data in context, we’re revolutionizing the observability industry and extending Dynatrace further to provide answer-driven analytics and automation for unlimited observability and security use cases.

Exploratory analytics – empowering people and data

While data is considered by some to be the new gold, it’s people that still make the difference. Gaining insights from data stored within Dynatrace has traditionally been limited to people within an organization who have specific expertise and training. This is no longer the case.

With the new Dynatrace user experience, we’re introducing new concepts and changing how people across organizations work and interact with data.

New user experience

How many user interfaces have you used that are defined by the data and data types they show rather than the use cases they support? How often have you spent time decluttering or trying to make sense of the information presented on a dashboard? How often have you wished you could interact with data in the same intuitive way you interact with information on your smartphone, quickly switching between visualizations, easily understanding the context behind a spike in a chart or diagram, or digging deeper to perform ad-hoc analysis?

When we started working on Strato—the new Dynatrace design language that powers our new user experience—we developed a few core principles to address the design challenges stated above:

• Designing software for DevSecOps use cases means handling data—large volumes of data that need to be accessible for in-depth analysis in an easy-to-digest interface. We therefore completely rethought the user experience: the interface is user-centric rather than data-centric. We designed Dynatrace in a way that places the user in the middle, offering a flexible UI—tailored to individual needs and deriving rich insights from different perspectives.
• The interface is simple—whether you’re a first-time user, an occasional user, or an SRE using Dynatrace as your single source of truth, the experience is simple and easy to learn.
• The interface offers infinite possibilities. Users need to be able to work efficiently regardless of how large their environments are. Sharing and collaborating with teams is now easier than ever before.

These principles all align with a single, overarching goal: making data and insights derived from analytics available to a wider audience. To achieve this, we designed the new Dynatrace user experience (UX) to facilitate collaboration with teams across organizations—IT, development, security, and business—and solve everyday problems. We focused on democratizing the user interface, making it less trivial and more accessible, and empowering teams to better understand data signals and make data-backed decisions.

User in context

We already mentioned above that putting data in context is vital for Dynatrace. This is also true from a user experience perspective. With Strato, we add user context to the Dynatrace UI.

Charts are now interactive—data points are clickable. Think of a chart that shows a spike in response time because of a deployment two hours earlier—any user can now hover over this data point and begin interacting with the underlying data, whether it’s a drill-down or just the context of the spike.

Introducing Every component and view in the Dynatrace web UI is interlinked based on user context or “intents”. Similar to what you know from your smartphone when sharing an image on your favorite social media channel, when opening a page within Dynatrace, you can easily pass and share the context of your analyses to any other app on the Dynatrace platform. The selection of available apps that are presented is completely context-sensitive and can even be expanded based on your needs.

Simplifying data analytics with Dashboards and Notebooks

In addition to new concepts revolutionizing the overall Dynatrace user experience, we’re introducing two new apps to the Dynatrace platform: Dynatrace® Dashboards, a complete overhaul of the dashboarding experience, and Dynatrace® Notebooks, for on-demand data exploration. These apps make it easier for more team members to explore, visualize, and collaborate on analytics projects. The following principles guided the development of these new capabilities:

• Offer drastically faster and simpler flows, guided by Strato, the new Dynatrace design system
• Fetch all Dynatrace data from one place and even combine it in a single query with Grail
• Integrate external data easily with Dynatrace functions
• Add flexibility and versatile filtering with variables
• Take context with you as you seamlessly navigate the Dynatrace UI using intents

Dashboards and Notebooks have individual strengths that make them the best choice for solving specific use cases.

Observe data with Dynatrace Dashboards

Dynatrace® Dashboards transforms complex data into easy-to-understand visualizations. Dashboards is your go-to app for quick and clear data overviews, whether you need a status-quo view that can be observed over time or you need to share aggregated views with management or business teams.

Dashboards offers an interactive experience with full support for all capabilities and datatypes offered by Grail, allowing you to query not only metrics, but also logs, events, and even external data. It serves as your starting point for further deep-dive analysis, offering more detailed drill-downs via Notebooks (see below).

The all-new Dashboards app is your answer to live data visualization and observation. From data to insights in seconds, and we’re just at the beginning.

Explore data with Dynatrace Notebooks

Dynatrace Notebooks is your on-demand window into data exploration. It addresses the challenge of finding the right data, cleaning, filtering and transforming the data and finally connecting with other data to understand underlying dependencies. You no longer need the help of a data scientist for such tasks—Notebooks enables every Dynatrace user to perform any type of analysis on data stored in Dynatrace.

Start creating data-driven documents and perform custom analytics. Depending on your use case, Notebooks can persist a status quo and create a snapshot whenever necessary or be “self-updating” using current data to always reflect the actual status. You can easily interact with any query result by “slicing and dicing” the data stored in Grail: advanced filters, refinements, aggregations, and sort orders, are just a click away. It’s even possible to harness the power of Davis by adding predictive forecasts to identify future trends with a simple click.

Suppose you need the limitless power of Grail. In that case, you can easily create and edit DQL queries to filter, join, and transform data any way you need it, or even extend Notebooks with custom logic and external data by adding ad-hoc functions powered by Dynatrace® AppEngine.

Whether you’re analyzing new opportunities, the latest vulnerability post-mortem, or your executive production report, Notebooks does it all and ensures that both your query and results are persisted and ready to be shared with your team members. Empowered by Grail, Notebooks is the Swiss Army knife of the Dynatrace platform—built for collaborative data exploration and analysis.

Summary

With these newly added capabilities, Dynatrace users can now perform any custom query, leveraging Dynatrace Grail AI-fueled graph analytics power. This delivers instant and precise answers for an unlimited array of use cases:

• Business impact: quickly identify impacted users by mapping observability and security findings to your business context and improve customer satisfaction by querying for e-commerce customers who are unable to finalize their check-outs due to a service outage.
• Automation: understand the potential impact of remediation actions on dependent components.
• Security: protect customers and brands by conducting application security forensics to identify, mitigate, and prevent data breaches.
• Business process health: show and analyze the health status of complex processes even if dependencies are non-transactional.
• Optimize: enable more efficient multicloud operations by predicting cloud performance and utilization over time to optimize resource allocation based on user needs.

The Dynatrace analytics platform converges security and observability data, enabling cost-effective end-to-end analytics at a large scale with long retention times, in context with your business, thus multiplying your value from data: every imaginable analysis of data in Dynatrace is now possible!

What’s next?

The new Dynatrace user experience, including the newly designed Dashboards, Notebooks, and Dynatrace Grail support for metrics and Dynatrace® Smartscape for DQL, will be available in Q2 2023. Grail support for Dynatrace PurePath® distributed traces will be open for preview in Q2 2023.

In the meantime, watch out for upcoming Observability Clinics and “Ask me anything” sessions covering the main topics of this blog post. You can either view a list of the next webinars on our website or follow us on LinkedIn to stay up to date with upcoming announcements and activities.

The post Expanded Grail data lakehouse and new Dynatrace user experience unlock boundless analytics appeared first on Dynatrace news.

Cloud environments—including multicloud, hybrid, and cloud-native ecosystems—offer unmatched agility, scalability, and cost-effectiveness, though they also present new challenges and complexities that are impossible to manage manually. As cloud environments become increasingly ubiquitous, so does the need for effective and efficient management.

Dynatrace integrates observability and security monitoring while leveraging causal AI to deliver answers and intelligent automation from data at an enormous scale. The Dynatrace platform provides real-time situational awareness of your cloud ecosystem to give you the confidence that your organization is operating as it should. With answer-driven automation powered by the new Dynatrace AutomationEngine, we’re releasing a solution that further extends our platform, putting answers into action, and empowering organizations to overcome cloud management challenges and unlock the full potential of their cloud strategy while reducing manual effort.

Context is king

Many modern enterprises suffer because their top technical talent is consumed with manual toil. According to the Global CIO Report 2022, 71% of CIOs say the explosion of data produced by cloud-native technology stacks is beyond human abilities to manage manually. Building effective and reliable systems is only possible with automation, which, in the past, proved difficult due to the following issues:

• Complexity of systems: The complexity of modern systems makes it difficult to gather all the necessary information to automate decision-making.
• Lack of standardization: The lack of standardization across tools and technologies makes it difficult to handle automation decisions and actions across different systems.
• Lack of data: In many cases, available data may not be enough to make informed decisions due to data silos, distribution, or negligence.
• Incorrect action on unreliable data: Automation systems without context-rich observability and security data often fail in pinpointing the best remediation actions to take, resulting in unwanted or incorrect reactions, as well as false-positive and false-negative alerts.
• Evolving threats: New security threats and vulnerabilities are constantly emerging. This makes it difficult for pure rule-based automation systems to adapt to new threats and vulnerabilities in real time.
• Primitive safety nets: Having no real-time automation feedback loop results in additional toil and escalation actions.

With the introduction of AutomationEngine, the Dynatrace platform has overcome these automation challenges

Dynatrace enables full observability by supporting open-source tools, as well as Dynatrace OneAgent and other data-ingest methods in enriching the classic observability pillars of metrics, traces, and logs with topology, user behavior, security, metadata, and network data. Dynatrace Davis® AI integrates all this context-rich data to provide answers within complex, multi-level systems and environments.

Extending the platform with Dynatrace AutomationEngine unlocks the power to combine observability, security, and business data with causal AI to easily automate BizDevSecOps workflows at enterprise scale.

Automate what matters

Extending the Dynatrace platform natively with AutomationEngine leads to causal-AI-powered, answer-driven automation. This helps organizations to achieve proactive prevention, efficient operation, faster remediation, strong protection, and business acceleration. Use cases include but are not limited to:

• Targeted notification and collaboration: connect the right teams and equip them with answers in real-time
• Closed-loop remediation: automatically remediate problems and vulnerabilities
• Quality and security gating: block vulnerabilities and regressions before they reach production
• Continuous vulnerability assessment: detect zero-day vulnerabilities
• Predictive automation: take maintenance measures before risking a potential failure
• Progressive delivery orchestration: orchestrate releases based on user and system feedback
• Intelligent infrastructure and application orchestration: auto-scale and resize smarter load distributions
• Preventive security configurations: block malicious IPs based on known attack patterns

Dynatrace Workflows, powered by AutomationEngine, puts automation use cases at your fingertips. Every automated workflow consists of easy configuration, extensive trigger options, out-of-the-box actions and integrations, and unprecedented extensibility by leveraging webhooks, JavaScript code, and application actions powered by AppEngine.

Easy workflow creation and maintenance

Workflow creation and maintenance at scale

Another way to ensure always up-to-date workflows and automation at scale, teams can use the automation-as-code approach available with Dynatrace’ own configuration as code solution named Monaco. By shifting your automation configuration left, Monaco moves automation definition closer to not only your source code, but also your testing, observability, security, and remediation capabilities. Automation-as-code allows for powerful and concise automation across the full software development lifecycle. Another big advantage of automation-as-code is the scale at which automation is enabled. By following a programmatical approach, developers, DevOps and SREs benefit from a single source of truth and streamlined collaboration. This helps to eliminate outdated manual runbooks, which are inevitably fail.

Proactive automation

You can configure a variety of event triggers  to initiate custom workflows based on your needs:

• On-demand trigger: manual execution of an otherwise tedious chain of tasks, for example, audit automation
• API-based trigger: direct calls embedded into existing toolchains, for example, pipeline integration
• Schedule-based trigger: scheduled execution of recurring tasks, for example, reporting
• Problem, security, or Davis event-based trigger: Davis causal-AI triggered actions that reduce mean time to repair (MTTR), for example, problem remediation
• Dynatrace Query Language (DQL) event-filter trigger: results of event filters applied to the observability, security, and business data in Dynatrace to trigger actions based on certain event types, for example, deployments of a specific service

Boundless automation

Every workflow consists of several tasks that can be executed or skipped based on the conditions of previous tasks, retried based on error logic, or looped to iterate over lists. To enable this, workflows are aware of variables and context and can execute DQL queries in real-time. Besides executing actions within the Dynatrace platform itself, Dynatrace AutomationEngine allows the safe triggering of actions to both cloud services and—by leveraging Dynatrace EdgeConnect—on-premises services. Dynatrace provides an ever-growing ecosystem of deep, out-of-the-box, third-party integrations that connect automation with collaboration, ITSM, AIOps, DevOps, and security tools.

For truly limitless automation capabilities, Dynatrace Workflows can issue HTTP requests to any API and execute custom Javascript code. By leveraging Dynatrace AppEngine, any application can publish universally-available actions for all workflows, thereby extending custom logic to automated workflows. This automation context, embedded into every app, unlocks even more automation use cases.

Dynatrace AutomationEngine leverages platform-wide event triggers and timezone-aware schedules with dynamic recurrence rules. It allows custom flow logic and parallel task execution with automatic retry, and can be configured-as-code, all to accommodate the complexities of global enterprises.

What’s next?

Interested and want to learn more? The new Dynatrace user experience, including the newly developed AutomationEngine, will be available within 90 days. In the meantime, watch out for upcoming Dynatrace Observability Clinics and live “Ask me anything” sessions covering the main topics of this blog post. You can see a list of the next webinars on our website or follow us on LinkedIn to stay up-to-date with upcoming announcements and activities.

The AutomationEngine actions and integrations ecosystem are being continuously developed and extended. If you’re looking for a certain third-party integration for automation that is currently missing, let us know by posting to the Dynatrace Community.

The post AutomationEngine tames cloud complexity with answer-driven automation appeared first on Dynatrace news.