As cloud complexity increases and security concerns mount, organizations need log analytics to discover and investigate issues and gain critical business intelligence. But exploring the breadth of log analytics scenarios with most log vendors often results in unexpectedly high monthly log bills and aggressive year-over-year costs. To give organizations the freedom to explore log analytics without barriers due to cost concerns, Dynatrace is proud to announce a new Dynatrace Platform Subscription (DPS) pricing model option called Retain with Included Queries.

With this new DPS pricing model option, customers can retain data at a fixed low cost with no additional cost to query for up to 35 days. This model provides a predictable way for customers to manage and analyze logs, drive log management tool consolidation, and reduce costs while gaining maximum value from their log data.

Based on customer feedback, we’re offering the Retain with Included Queries pricing model as an alternative to our existing usage-based plan. Both plans offer the same low ingest price. However, the new all-access plan combines retention and queries into one low price to simplify scoping and budgeting.

Retain with Included-Query pricing simplifies forecasting and annual usage calculation costs. Customers who choose this pricing option get:

• Retention cost: $0.02 per GiB per day
• No cost to query for up to 35 days
• Ingest cost: $0.20 per GiB ingested (no change)

With this approach, the whole team can leverage the power of Grail queries and dashboards without worrying about limiting query usage. Customers can configure the Retain with Included Queries option with retention periods ranging from 10 to 35 days. Customers requiring longer retention periods should opt for our existing usage-based pricing, which supports retention for up to 10 years.

Queries are included

• Predictable pricing: If you know the number of logs you ingest daily, then you’ll know roughly your total annual cost upfront, providing peace of mind and less managerial overhead.
• Simple scoping: Remove the complexity associated with predicting query search volumes. Realize cost savings immediately for high-query usage scenarios. Get started quickly!
• No cost management required: Once your configured retention period ends (a maximum of 35 days), logs are automatically deleted. No oversight is needed over query usage.

Usage-based pricing is still an option

Over time, our existing usage-based pricing is the more cost-optimized option, as you only pay for the queries your users execute, and you benefit from the competitive $0.0007 per GiB per day to retain logs for up to 10 years. Queries are charged at $0.0035 per GiB scanned. Usage-based pricing is ideal for organizations with longer retention requirements and known query patterns. This pricing flexibility allows customers to optimize their log analysis expenses by paying only for what they use.

Cost-efficient:

• Lowest upfront cost
• Charges are strictly based on query execution

Scalability:

• Ideal for businesses with varying query demands
• Adapt dynamically to usage patterns

Retention:

• Supports log storage from 1 day to 10 years
• Optimal for longer-term log analytics needs

Guidance on using both plans

The Retain with Included Queries pricing option is a great way to get started while you learn about your query usage. Dynatrace includes a ready-made cost dashboard that provides insights into query usage and DQL best practices. Once you develop best practices and are confident with your consumption patterns, you can switch to usage-based pricing to maximize the value of your DPS investment.

Innovations on the horizon*

We’re very excited about our new Retain with Included Queries pricing, but we expect to deliver more updates. This pricing model is part of our plan to introduce new features that help customers align the right pricing strategies to their use cases. With these features, customers can easily see, manage, and choose how to align the Retain with Included Queries pricing with the usage-based pricing model.

Customers will soon be able to mix and match log pricing options on a per-bucket basis and provide users with access to both models simultaneously. This flexibility will allow customers to optimize the pricing selection based on the anticipated use case associated with each bucket, yielding even greater savings and value.

Retain with Included Queries: Start here

With the Retain with Included Queries pricing model, Dynatrace now offers a more cost-effective way to get started with log analytics. Drive efficiency and get more value out your logs with this predictable pricing model while you’re building your log analytics practices.

* Disclaimer: This publication may include references to the planned testing, release, and/or availability of Dynatrace products and services. The information provided in this publication is for informational purposes only; its contents are subject to change without notice, and it should not be relied on in making a purchasing decision. The information is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for products remains at the sole discretion of Dynatrace

The post Predictable costs for Log Management & Analytics with new simplified licensing plan appeared first on Dynatrace news.

Understanding that the first mile of getting data in can often be the hardest, Dynatrace continues to invest in log ingest, offering a range of out-of-the-box solutions within the Dynatrace Platform and apps. We’re excited to announce several log management innovations, including native support for Syslog messages, seamless integration with AWS Firehose, an agentless approach using Kubernetes Platform Monitoring solution with Fluent Bit, a new out-of-the-box ingest dashboard, and OpenPipeline ingest improvements.

These developments open up new use cases, allowing Dynatrace customers to harness even more data for comprehensive AI-driven insights, faster troubleshooting, and improved operational efficiency.

Let’s delve deeper into how these capabilities can transform your observability strategy, starting with our new syslog support.

Native support for Syslog messages

Syslog messages are generated by default in Linux and Unix operating systems, security devices, network devices, and applications such as web servers and databases. Native support for syslog messages extends our infrastructure log support to all Linux/Unix systems and network devices. The more data ingestion channels you provide to the Dynatrace Davis® AI engine, the more comprehensive Dynatrace automated root cause analysis becomes.

Customers can also proactively address issues using Davis AI’s predictive analytics capabilities by analyzing network log content, such as retries or anomalies in performance response times.

Dynatrace natively supports Syslog using ActiveGate (preferred method) or the OpenTelemetry (OTel) collector. Many syslog producers lack authentication and have varying security capabilities, such as TLS encryption. Dynatrace ActiveGate addresses these issues by enforcing configurable security settings and ensuring data uniformity. It also enhances syslog messages with additional context and optimizes network traffic, improving overall system resilience and security. This enhancement lays the foundation for the broader, more integrated observability that Dynatrace continues to expand with each new feature.

Customers have had a positive response to our native syslog implementation, noting its easy setup and efficiency. A $20 billion Germany-based financial services company told us they found the process of pushing Syslog messages to Dynatrace natively to be seamless. Another customer based in Germany, a $23 billion medical technology company, told us they appreciate the value of using a native channel to push syslog messages from network devices directly to Dynatrace, bypassing the need for FluentD or a standalone OpenTelemetry collector.

This streamlined approach enhances both usability and integration, making syslog management simpler and more effective.

Seamless integration with AWS Firehose

Dynatrace is also enhancing our observability logs offerings for AWS services for cloud-native applications. By integrating AWS Firehose into the Dynatrace platform, you can address high-impact issues quickly through real-time, high-frequency log analytics.

This integration with AWS Firehose simplifies observability by removing intermediary components, which allows seamless log capture and analysis directly in the Grail data lakehouse. Logs are immediately available for troubleshooting, security investigations, and auditing, becoming integral to the platform alongside traces and metrics.

Dynatrace supports scalable data ingestion, ensuring your observability infrastructure grows with your cloud environment. The setup is straightforward, using API keys, CloudFormation templates, or the AWS web console.

Dynatrace also provides contextual insights by linking logs to problems detected by Davis AI, enabling quick access to relevant details. The platform also offers proactive analysis through Notebooks for visualizing log data and exploring error rates. Dynatrace support for AWS Firehose includes Lambda logs, Amazon virtual private cloud (VPC) flow logs, S3 logs, and CloudWatch. This seamless integration not only enhances AWS observability but also ties into the greater context of how Dynatrace unifies cloud-native observability across multiple platforms.

Customers have responded enthusiastically to our AWS Firehose implementation. Our approach provides seamless cloud log integrations you can configure directly in the AWS console or through provided CloudFormation templates. This setup eliminates the need for additional middle-layer components, making it straightforward and efficient.

A key advantage of this integration is its high throughput aligned with Grail, ensuring optimal performance. What’s more, logs ingested using AWS Firehose are enriched with cloud context, enabling in-context analysis within the platform. This capability enhances the overall observability and insights that customers can gain from their cloud environments.

Kubernetes Platform Monitoring using Fluent Bit for cloud-native environments

One of the log management innovations we’re excited to share is the new Kubernetes platform monitoring solution with Fluent Bit, offering a cloud-native, API-based deployment model. With this innovation, Dynatrace makes it easier for teams to stream logs from Kubernetes environments into Dynatrace through a more lightweight and streamlined setup, accelerating time to value. Customers get advanced health analytics out of the box and automated root cause analysis by Davis® AI when they ingest Kubernetes workloads, traces, logs, and metrics into the Dynatrace Grail data lakehouse.

For organizations who already use Fluent Bit as part of their tech stack to configure pipelines and enrich log data, this modern approach enables teams to gain answers in context based on logs, powered by the Dynatrace platform’s automation and problem detection. With all data in one place and in context, the new Kubernetes platform monitoring solution provides easy filters by namespace, cluster, workloads, nodes, services, pods, and containers. Integrating with Fluent Bit for Kubernetes log ingestion is important for ensuring teams are capturing critical data for troubleshooting and issue remediation.

Dynatrace enhances Fluent Bit’s log management by integrating observability signals like traces, events, and metrics, providing a complete view of cloud-native application performance. It automates log analysis, eliminates manual correlation, and offers broader visibility through ready-made dashboards and health checks. Log configuration is simplified, while advanced analytics powered by Davis AI bubbles up critical health signals, and provides automated root cause analysis, predictive AI for remediation, generative AI for query writing, performance baselining, and anomaly detection. This innovation ties into the broader effort to simplify and enhance log management across diverse environments, further integrating Kubernetes observability into the Dynatrace ecosystem.

Out-of-the-box logs ingest dashboard

Coming soon is an out-of-the-box logs ingest dashboard that enables you to easily manage your ingest channels.

The dashboard tracks a histogram chart of total storage utilized with logs daily. It also tracks the top five log producers by entity. You can see in a table retention periods by the number of logs and storage they consumed.

The dashboard also breaks down log volume by Grail buckets, showing you what buckets consume the most storage. Grail buckets can help enterprises categorize their logs by retention periods, types of logs such as audit logs, or by organizations that utilize the logs. Think of it like individual bookshelves in a library, where each shelf is dedicated to a specific genre or topic. Just as books are organized on these shelves for easy access and retrieval, data log records are stored in buckets based on their type or purpose, allowing for efficient management and quick querying. This organization ensures that when you need specific information, you can go directly to the relevant shelf or bucket, saving time and effort in finding what you need.

This final piece of the puzzle ensures that your log data is not only easily ingested but also effectively managed, tying together the full spectrum of observability enhancements in the Dynatrace platform. The logs ingest dashboard is currently in tech preview, with GA expected soon. Please follow up with your account team to get early access.

OpenPipeline ingest improvements save money and improve query performance

Lastly, the Dynatrace feature OpenPipeline unifies how we ingest, transform, enrich, and process all observability signals, including logs. This is a significant upgrade to our log processing pipeline capabilities. We now support the following for log ingest:

• Log content up to 512K each
• Log metric counts up to 1000
• Log attributes up to 2.5KB
• Number of log attributes up to 250
• Logs ingestion API payload up to 10MB
• Converting logs into Business Events saves money
• Masking sensitive data
• Setting security context
• Extracting metrics from logs and business events saves money
• Parsing JSON before storing logs in Grail for faster analytics

Customers can save money by converting logs into metrics and business events, which are ingested into predefined buckets, making queries faster without needing to span the entire Grail dataset. This approach also enhances security by allowing customizable security contexts for each log and masking sensitive data before ingestion, while simplifying analytics by pre-parsing JSON. With OpenPipeline, customers can add, remove, rename fields, parse, and mask all incoming logs.

Building on these ingest improvements, these innovations further enhance data analytics and precision by introducing advanced features like OpenPipeline.

Dynatrace log management innovations expand data analytics and precision

Dynatrace continues to lead the way in log management and observability with its latest advancements. By introducing support for syslog messages, AWS Firehose integration, the agentless Fluent Bit setup for Kubernetes environments, a powerful out-of-the-box ingest dashboard and our new OpenPipeline, our customers can achieve deeper insights, faster troubleshooting, and more efficient operations across their hybrid cloud ecosystems. These innovations not only expand the breadth of data available for analysis within the Dynatrace platform but also enhance the precision and effectiveness of our AI-driven capabilities, which results in more actionable, automatable outcomes.

Each of these developments interconnects to create a more comprehensive and powerful observability solution, designed to meet the evolving needs of our customers. As we continue to refine and expand our offerings, our commitment remains focused on empowering organizations to proactively manage their infrastructure and applications with unparalleled clarity and confidence.

We encourage our customers to explore these new capabilities and see how they can further optimize their observability practices.

Try out these new Dynatrace log management innovations

If you’re a customer, go to Dynatrace Playground to check out the new capabilities. If you’re looking into Dynatrace, check out our free trial.

To learn more about these technologies, see details in the following blogs:

• AWS Firehose
• Syslog with AG
• Syslog with OpenTelemetry
• FluentBit logs from Kubernetes
• OpenPipeline

The post From syslog to AWS Firehose: Dynatrace log management innovations that enhance observability appeared first on Dynatrace news.

After organizations struggled to recover from the CrowdStrike update outage on their Windows hosts in July, teams are now looking for ways to bolster the resiliency of their software update and continuous delivery practices.

The widespread impact of the CrowdStrike issue demonstrated how critical it is to avoid outages and failures during software deployments to maintain user trust and satisfaction. Traditional deployment techniques that roll out updates or patches directly into full production can present significant risks and lead to potential downtime.

Modern deployment techniques using progressive delivery—such as rolling, blue/green, canary, and feature flagging—offer a more controlled and gradual approach to releasing software. By implementing these strategies, organizations can minimize the impact of potential failures and ensure a smoother transition for users.

Eliminating the potential for outages may not be possible in all situations. That said, a unified observability and security platform such as Dynatrace can enhance modern deployment practices and enable teams to proactively monitor performance, validate changes, and best protect their downstream customers and end users from disruptions.

Bolster continuous delivery with software development lifecycle integrations

Software testing is critical, yet issues can still make it into production that negatively impact the customer experience. Dynatrace strives to protect its own customers through extensive automated test validation, complemented by staged rollouts to customer segments, which include quick feedback loops to determine whether the software is safe to continue deployments. Customers can adopt many modern and safe continuous delivery practices with Dynatrace in combination with their existing software pipelines.

By embedding Dynatrace observability into their own CI/CD pipeline, customers can prevent issues from ever leaving pre-production environments. This integration enhances the pre-release phase and plays a crucial role in the quick feedback cycle after deployment, allowing teams to identify issues immediately. Furthermore, the Dynatrace Site Reliability Guardian ensures service-level objectives (SLOs) and further validation thresholds are not violated both before and after deployment.

Progressive delivery methods to release higher-quality software, faster

The following common progressive delivery methods enable organizations to release software in a more controlled manner. 

Rolling deployments

In a rolling deployment, software is gradually rolled out, replacing previous software in a serial one-by-one fashion or in batch sets. Dynatrace can monitor production environments for performance degradations and outage events that may cause customers to lose access. The Dynatrace AutomationEngine, in conjunction with the existing deployment platforms, enables immediate and automatic software rollbacks to previous versions, limiting the blast radius for customers.

Blue/green deployments

A blue/green deployment strategy involves selecting a “blue” group to run the new software while the “green” group continues to run the previous version. The Davis AI engine immediately recognizes any anomalies, performance issues, or outages between the two groups. In case there is a degradation of service, Dynatrace picks it up and, embedded by the pipeline, can ensure all customers are rooted back to the proven, stable deployment.

Canary deployments

The Canary deployment strategy releases software to customers in incremental phases, gradually increasing the production load on the new deployment. Dynatrace enables customers to set quality measures or SLO targets for performance, outages, or other usage metrics to mitigate risk. By integrating AutomationEngine into the pipeline, customers can safely increase or decrease the load on canary deployments.

Feature flagging

Feature flagging is another popular technique used in progressive delivery that allows teams to roll out new features incrementally and with minimal risk. It supports A/B testing, canary releases, and quick rollbacks, ensuring smoother transitions and more controlled feature releases. The result is testing features in production with specific user segments, gathering feedback and making data-driven decisions on a broader rollout. Embedding feature flags in the codebase can make it easier to control the visibility and behavior of features in real time without redeploying or disrupting an application. This leads to enhanced agility, improves quality, and accelerates time-to-market, all while maintaining a seamless user experience.

As organizations implement progressive delivery strategies to enhance their release processes, integrating automated validation tools such as the Dynatrace Site Reliability Guardian (SRG) becomes essential for further optimizing these practices and ensuring robust performance monitoring

Site Reliability Guardian and AutomationEngine bolster continuous delivery tools and technologies

Dynatrace Site Reliability Guardian (SRG) integrates with continuous deployment practices and platforms to provide the following capabilities:

1. Automated change impact analysis: SRG automates analyzing the impact of changes on service availability, performance, and capacity objectives across various systems before a deployment goes live or during any of the deployment strategies.
2. Integration with CI/CD pipelines: Teams can integrate SRG into existing delivery pipelines including Jenkins, Github, GitLab, AWS, or Azure pipelines. This integration enables teams to validate releases automatically as part of their software development lifecycle before they are released to customers.
3. Workflow automation: AutomationEngine and workflows automate the execution of guardians or problem remediation. This can be tied to specific events such as deployments or configuration changes, allowing for automated validation and response to changes.
4. Service-level objectives (SLOs): SLOs enable site reliability engineers (SREs) to manage and track thresholds for critical services. The Davis AI engine proactively monitors these SLOs for degradations and acts before an SLO violation happens.
5. Automated release validation: The platform supports automated release validation for security and quality gates to ensure that only high-quality code progresses through the delivery pipeline. This integration reduces the risk of deploying faulty code to production.

Embed Dynatrace into your release process and gain even more control

The Dynatrace platform is an essential tool for customers deploying software. It provides critical data that enables rapid, fact-based decisions about continuing or rolling back deployments. By offering real-time performance metrics and insights into application health, the Dynatrace platform empowers teams to assess the impact of changes quickly. This capability minimizes downtime and ensures potential issues are addressed before affecting end users, allowing organizations to navigate software deployment complexities and enhance reliability confidently.

Enhance continuous delivery quality and manage the risk of another CrowdStrike update outage

Incidents like the CrowdStrike update outage illustrate the importance of adopting modern progressive delivery strategies to enhance software reliability and customer satisfaction. By adopting approaches like rolling, blue/green, and canary deployments, teams can mitigate risks associated with large-scale releases and ensure a smoother user experience. Integrating Dynatrace into these processes provides invaluable insights and automated monitoring capabilities, allowing DevOps teams to detect issues early and respond swiftly.

As organizations continue to navigate the complexities of software delivery, prioritizing modern deployment techniques and leveraging robust monitoring solutions will be key to avoiding outages and failures. Embracing these practices will ultimately lead to more resilient applications, happier users, and a stronger competitive edge in the market.

Contact us to learn how you can bolster the resiliency of your progressive delivery techniques with AI-driven automation that can help you avoid the effects of an outage like CrowdStrike.

The post CrowdStrike update outage: Managing continuous delivery and deployment risk with Dynatrace appeared first on Dynatrace news.

Logs provide answers, but monitoring is a challenge

Manual tagging is error-prone

Making sure your required logs are monitored is a task distributed between the data owner and the monitoring administrator. Often, it comes down to provisioning YAML configuration files and listing the files or log sources required for monitoring. This manual, error-prone approach can lead to monitoring gaps, which become critical when a host or service has an outage or incident.

Finding the right logs is cumbersome

Even if your logs are monitored, you need to make sense of the vast data volume. As the scale and complexity of your tech stack grows, you might need to navigate the maze of hosts or Kubernetes clusters, apps, and microservices and understand the relevance and risks associated with logs originating from these entities. Challenges compound: Manual tagging of log sources has long been difficult regarding monitoring coverage. And you can’t assume the tagging is 100% correct to pinpoint the correct logs.

In the past, more work was needed to understand the context of log data. What about correlated trace data, host metrics, real-time vulnerability scanning results, or log messages captured just before an incident occurs? This context is vital to understanding issues.

Dynatrace automatically puts logs into context

Dynatrace Log Management and Analytics directly addresses these challenges. First, OneAgent takes care of log autodiscovery. Once logs are selected for monitoring, OneAgent enriches log data with the topological context you need. For example, OneAgent helps you monitor the logs from a Kubernetes environment with automatic enrichment that identifies the right cluster, namespace, container, and pod ID.

Once logs are stored in Dynatrace Grail™, our purpose-built data lakehouse for observability data, the logs are automatically shown in the right context. Finding answers begins with opening the right app for your use case.

You can easily pivot between a hot Kubernetes cluster and the log file related to the issue in 2-3 clicks in these Dynatrace® Apps: Infrastructure & Observability (I&O), Databases, Clouds, and Kubernetes.

Open a host, cluster, cloud service, or database view in one of these apps, and you immediately see logs alongside other relevant metrics, processes, SLOs, events, vulnerabilities, and data offered by the app.

By eliminating slow and manual correlation, lack of context, and getting visibility into the surrounding data, you reduce the risk of prolonged outages, mean time to repair, and tool sprawl.

Get quicker answers

Let’s look at how logs in context can make your teams more effective.

Log histograms: Insight into log volumes and patterns

Open one of these Dynatrace Apps and select Logs for any listed entity (host, Kubernetes workload, cloud service, or database instance):

• Infrastructure & Operations
• Kubernetes
• Databases
• Clouds

You’ll see a histogram chart of log data with various severity levels (such as Error, Info, or Warning) relevant to the selected Dynatrace entity, giving you a clear understanding of log patterns and volumes over time. Is there a sudden spike in errors? A sudden drop in received log data? Depending on which app is in use, one glance at a histogram provides invaluable insight into managing clouds, databases, Kubernetes environments, and infrastructure.

Log analytics simplified: Deeper insights, no DQL required

Your team will immediately notice the streamlined log analysis capabilities below the histogram. Jump directly into log insights by selecting a recommended query, for example, to see the errors related to a problem detected by Davis® AI during the selected timeframe. Furthermore, your team can easily access all error logs within the specified timeframe displayed on the histogram or view all logs within that timeframe, all without writing any queries from scratch.

Surrounding logs display: Effortlessly navigate log context

You can see the result after opening a recommended query without leaving an app’s context. Upon expanding a single log entry, all relevant context provided by OneAgent during the ingestion process is displayed, making it easy to expand your analysis to the infrastructure or entity related to the error logs. For a single log record found, you can easily see the surrounding logs.

Look at this example of an online store payment service generating errors. The application owner found error logs related to unsupported credit cards. Select Surrounding logs to view the log messages for the whole transaction, based on the trace ID, that ended up with an error and a failed order.

In Infrastructure & Operations, surrounding logs can also be displayed based on other criteria, like the host file or log source from which logs are collected. This allows quick and easy troubleshooting without writing or editing queries.

Logs in context across Dynatrace Apps

• Infrastructure & Operations leverages advanced AI capabilities that automatically discover and map all components within your infrastructure, including hosts, virtual machines, containers, and cloud instances.
• Databases offers comprehensive database monitoring capabilities, providing organizations with real-time visibility into the performance and health of their database environments.
• Clouds is a central hub for monitoring and managing multicloud environments, providing organizations with a unified view of their cloud infrastructure and services.
• Kubernetes delivers comprehensive monitoring and management capabilities for Kubernetes environments, enabling organizations to ensure the performance, availability, and scalability of their containerized workloads.

Stay tuned for even wider support of log data embedded seamlessly into the context of Dynatrace Apps, and better ways to get answers from logs without writing queries.

See for yourself

• Watch a demo of logs in context within various Dynatrace Apps in this Dynatrace University course.
• Open your free Dynatrace trial now to get started.

The post Unlock log analytics: Seamless insights without writing queries appeared first on Dynatrace news.