Hans Lougas | Dynatrace news https://www.dynatrace.com/news/blog/author/hans-lougas/ The tech industry is moving fast and our customers are as well. Stay up-to-date with the latest trends, best practices, thought leadership, and our solution's biweekly feature releases. Wed, 01 Jul 2026 09:48:05 +0000 en hourly 1 How Dynatrace supercharged log observability in 2025 https://www.dynatrace.com/news/blog/how-dynatrace-supercharged-log-observability-in-2025/ https://www.dynatrace.com/news/blog/how-dynatrace-supercharged-log-observability-in-2025/#respond Thu, 15 Jan 2026 17:18:49 +0000 https://www.dynatrace.com/news/?p=72456 Dynatrace Logs icon

Large enterprises such as Western Union, Vodafone, and United Airlines are ditching legacy log solutions in favor of a single, unified observability platform that delivers real-time insights and scalability at the petabyte level, as you’ll hear firsthand from them at Perform 2026. In this blog post, we’ll look back at the log-focused Dynatrace product releases […]

The post How Dynatrace supercharged log observability in 2025 appeared first on Dynatrace news.

]]>
Dynatrace Logs icon

Large enterprises such as Western Union, Vodafone, and United Airlines are ditching legacy log solutions in favor of a single, unified observability platform that delivers real-time insights and scalability at the petabyte level, as you’ll hear firsthand from them at Perform 2026.

In this blog post, we’ll look back at the log-focused Dynatrace product releases of 2025, while keeping in mind the three benefits that customers love most about Dynatrace:

  1. Fast log onboarding with unified ingestion from any source
  2. It’s easy to get started, yet powerful for your daily work
  3. Productivity boosts with Davis AI

Boost productivity with Davis AI

The promise of “Logs in Context” is simple:
Find the right log line at the right time, automatically and powered by AI.

The magic of Dynatrace is not a single feature or hyped AI. It’s the sum of many Dynatrace capabilities that comprise the foundation of the Dynatrace platform: Grail®, Smartscape®, Davis® AI, OpenPipeline®, and many others, that come at no extra cost, providing the automation and assistance you need.

Easily identify root causes and create tickets using the Dynatrace Problems app and logs.
Figure 1. Easily identify root causes and create tickets using the Dynatrace Problems app and logs.

If you aren’t yet using Dynatrace for your logs, stop stitching together clues across tools and say goodbye to manual swivel chair ops:

  • Logs in context: The right log lines appear automatically within the workflow or Dynatrace app you’re using. Whether that’s troubleshooting a service, reviewing Kubernetes node health, or investigating performance incidents of Infrastructure or cloud native apps.
  • Free of charge: Every in-context query, including surrounding logs, is now zero-rated (non-billable) when you view logs inside these Dynatrace core apps: Clouds, Infrastructure & Operations, Services, and Distributed Traces. While these apps don’t generate query consumption, ingestion and retention consumption are billed individually. We’re delivering the logs you need to take action – instantly, efficiently, and automatically correlated.
  • Leverage the power of Dynatrace Davis AI: With Dynatrace, features like “Explain logs” dramatically shorten time to action. Our customers report that their teams can more easily understand the possible causes and impacts of incidents without having to manually search for error codes in logs on Google.
  • By leveraging Davis AI, Workflow Automation, and integrations such as our ServiceNow partnership, customers can dramatically reduce the number of incidents; one of our customers reported reducing MTTI by 90%.

AI summaries are available across the Dynatrace platform and MCP server.

Explore logs, expand log messages, and comprehend them faster using the “explain log” AI feature.
Figure 2. Explore logs, expand log messages, and comprehend them faster using the “explain log” AI feature.

With Dynatrace, observability is not limited to cloud native apps. These features work seamlessly across cloud native, on-premises, hybrid, and traditional IT stacks. So, whether you’re on Kubernetes, a Mainframe, or an AWS Lambda function, the experience is the same.

Effortless for everyone, powerful for experts

Once your logs are ingested, you need to be able to understand them. This is where our Logs app shines for both new and expert Dynatrace users.

Pre-defined and admin-curated views boost productivity

Earlier this year, we improved the simplicity of applying complex and advanced queries with new data segmentation and advanced filters.

Using segments, admins and power-users can provide reusable and pre-scoped filters. When paired with dynamic variables, users can easily modify filter conditions.

Simultaneously, we continued enhancing the Logs app to provide advanced click-to-filter capabilities in various areas, like pinning frequent queries and filters:

  • Filter field: Suggest attributes, operators, and entities
  • Facets: Gain a quick understanding of patterns and groups, or build queries
  • Advanced filtering: Intuitive click-to-filter side pane, including JSON-structure log support with nesting
Combine segments and facets to create a pre-filtered view
Figure 3. Combine segments and facets to create a pre-filtered view

JSON‑structured log handling

Log messages aren’t always clean. A field might be hidden inside a nested message attribute or buried three levels deep in nested JSON.

Dynatrace log handling:

  • Detects and normalizes JSON.
  • Exposes nested fields in the UI without manual mapping.
  • Provides human-readable log messages in the results across all apps that use logs.

This way, you and your users can focus on analysis, not plumbing and normalizing logs.

Free text search surfaces the content you're looking for instantly, with human-readable results, even for JSON-structured log records
Figure 4. Free text search surfaces the content you’re looking for instantly, with human-readable results, even for JSON-structured log records

Correlation at scale

With Traces on Grail, your traces are automatically correlated in context with surfaced logs within the Distributed Tracing app, including associated exceptions.

The value you and your teams gain

If you’re accustomed to working with traces, you can continue using your troubleshooting routine and easily navigate from traces to logs and error exception messages. If you prefer to start your work by focusing on logs, you can achieve the same outcome.

The Dynatrace Distributed Tracing app automatically links logs with traces or spans.
Figure 5. The Dynatrace Distributed Tracing app automatically links logs to traces or spans.

Remember, Logs in context are free with the Distributed Tracing app!

Fast log onboarding with unified ingestion from any source

You want all your logs, and you want them fast. You don’t want to wrestle with YAML files, forward scripts, or configure custom collectors.

Centralized configuration, self-service management, and enabling teams with granular permissions to collect and ingest logs—these are what customers asked for:

  • OneAgent + Journald – Enhanced capabilities for automatically capturing logs on Linux machines with a single, centralized, configured agent: Dynatrace OneAgent®. Just deploy and watch the logs magically appear in your tenant.

Kubernetes logging made easy – The Dynatrace Kubernetes Logs Module gives you complete visibility without requiring OneAgent to operate in Full-Stack mode or to configure OTel manually.

Onboarding your Kubernetes cluster and logs using the Log Onboarding Wizard.
Figure 6. Onboarding your Kubernetes cluster and logs using the Log Onboarding Wizard.
  • Log Onboarding Wizard – To further simplify the onboarding experience, we’ve introduced a new wizard across several apps. When logs are missing, or you manually launch the wizard, it provides guided steps to onboard your logs, including creating an API key.
If you already have a standardized intake process in place for your teams, simply don’t provide one or all of the required permissions. Then your users won't be able to see the wizard or onboarding recommendations.
Figure 7. If you already have a standardized intake process in place for your teams, simply don’t provide one or all of the required permissions. Then your users won’t be able to see the wizard or onboarding recommendations.

Scale that never breaks

You can ingest up to 1 PB of logs per day per tenant, which should eliminate most sizing or scaling headaches. This bandwidth is part of the Dynatrace SaaS magic: Dynatrace Grail stores and processes everything in an indexless manner and using schema on-read. At the same time, OpenPipeline® routes the telemetry according to your rules and requirements defined in the pipelines.

  • Thousands of pipelines and self-service: Every pipeline has fine-grained permissions, so teams can create isolated pipelines and self-service onboarding, processing, and routing to buckets for retention.
  • 120+ parsing processors – From JSON normalization to custom field extraction, you can assign a processor to a pipeline and let OneAgent do the matching magic for you. Are you using OpenTelemetry or Cribl? Matching conditions or technology attribution offers you the same experience, regardless of the log source.
    10 MB log records – In our Go big with Dynatrace blog post, we discussed why large log records aren’t an anomaly and how customers benefit from out-of-the-box support for large log records.
    Figure 8. 10 MB log records – In our Go big with Dynatrace blog post, we discussed why large log records aren’t an anomaly and how customers benefit from out-of-the-box support for large log records.

With Dynatrace, your log ingestion stays ahead of your growth curve, no matter how many new sources you add.

Keep your costs predictable

Large enterprises often need to charge back to internal business units. We’ve introduced increased flexibility for existing features related to chargebacks:

  • Retain with Included Queries: Configurable on the individual bucket level, and seamlessly combinable with the established usage-based IRQ model.
  • Cost Allocation: Attribute your logs, metrics, and traces with business‑unit and product labels. This supports your FinOps efforts, as recently discussed in our blog post, Cost Allocation for Logs.

Best Practices: Not everything we delivered in 2025 was a product enhancement. We’ve also delivered a new best practices section in our product documentation, based on field feedback from pre-sales, post-sales, and support teams.

If you prefer to watch a webinar recording instead of reading, we recorded a video that walks you through all the best practices detailed in this blog post.

Dynatrace YouTube Series  | Optimize your logs: Save money and boost performance

Ready to get started?

Let’s make observability effortless, not overwhelming.

Your team can spend less time chasing logs and more time delivering value. Dive in today and experience the power of an observability platform that was built for the future of IT.

If you’re using Dynatrace SaaS with a DPS contract, all the features mentioned in this blog post are available to you. If you’re not, why not start a free trial today and experience the value yourself?

Resources

Dynatrace University – Free training that covers everything from basic log ingestion to advanced analytics.

Dynatrace Playground – Our free sandbox tenant with sample log files, ready to explore log

State of Log Management 2026 – Download the report to explore benchmark data on how AI workloads are exploding log volume and costs, and why unified observability is now essential for reliable, trustworthy AI.

The post How Dynatrace supercharged log observability in 2025 appeared first on Dynatrace news.

]]>
https://www.dynatrace.com/news/blog/how-dynatrace-supercharged-log-observability-in-2025/feed/ 0
Unlock log analytics: Seamless insights without writing queries https://www.dynatrace.com/news/blog/log-analytics-seamless-insights-without-writing-queries/ https://www.dynatrace.com/news/blog/log-analytics-seamless-insights-without-writing-queries/#respond Tue, 28 May 2024 14:48:22 +0000 https://www.dynatrace.com/news/?p=64183

Logs are an integral part of the daily workflow for your DevOps and SRE teams to understand what’s happening in your tech stack. No matter the industry you operate in or the scale of your business, getting value from log data is often slowed down by challenges: making sure the right logs are monitored, finding the relevant logs when you need answers, and making sense of logs in the context of other data like traces, events, and metrics.

The post Unlock log analytics: Seamless insights without writing queries appeared first on Dynatrace news.

]]>

Logs provide answers, but monitoring is a challenge

Manual tagging is error-prone

Making sure your required logs are monitored is a task distributed between the data owner and the monitoring administrator. Often, it comes down to provisioning YAML configuration files and listing the files or log sources required for monitoring. This manual, error-prone approach can lead to monitoring gaps, which become critical when a host or service has an outage or incident.

Finding the right logs is cumbersome

Even if your logs are monitored, you need to make sense of the vast data volume. As the scale and complexity of your tech stack grows, you might need to navigate the maze of hosts or Kubernetes clusters, apps, and microservices and understand the relevance and risks associated with logs originating from these entities. Challenges compound: Manual tagging of log sources has long been difficult regarding monitoring coverage. And you can’t assume the tagging is 100% correct to pinpoint the correct logs.

In the past, more work was needed to understand the context of log data. What about correlated trace data, host metrics, real-time vulnerability scanning results, or log messages captured just before an incident occurs? This context is vital to understanding issues.

Dynatrace automatically puts logs into context

Dynatrace Log Management and Analytics directly addresses these challenges. First, OneAgent takes care of log autodiscovery. Once logs are selected for monitoring, OneAgent enriches log data with the topological context you need. For example, OneAgent helps you monitor the logs from a Kubernetes environment with automatic enrichment that identifies the right cluster, namespace, container, and pod ID.

Once logs are stored in Dynatrace Grail™, our purpose-built data lakehouse for observability data, the logs are automatically shown in the right context. Finding answers begins with opening the right app for your use case.

Kubernetes logs in context in Dynatrace screenshot

You can easily pivot between a hot Kubernetes cluster and the log file related to the issue in 2-3 clicks in these Dynatrace® Apps: Infrastructure & Observability (I&O), Databases, Clouds, and Kubernetes.

Open a host, cluster, cloud service, or database view in one of these apps, and you immediately see logs alongside other relevant metrics, processes, SLOs, events, vulnerabilities, and data offered by the app.

By eliminating slow and manual correlation, lack of context, and getting visibility into the surrounding data, you reduce the risk of prolonged outages, mean time to repair, and tool sprawl.

Log data in Dynatrace

Get quicker answers

Let’s look at how logs in context can make your teams more effective.

Video thumbnail

Log histograms: Insight into log volumes and patterns

Open one of these Dynatrace Apps and select Logs for any listed entity (host, Kubernetes workload, cloud service, or database instance):

  • Infrastructure & Operations
  • Kubernetes
  • Databases
  • Clouds

You’ll see a histogram chart of log data with various severity levels (such as Error, Info, or Warning) relevant to the selected Dynatrace entity, giving you a clear understanding of log patterns and volumes over time. Is there a sudden spike in errors? A sudden drop in received log data? Depending on which app is in use, one glance at a histogram provides invaluable insight into managing clouds, databases, Kubernetes environments, and infrastructure.

hosts logs in context

Log analytics simplified: Deeper insights, no DQL required

Your team will immediately notice the streamlined log analysis capabilities below the histogram. Jump directly into log insights by selecting a recommended query, for example, to see the errors related to a problem detected by Davis® AI during the selected timeframe. Furthermore, your team can easily access all error logs within the specified timeframe displayed on the histogram or view all logs within that timeframe, all without writing any queries from scratch.

Surrounding logs display: Effortlessly navigate log context

You can see the result after opening a recommended query without leaving an app’s context. Upon expanding a single log entry, all relevant context provided by OneAgent during the ingestion process is displayed, making it easy to expand your analysis to the infrastructure or entity related to the error logs. For a single log record found, you can easily see the surrounding logs.

Look at this example of an online store payment service generating errors. The application owner found error logs related to unsupported credit cards. Select Surrounding logs to view the log messages for the whole transaction, based on the trace ID, that ended up with an error and a failed order.

Surrounding logs

In Infrastructure & Operations, surrounding logs can also be displayed based on other criteria, like the host file or log source from which logs are collected. This allows quick and easy troubleshooting without writing or editing queries.

Logs in context across Dynatrace Apps

  • Infrastructure & Operations leverages advanced AI capabilities that automatically discover and map all components within your infrastructure, including hosts, virtual machines, containers, and cloud instances.
  • Databases offers comprehensive database monitoring capabilities, providing organizations with real-time visibility into the performance and health of their database environments.
  • Clouds is a central hub for monitoring and managing multicloud environments, providing organizations with a unified view of their cloud infrastructure and services.
  • Kubernetes delivers comprehensive monitoring and management capabilities for Kubernetes environments, enabling organizations to ensure the performance, availability, and scalability of their containerized workloads.

Stay tuned for even wider support of log data embedded seamlessly into the context of Dynatrace Apps, and better ways to get answers from logs without writing queries.

See for yourself

Already have a Dynatrace account? See logs in context for yourself in the Dynatrace Playground.

The post Unlock log analytics: Seamless insights without writing queries appeared first on Dynatrace news.

]]>
https://www.dynatrace.com/news/blog/log-analytics-seamless-insights-without-writing-queries/feed/ 0
Enhance data management with Grail: Ultimate guide to custom buckets and security policies https://www.dynatrace.com/news/blog/enhance-data-management-with-grail-ultimate-guide-to-custom-buckets-and-security-policies/ https://www.dynatrace.com/news/blog/enhance-data-management-with-grail-ultimate-guide-to-custom-buckets-and-security-policies/#respond Fri, 06 Oct 2023 14:26:12 +0000 https://www.dynatrace.com/news/?p=59912 Application Security graphic

Logs now complete the observability picture alongside traces and metrics in the Dynatrace Grail™ data lakehouse. By following best practices for setting up buckets with security context in Grail, the appropriate log data is now always at the fingertips of the right teams.

The post Enhance data management with Grail: Ultimate guide to custom buckets and security policies appeared first on Dynatrace news.

]]>
Application Security graphic

Grail: Enterprise-ready data lakehouse

Grail, the Dynatrace causational data lakehouse, was explicitly designed for observability and security data, with artificial intelligence integrated into its foundation. We’ve further enhanced its capabilities to meet the high standards of large enterprises by incorporating record-level permission policies.

To fully utilize Grail features, it’s recommended that you incorporate its unique buckets and security policies at the beginning of your observability journey. Utilizing built-in mechanisms and customizing organization-specific policies maximizes the benefits of Grail capabilities.

Custom data buckets for faster queries, increased control, and custom retention periods

>> Scroll down to the bottom of this blog post to view a ~7-minute video demonstration of custom data buckets.

The first layer in the Grail data model consists of buckets and tables (and views for entities, which is outside this blog post’s scope).

Tables are a physical data model, essentially the type of observability data that you can store. Buckets are similar to folders, a physical storage location.

There is a default bucket for each table. Here is the list of tables and corresponding default buckets in Grail.

Table name Default bucket
logs default_logs
events default_events
metrics default_metrics
bizevents default_bizevents
dt.system.events dt_system_events
spans default_spans

The default buckets let you ingest data immediately, but you can also create additional custom buckets to make the most of Grail.

Address specific use cases with custom buckets

It’s logical to segregate high-volume data into its own bucket. This allows the data to be frequently queried and used separately from other scenarios.

For example, a separate bucket could be used for detailed logs from Dynatrace Synthetic nodes. Debug-level logs, which also generate high volumes and have a shorter lifespan or value period than other logs, could similarly benefit from dedicated storage. Keeping these logs separate decreases the data volume for other troubleshooting logs. This improves query speeds and reduces related costs for all other teams and apps.

Custom data buckets with Dynatrace Grail

Address organizational structure with custom buckets

Depending on your organization’s structure, you may find it beneficial to keep logs used by specific business units or departments in separate buckets. This approach makes queries faster for individual units, as they only query relevant logs, and it ensures distinct access separation.

Suppose a single Grail environment is central storage for pre-production and production systems. In that case, the use of separate buckets makes it easier to distinguish different stages, keeping production data separate from development or staging data.

Adopting this level of data segmentation helps to maximize Grail’s performance potential. If your typical queries only target a specific use case, business unit, or production stage, ensuring they don’t include unrelated buckets helps maintain efficiency and relevance.

Custom buckets unlock different retention periods.

Segmenting your data into multiple buckets also puts you in control of the data retention period. The simplicity of storing data in Grail is reflected in its retention policies; you choose how long to store each portion of your data, and you never have to think about managing archives or retrieving archived data. Coupled with the transparent pricing of GiB/day, you can set up buckets to exactly match your business needs.

While the built-in default_logs bucket has a retention period of 35 days, you have more options to choose from. Use Grail’s public bucket management API to create new buckets for which you can select data retention periods of 1 day to 10 years , or use the new Storage Management app to that with just a few clicks.

In conjunction with the previous example of keeping high-volume and short-lived logs separate, you might also need to keep your application data longer. For example, transaction data and user-profile logs might need to be retained for 12 or 18 months.

Use buckets to query only the log data you need

Whether looking for a “needle in a haystack” or reporting on data stored in Grail, you can start an advanced query with DQL by fetching data from one of the tables. At this point, you should familiarize yourself with the blog post Tailored access management, Part 2: Onboard users to Grail and AppEngine, which covers access to Grail tables and buckets.

Now, let’s take a look at a query example that puts this all into use.

fetch logs

| filter loglevel=="ERROR"

In this example, we query a certain table (logs) and filter the results by a field (loglevel) with a certain value (ERROR). Note that with such a query, you fetch logs from all the buckets the end-user can access.

Custom data buckets with Dynatrace Grail

Although this initially only includes the default bucket, you might also include other buckets (if these are available to the user). As you bring in more data and users to Grail, relying just on the default buckets is not the optimal setup.

This is where filtering on custom buckets comes in. This allows you to query data from a specific bucket.

fetch logs

| filter dt.system.bucket=="prod_infra_logs" and loglevel=="ERROR"

This example now includes an additional filter that restricts data retrieval from a certain bucket (prod_infra_logs).

Custom data buckets with Dynatrace Grail

Using buckets to query only the data you need significantly speeds up queries and reduces query costs.

Buckets for data with high-security requirements

Buckets can also be used for managing high-level access control of data.

You can store access control logs or payment provider events in separate buckets to grant only limited SecOps or business administrators access to these logs.

Custom data buckets with Dynatrace Grail

Grail and AppEngine’s new policy-based access management provides a way to do this with buckets. For example, by adding a WHERE clause to the policy statement, you can define a specific bucket (=) or a range of buckets (STARTSWITH). In this example, the policy grants access to all buckets that have names starting with prod_infra_.

ALLOW storage:buckets:read WHERE storage:bucket-name STARTSWITH "prod_infra_";

However, creating access policies solely on the bucket and table level is not scalable in a enterprise landscape, as one Dynatrace tenant can have a limited number of custom buckets. Instead, access control based on specific attributes like host groups or team assignments can be achieved using different policies that are based on supported attributes or security context.

Record-level permissions and security context

As covered in the previously linked blog post about access management, Dynatrace Grail brings a new architecture to permissions management. The new approach that uses security policies provides you with new dynamic controls for user authorization.

As using custom buckets opened up a basic approach to access where users could get access to a whole bucket, record-level permissions allow you to take a fine-grained approach.

This means that whenever you run a DQL query to fetch data from Grail, your policy-based access rights are evaluated, and records without defined access are filtered out.

This means your teams’ permissions are not constrained by data management decisions on a bucket level. Let’s say it makes sense to consolidate all short-living app debug logs to a bucket that has a short retention period. With record-level permissions, you can now ensure multiple app owners can see only their data in that bucket.

Another example would be a business unit admin who needs to have access to departmental data across buckets.

Custom data buckets with Dynatrace Grail

Permissions based on DQL fields and security context

To implement this in the Log Management and Analytics context, you can create policies with additional clauses that provide access.

ALLOW storage:logs:read
WHERE storage:k8s.namespace.name="abc"

 AND storage:dt.host_group.id STARTSWITH "org1-";

In this example, the policy allows access to logs that have a certain Kubernetes namespace (abc) and originate from a host group whose name must start with a specific string (org1-).

The list of standard table fields used in security policies provides flexibility for defining individual policies.

DQL fields Mainly used with
event.kind events, bizevents
event.type events, bizevents
event.provider events, bizevents
k8s.namespace.name events, bizevents, logs, metrics, spans
k8s.cluster.name events, bizevents, logs, metrics, spans
host.name events, bizevents, logs, metrics, spans
dt.host_group.id events, bizevents, logs, metrics, spans
metric.key metrics
service.name events, bizevents, logs, metrics, spans
log.source logs
dt.security_context events, bizevents, system, logs, metrics, spans, entities
gcp.project.id events, bizevents, logs, metrics
aws.account.id events, bizevents, logs, metrics
azure.subscription events, bizevents, logs, metrics
azure.resource.group events, bizevents, logs, metrics

When you look at the list of DQL fields, you’ll notice one reserved field, dt.security_context.

You can assign a value to dt.security_context during data ingest for use in a security policy, which is not covered by the list of previous DQL fields. You can explicitly set a value for dt.security_context for some logs, or take the value of an existing field.

Log monitoring security context in Dynatrace settings

In this example, logs from a particular source (dsfm) are enriched with a literal value for dt.security_context (sec-lvl-7) during log ingest. A policy with a specific clause can provide access to only logs with this security context. Security context rule management is available via settings In the Dynatrace web UI and API.

This approach to granular record-level permissions opens up the flexibility needed in enterprise environments. You can craft policies based on existing fields like Kubernetes cluster or namespace, a host or a host group, a service name, or a log source. Or you can use security context for any other use cases, like granting access based on an AWS account, a GCP project, an Azure subscription, a username, or a team name.

Take the first step now

Many organizations have found immediate value in working with logs in Grail. Starting from optimizing their business and opening revenue streams based on data in logs to unlocking real-time insights from observability data and eliminating hours of manual work per process, as the Bank of Montreal did recently. Utilizing the core aspects of Grail, like buckets and permissions, sets organizations on the path to success.

Next steps

  • Start a Dynatrace free trial and explore Log Management and Analytics powered by Grail
  • Read our documentation explaining buckets, permissions in Grail, security context for logs, and IAM
  • Record-level permissions for Grail are generally available with Dynatrace version 1.277. Custom buckets, in addition to bucket and table permissions, are available with Dynatrace version 1.265.
  • Use Storage Management app to create and manage custom Grail buckets and unlock custom retention times for your data since Dynatrace version 1.281.

Special thanks to Dominik Punz and Christian Kiesewetter for contributing to this blog post.

Video demo

Watch this 7-minute video to see how you can use custom data buckets to separate use cases, data retention, and access permissions in Dynatrace.

Video thumbnail
Use buckets to separate use cases, data retention, and access permissions (7-minute video)

The post Enhance data management with Grail: Ultimate guide to custom buckets and security policies appeared first on Dynatrace news.

]]>
https://www.dynatrace.com/news/blog/enhance-data-management-with-grail-ultimate-guide-to-custom-buckets-and-security-policies/feed/ 0
Stay in control of your data retention with Dynatrace Grail—from 10 days to 10 years https://www.dynatrace.com/news/blog/stay-in-control-of-your-data-retention/ https://www.dynatrace.com/news/blog/stay-in-control-of-your-data-retention/#respond Fri, 28 Apr 2023 08:00:16 +0000 https://www.dynatrace.com/news/?p=57284 Database graphic

Managing observability and business-data storage is essential to getting data-driven answers and setting up automation workflows. Traditionally, these efforts have led to compromises in cost, business requirements, and compliance with applicable regulations. And relying on an archive-and-retrieve solution isn’t an option because it’s slow and expensive to get value from your data. Thankfully, the new custom buckets in the Dynatrace Grail™ data lakehouse keep you in control of your data, make your data available at all times, and abolish data management overhead.

The post Stay in control of your data retention with Dynatrace Grail—from 10 days to 10 years appeared first on Dynatrace news.

]]>
Database graphic

Optimize cost and availability while staying compliant

Observability data like logs and metrics provide automated answers, root cause detection, and security issues.

Customer decisions about data retention are often determined by important security, privacy, and legal issues. Customers must comply with internal and external policies and regulations that might demand them to keep specific data stored for a minimum period of time (for example, audit logs). However, the opposite is also true—in some cases data must be deleted after a certain period of time. This is the case when a company no longer has legal grounds to retain its customer data, as outlined in privacy protection regulations.

Often customers make business decisions about data retention based on the value they get from keeping historical data and the associated data retention costs. This means compromising between keeping data available as long as possible for analysis while juggling the costs and overhead of storage, archiving, and retrieval. For example, suppose data has to be retained for a longer period because of legal or business reasons. In such a case, the data is archived in cold storage where it can only be accessed for analysis following a delay, re-ingestion into a log analysis tool, and reindexing to prepare the data for analysis.

Ultimately this leads to a lose-lose situation for customers—they have to pay for and maintain data storage but they can’t get answers from their data quickly and effortlessly when needed.

Grail gives you control and the answers from data

With Grail, Dynatrace provides control over data retention and access policies for granular portions of data called “buckets.” This allows you to design data management and retention policies based on individual requirements, starting from days of retention up to a decade.

By introducing control over data retention, Dynatrace doesn’t impose any additional complexities. Even with the flexibility of buckets, there is no additional overhead of data storage management, no archiving, no retrieval from archives, and no performance degradations when using retained data for answers.

The price of data retention is always transparent and uniform, based on the number of days the data is retained, with no hidden fees for managing data. The same applies to querying data with transparent pricing based on read-data volume, with no extra costs for querying older data.

Use buckets for any use case in a secure way

When using Log Management and Analytics or Business Observability with Grail, you can create custom buckets with specified data-retention periods. For example, you can route incoming log data to a specific bucket so selected team members can access it.

App developers might need to read logs from their environment for debugging purposes, but only for a specific timeframe. With Grail, it’s easy to create a bucket with ten days of retention time and provide all developers access to the data.

Infrastructure teams may need to work with host logs from recent months or quarters. To do this, infrastructure logs can be routed to a bucket with a retention period of three months to a year.

Local regulation often requires that security or audit logs be retained for 7 to 10 years. Such logs can be collected in a bucket with the required retention period, with only the security operations team having access to the logs.

A bucket can be wiped if, at any point in time, there is a need to delete the data stored in it. The reasons for this can vary from a changing business justification to data-privacy regulations. It’s also possible to extend or shorten a bucket’s retention period, which impacts how long existing data in a bucket is stored.

To support configuration-as-code for enterprise environments, creating, updating, and deleting data buckets in Grail is available through an API endpoint. This allows you to create new buckets, change the retention period of existing buckets, or delete buckets via an API call.

Bucket management follows a strict permission policy approach, where only users with corresponding permissions can create, update, or delete buckets. Every API call is saved in audit logs to document the complete picture of activities in your environments.

Get value from your data with the Dynatrace Grail today

The post Stay in control of your data retention with Dynatrace Grail—from 10 days to 10 years appeared first on Dynatrace news.

]]>
https://www.dynatrace.com/news/blog/stay-in-control-of-your-data-retention/feed/ 0