Protecting Personally Identifiable Information (PII) data in logs is crucial for supporting privacy and compliance. While Dynatrace OneAgent can mask data at capture/source, sensitive information like social security numbers (SSNs), payment details, IP addresses, or email addresses might still get through — especially when logs are ingested via other mechanisms such as directly using an API, via Fluent Bit, Cribl, or using the OpenTelemetry collector.

In this blog, practitioners will walk away with an in-depth understanding of how to create your own parsing rules. We’ll focus on masking/obfuscating email addresses in log events during the ingest process, leveraging Dynatrace OpenPipeline before logs are retained. Outside of this blog’s scope is the attribute-based access capability and mask on read.

Best Practice: Before we start masking data at the time of ingest, we’ll validate our configuration within a Dynatrace Notebook to prevent unwanted data loss caused by inaccurate patterns. We’ll walk through:

• Identifying patterns
• Creating a masking pipeline in OpenPipeline
• Validating the results

Step 1: Simulate logs with sensitive data

To demonstrate, we’ll work with logs containing email addresses and IP addresses. These logs use example domains and IPs (e.g., example.com and 203.0.113.x) for documentation purposes.

Here are the eight sample logs sent from various demo applications to the Dynatrace Playground:

• 2025-04-22 11:46:38 [ERROR|[203.0.113.13]
  |K9p8Q3xJwZrStUv4XyZ7AbC2n5M6h8T1v0L2r4 
  |com.example.exmplstore.security.examplePersistentTokenRepository] 
  Can't find credentials for series marie_curie@example.com

• 2025-04-22 13:50:43 [INFO 
  |[203.0.113.112] |A1b2C3d4EfGhIjK5LmN6oPq7RsT8uVw9XyZ0 
  |class com.example.exmplfacades.order.exampleCheckoutLogger Checkout ABC] 
  Receive API Request:method=placePayOrder, cartCode=302132310, 
  customerID=freddie@example.com
  |com.example.exmplstore.checkout.request.PayPlaceOrderRequestDto@3d3a53d5|]

• 2025-04-22 13:58:24 [INFO |||com.example.exmpl.BusinessProcessLoggingAspect] 
  Finish Action:[ PerformSubscriptionAction ], BusinessProcessCode: 
  [ customerRegistrationProcess-martin_luther@example.com-1745294290737],
  OrderCode:[ n/a ]

• 2025-04-29 10:16:13 [INFO ||
  |com.example.exmplmarketing.action.exampleSendCustomerNotificationAction] 
  Successfully sent email forgottenPassword message for process 
  forgottenPasswordProcess-jane_austen@example.com-1745885767984

• 2025-04-29 10:24:26 [INFO |[203.0.113.14] 
  |M3n4P5q6RsTuVwXyZ7aBc8DeF9gHi0JkL2|com.example.exmpl.UserDeleteInterceptor] 
  Deleting userId: cleopatra@example.com, actioned by userId: anonymous

• 2025-04-29 10:24:36 [INFO |[203.0.113.19] 
  |T2u3V4w5XyZaBc6DeF7gHi8JkL9mNo0PqR1|com.example.exmpl.AdvantageApiClient] 
  Loyalty: Check loyalty account exist for email: leonardo_davinci@example.com , 
  wodCorrelationId 2fee137d-915e-46fb-b390-c69aae3f150

• 2025-04-29 10:22:47 [INFO ||
  |com.example.exmplbusproc.aop.BusinessProcessLoggingAspect] 
  Begin Action: [ exampleAdvantageLinkEmailAction ], BusinessProcessCode: 
  [ advantageLinkEmailProcess-albert_einstein@example.com-1745886166709], 
  OrderCode:[ n/a ]

• 10.96.152.11 - - [29/Apr/2025:10:33:43 +1000] 
  "GET /reminder/shakespeare@example.com/1 HTTP/1.1" 200 90 
  "https://www.example.com/my-account/order-details/306399901" 
  "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 
  (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36" 203.0.113.145 - 
  T2u3V4w5XyZaBc6DeF7gHi8JkL9mNo0PqR1 100 

As you can see, all of these logs contain email addresses, which must be masked before storing in Dynatrace Grail™.

Step 2: Identify patterns in logs

We can use the Notebooks app to query stored logs and search for log lines that contain email addresses. To do so, we create a new notebook, add a DQL section, and execute the following command:

fetch logs
| filter matchesPattern(content, "LD '@' [A-Za-z0-9'_']* '.'
[A-Za-z]* (LD | EOS)") 

The second part of this command executes a pattern matching command, which makes use of the Dynatrace Pattern Language (DPL) to define the pattern and reduces the result to show only those entries that contain an email address.

Let’s have a closer look at the DPL pattern itself:

LD '@' [A-Za-z0-9'-']* '.' [A-Za-z]* LD EOS 

and break down the elements:

• LD: Matches any characters (letters, digits, or others) until the next non-optional matcher (in this case, ‘@’) within the scope of a line.
• '@': Matches the actual @ symbol.
• [A-Za-z0-9'-']*: Matches zero or more letters, digits, or hyphens, forming the domain part of the email address.
• '.': Matches the literal “.” (e.g. in .com).
• [A-Za-z]*: Matches zero or more letters, representing the top-level domain. Note: If you wish to match double-dotted TLD’s like .co.uk use this pattern instead: [A-Za-z’.’]*
• (LD | EOS): The pattern matches either a sequence of characters (letters, digits, or others) via LD to the end of the line after the email address. However, if the last word in the log line is the email address, then optionally the EOS will match the end of string.

Step 3: Analyze logs for email address patterns

After confirming email addresses exist, we examine the log lines to identify patterns preceding them. Here are the literals just before the email addresses, found in our returned sample logs:

• order confirmation email sent to
• find credentials for series
• customerID=
• customerRegistrationProcess-
• forgottenPasswordProcess-
• Deleting userId: 
• Check loyalty account exist for email: 
• advantageLinkEmailProcess-
• GET /reminder/ 

These literals will help us target the email addresses for masking.

Step 4: Check the pattern to test the process of masking the email addresses

We will now parse the email addresses and mask the local part (before the @ symbol) using Dynatrace Query Language (DQL). This is parsing at query time to test whether the pattern is valid or not. It does not change or update the unmasked data that is already stored in Grail. This step ensures that the pattern we’ll be using in the following steps with OpenPipeline are valid. To achieve this, we must open the DPL architect by selecting the 3 dots on the right hand of a row-element in the content column and select “Extract fields.”

Parse the email field

We will use the following parsing pattern with DQL to extract email addresses based on the literals identified:

Pattern:

 
LD  
( 
'order confirmation email sent to' |  
'find credentials for series' | 
'customerID=' |  
'customerRegistrationProcess-' |  
'forgottenPasswordProcess-' |  
'Deleting userId: ' |  
'Check loyalty account exist for email: ' | 'advantageLinkEmailProcess-' |  
'GET /reminder/') LD:email 
 

Breakdown: 

• LD 
  Matches any non-whitespace characters before the literal.
• ('Can't find credentials for series' | ...) 
  Matches any of the specified literals.
• LD:email 
  Captures the email address into a field named email.

Validate the parsing rule using DPL architect in Dynatrace Notebooks by pasting the parse command (starting with LD):

Make sure you select the ‘Add to preview’ action button to ensure there are no “Unmatched records” and all the matched records based on the previous filter can be truly passed:

Test the process of masking the email

In the screenshot below, you can notice that the logs do contain unmasked email addresses:

Test replacing and obfuscating the email address in the log content with a masked value (e.g., xyz):
Add below your existing query the fieldsAdd operation:

| fieldsAdd content = replacePattern(content,  
 
" 
<< 
( 
 'order confirmation email sent to' | 
'find credentials for series ' |  
 'customerID=' |  
 'customerRegistrationProcess-' |  
 'forgottenPasswordProcess-' |  
 'Deleting userId: ' |  
 'Check loyalty account exist for email: ' |  
 'advantageLinkEmailProcess-' |  
 'GET /reminder/' 
) LD:email  
>> 
'@' 
",  
 
“xyz”) 
 

This replaces the email address (stored in the email field) with xyz in the content field.

What it does:

This DQL command modifies the content field in a Dynatrace log record by replacing specific patterns defined – email addresses in this case – with the defined string “xyz”.
Let’s have a closer look at the technical details:

1. replacePattern Function:
   The replacePattern function searches the content field for matches of a specified pattern and replaces them with a given string (here, “xyz”).
2. DPL Modifier – Lookaround:
   Positive Look Behind Modifier <<
   Pattern:

<<
(
'for series' |
'customerID=' | 
'order confirmation email sent to' |
'customerRegistrationProcess-' |
' process forgottenPasswordProcess-' |
'userId: ' |
'Check OnePass account exist for email: ' |
'MobileApp customerRef=' |
'advantageLinkEmailProcess-' |
'GET /reminder/'
)

1. • Explanation: The << modifier looks up to 64 bytes before the current position in the log to check for any of the specified phrases. If one is found, the pattern continues. Our pattern includes <<('customerID=' 'GET /reminder/'). If the log line contains customerID=marie_currie@example.com, the modifier confirms that customerID= appears within 64 bytes before the email address, so the match succeeds. If that phrase isn’t found in the preceding 64 bytes, the match fails. Read more about DPL modifiers in our product documentation
2. Positive Look Ahead >>
   Pattern:
   LD:email >> ‘@' Explanation:  The >> modifier performs a look-ahead check. It ensures the pattern only matches if a specific condition appears after the current position. In this case, after capturing the local part of the email with LD:email, the pattern verifies that an @ symbol follows. This confirms the captured text is indeed part of an email address. For example, take the logline “Can't find credentials for series marie_currie@example.com” The pattern first uses LD:email to capture marie_currie as the local part. The >> '@' check then looks ahead and confirms that @ immediately follows. Because the condition is met, the match succeeds. But if the @ symbol were missing (e.g., Can't find credentials for series marie_curieexample.com), the match would fail.
3. Replacement:
   The matched pattern (e.g., marie_currie) is replaced with “xyz”. This masks the local part of the email address while leaving the rest of the log entry intact. For instance:
   • Before: customerID=marie_currie@example.com
   • After: xyz@example.com
4. fieldsAdd content = …: 
   The modified content (with the email local part masked) is stored back into the content field, overwriting the original value.

Additional guidance:

Follow these steps if you wish to mask the entire email address, including the domain:
If you wish to mask or extract the entire email address, you could use a pattern like below:

<<
( 
'order confirmation email sent to ' | 
'find credentials for series ' |  
'customerID=' |  
'customerRegistrationProcess-' |  
'forgottenPasswordProcess-' |  
'Deleting userId: ' |  
'Check loyalty account exist for email: ' |  
'advantageLinkEmailProcess-' |  
'GET /reminder/' 
) 
(LD '@'[A-Za-z0-9'-'']* '.' [A-Za-z]*):email 

In addition to the modifiers and pattern matching literals that were explained earlier, below is the explanation of the last line on how the breakdown of the DPL pattern language:

(LD '@'[A-Za-z0-9'-'']* '.' [A-Za-z]*):email 

1. LD: Matches line data.
2. '@‘: Matches the ‘@’ symbol.
3. [A-Za-z0-9'-'']*: Matches any sequence of alphanumeric characters, hyphens, and single quotes, occurring zero or more times.  Read more here.
4. '.': Matches the ‘.’ (dot) character.
5. [A-Za-z]*: Matches any sequence of alphabetic characters, occurring zero or more times.
6. :email: Captures the matched data and labels it as email. This is not needed if you don’t wish to capture the email address.

Step 5: Filter logs to avoid unintended masking

To ensure we only mask logs from specific sources and pattern containers, apply additional filters to isolate those logs only.

As a best practice, you should pre-filter your logs, by applying this filter directly underneath your fetch logs statement.

Filter by Container Names:

| filter k8s.container.name == "checkout"
OR k8s.container.name == "astroshop"
OR k8s.container.name == "aks-playground"

Filter by Content Patterns:

filter  
( 
matchesPhrase(content, "find credentials for series ") OR  
matchesPhrase(content, "order confirmation email sent to ") | 
matchesPhrase(content, "customerID=") OR  
matchesPhrase(content, "customerRegistrationProcess-") OR  
matchesPhrase(content, "forgottenPasswordProcess-") OR  
matchesPhrase(content, "Deleting userId: ") OR  
matchesPhrase(content, "Check loyalty account exist for email: ") OR 
matchesPhrase(content, "advantageLinkEmailProcess-") OR 
matchesPhrase(content, "GET /reminder/") 
) 

Complete DQL Query

Here’s the full DQL query combining all steps for your reference and to copy/paste:

fetch logs 
| filter  
( 
k8s.container.name == "checkout" OR  
k8s.container.name == "astroshop" OR  
k8s.container.name == "aks-playground" 
) 
AND 
// Match only logs that have specific phrases  
( 
matchesPhrase(content, "find credentials for series ") OR  
matchesPhrase(content, "order confirmation email sent to ") OR  
matchesPhrase(content, "customerID=") OR  
matchesPhrase(content, "customerRegistrationProcess-") OR  
matchesPhrase(content, "forgottenPasswordProcess-") OR  
matchesPhrase(content, "Deleting userId: ") OR  
matchesPhrase(content, "Check loyalty account exist for email: ") OR 
matchesPhrase(content, "advantageLinkEmailProcess-") OR 
matchesPhrase(content, "GET /reminder/") 
) 
| // extract the contents just after these literals that contains the localpart/username within the email address and replace the pattern that comes after these strings. 
fieldsAdd content = replacePattern(content,  
 
" 
<< 
( 
'order confirmation email sent to' | 
'find credentials for series ' |  
'customerID=' |  
'customerRegistrationProcess-' |  
'forgottenPasswordProcess-' |  
'Deleting userId: ' |  
'Check loyalty account exist for email: ' |    
'advantageLinkEmailProcess-' |  
 'GET /reminder/' 
) LD:email  
>> 
'@' 
",  
 
“xyz”) 

Store this query in a Dynatrace Notebook for future reference, and feel free to select the Run button to validate that the output matches your expectations.

All our earlier steps did not actually mask the incoming logs, but applied our masking at read/query, and secondly, just for us as the users of the Notebook.

The Notebook and its DQL snippet simply masked the data we visualized. Although this is useful for preventing users from displaying unmasked data when automated, it is even better if logs with PII are masked during ingest time.

Now that we have validated that our pattern works, we can use OpenPipeline to ensure that any log matching the DQL processor rule will undergo a series of processes that transform the log events during the ingest process.

Step 6: Set up OpenPipeline for real-time masking

Let’s configure OpenPipeline to mask email addresses at log ingestion.

1. Launch OpenPipeline in Settings app: Navigate to OpenPipeline using search or the CTRL+K shortcut. Then select Logs from the OpenPipeline menu.
2. Create a New Pipeline:
   • Go to Pipelines and create a new pipeline.
   • Provide a meaningful name (e.g., “Mask Email Addresses”).
   • In the Processing tab, add a DQL processor.
3. Configure the DQL Processor:
   • Name the processor (e.g., “mask email address in astroshop OR checkout OR aks-playground”).
   • Replace the default matching condition true with our patterns defined:

   • ( 
     k8s.container.name == "astroshop" OR  
     k8s.container.name == "checkout" OR  
     k8s.container.name == "aks-playground" 
     ) AND 
     // Match only logs that have specific phrases 
     ( 
     matchesPhrase(content, "order confirmation email sent to ") OR matchesPhrase(content, "find credentials for series ") OR  
     matchesPhrase(content, "customerID=") OR  
     matchesPhrase(content, "customerRegistrationProcess-") OR  
     matchesPhrase(content, "forgottenPasswordProcess-") OR  
     matchesPhrase(content, "Deleting userId: ") OR  
     matchesPhrase(content, "Check loyalty account exist for email: ") OR 
     matchesPhrase(content, "advantageLinkEmailProcess-") OR 
     matchesPhrase(content, "GET /reminder/") 
     )

   • Define what action should be applied when the condition matches, by defining the DQL processor:

     fieldsAdd content = replacePattern(content,  
      
     " 
     << 
     ( 
      'find credentials for series ' |  
      'order confirmation email sent to ' |  
      'customerID=' |  
      'customerRegistrationProcess-' |  
      'forgottenPasswordProcess-' |  
      'Deleting userId: ' |  
      'Check loyalty account exist for email: ' |  
      'advantageLinkEmailProcess-' |  
      'GET /reminder/' 
     ) LD:email  
     >> 
     '@' 
     ",  
      
     “xyz”)

While it is suggested to test your new masking with sample data, you can alternatively input ‘example’ as text and select to save the processor and pipeline, as we have validated them earlier in our Notebook.

4. Set up and define Dynamic Routing:

• Select the Dynamic routing tab
• Create a new Dynamic route named “Email ID masking for specific container names.”
• Use the same condition as the matching condition before.
• Connect the route to the pipeline you’ve just created.

Step 7: Validate the masking

To confirm the email addresses are masked:

1. Re-run the DQL query in your Dynatrace Notebook, but this time without the parsing.
2. Check the content field in the logs to ensure email addresses are replaced with xyz.

For example, marie_currie@example.com should now appear as xyz@example.com.
We can verify this by looking for these logs, and we can observe that the email addresses have been fully masked without any parsing at query time. Additionally, you can notice that the logs have the dt.openpipeline.pipelines attribute attached with the pipeline that was used during ingest time. This also confirms that the log was processed in that pipeline.

Conclusion

In this blog, we used email masking at ingest to demonstrate how to effectively obfuscate various types of sensitive data in logs using OpenPipeline, protecting sensitive data in real-time during the ingest process. By leveraging OpenPipeline’s powerful DQL-based processing and dynamic routing capabilities, you can precisely target specific Kubernetes containers or just any log source with patterns to enhance data privacy and security.

Take the next step

Explore OpenPipeline’s advanced features to:

• Mask sensitive data like SSNs, IDs, and more
• Streamline log management workflows with dynamic routing
• Build custom pipelines in Dynatrace to control your log data

Streamline Compliance

For comprehensive data-subject rights management, consider the Sensitive Data Center. app. Efficiently manage end-user personal data requests in Grail, with support for regulations like GDPR and CCPA.

…

Addendum: Exploring this approach in the Dynatrace Playground tenant:

A sample dataset, preloaded with tailored pattern matching and masking for the following steps and explanations, is available in the Dynatrace Playground tenant within this Dynatrace Notebook for hands-on exploration.

The individual steps can also be observed in the playground tenant with the logs ingested from the astroshop demo, while unfortunately, you lack the permissions to configure new pipelines and processors in Playground tenant.

If you look to test this within your personal tenant or a free trial tenant without actual logs containing PII, or running Astroshop yourself, you can download a copy of this Notebook or test with the pattern below, using the data command to generate sample data in a Dynatrace Notebook.

data  
record(content = "2025-04-22 11:46:38 [ERROR|[203.0.113.13] |K9p8Q3xJwZrStUv4XyZ7AbC2n5M6h8T1v0L2r4 |com.example.exmplstore.security.examplePersistentTokenRepository] Can't find credentials for series marie_curie@example.com", log.source = "bash-script"), 
record(content = "2025-04-22 13:50:43 [INFO |[203.0.113.112] |A1b2C3d4EfGhIjK5LmN6oPq7RsT8uVw9XyZ0 |class com.example.exmplfacades.order.exampleCheckoutLogger Checkout ABC] Receive API Request:method=placePayOrder, cartCode=302132310, customerID=freddie@example.com|com.example.exmplstore.checkout.request.PayPlaceOrderRequestDto@3d3a53d5|", log.source = "bash-script"), 
record(content = "2025-04-22 13:58:24 [INFO |||com.example.exmpl.BusinessProcessLoggingAspect] Finish Action: [ PerformSubscriptionAction ], BusinessProcessCode: [ customerRegistrationProcess-martin_luther@example.com-1745294290737], OrderCode:[ n/a ]", log.source = "bash-script"), 
record(content = "2025-04-29 10:16:13 [INFO |||com.example.exmplmarketing.action.exampleSendCustomerNotificationAction] Successfully sent email forgottenPassword message for process forgottenPasswordProcess-jane_austen@example.com-1745885767984", log.source = "bash-script"), 
record(content = "2025-04-29 10:24:26 [INFO |[203.0.113.14] |M3n4P5q6RsTuVwXyZ7aBc8DeF9gHi0JkL2|com.example.exmpl.UserDeleteInterceptor] Deleting userId: cleopatra@example.com, actioned by userId: anonymous", log.source = "bash-script"), 
record(content = "2025-04-29 10:24:36 [INFO |[203.0.113.19] |T2u3V4w5XyZaBc6DeF7gHi8JkL9mNo0PqR1|com.example.exmpl.AdvantageApiClient] Loyalty: Check loyalty account exist for email: leonardo_davinci@example.com , wodCorrelationId 2fee137d-915e-46fb-b390-c69aae3f150", log.source = "bash-script"), 
record(content = "2025-04-29 10:22:47 [INFO |||com.example.exmplbusproc.aop.BusinessProcessLoggingAspect] Begin Action: [ exampleAdvantageLinkEmailAction ], BusinessProcessCode: [ advantageLinkEmailProcess-albert_einstein@example.com-1745886166709], OrderCode:[ n/a ]", log.source = "bash-script"), 
record(content = "10.96.152.11 - - [29/Apr/2025:10:33:43 +1000] \"GET /reminder/shakespeare@example.com/1 HTTP/1.1\" 200 90 \"https://www.example.com/my-account/order-details/306399901\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36\" 203.0.113.145 - T2u3V4w5XyZaBc6DeF7gHi8JkL9mNo0PqR1 100", log.source = "bash-script") 
 
| filter matchesPattern(content, "LD '@' [A-Za-z0-9'_']* '.' [A-Za-z]* (LD | EOS)")

The post How to mask PII like email addresses appearing in logs with Dynatrace: An advanced use case appeared first on Dynatrace news.

Log forwarding includes metrics, spans, and events

Logs are a core pillar of observability, and in many organizations, logs serve at least a dual purpose. They drive day-to-day troubleshooting, root cause analysis, security investigations, and many other use cases.

Logs also need to be available for compliance audits, Business Intelligence (BI) analysis, and traditional Security Information Event Management (SIEM) solutions. The challenge is doing this without relying on bolted-on forwarders or rigid, costly integrations.

With the introduction of the new Dynatrace OpenPipeline capability, we’re providing you with the freedom to precisely define your organization’s forwarding, paired with flexible and unified log collection and processing.

_{Figure 1. OpenPipeline provides rich configuration and customization options for each forwarding setup.}

Update – June 2026

Data egress will be in General Availability starting June 17, 2026.

OpenPipeline now supports forwarding for all data types ingested into Dynatrace, including metrics, spans, and events, giving you a single, unified pipeline for your entire observability dataset. The same principles described below apply across the board: forward before or after processing, export in open formats, and route to the storage or third-party destination of your choice, regardless of signal type.

Read on for the full details of how OpenPipeline forwarding works.

Centralized log collection and processing

OneAgent, Cribl, OpenTelemetry, AWS Firehose, journalD, Logstash, and Fluent Bit … Have you heard of any of these tools? Maybe you’re using all of these at the same time.

With Dynatrace, you have the choice of using one or all of these tools simultaneously. Dynatrace OpenPipeline allows you to process and transform them in the same way, regardless of their source, and to move seamlessly from one shipping method to another. Just as our customer, United Wholesale Mortgages, successfully consolidated their tools while moving logs off Splunk.

Forwarding logs un/processed

Depending on your industry or geography, you might be required to retain logs in an unprocessed state on third-party-managed storage, just as you might have used tape backups in the old days. With Dynatrace OpenPipeline, you can configure a forward-before-process to retain log events in an unaltered state.

Alternatively, you can first send log events to your defined processing pipeline, extract or convert the logs into metrics, and even drop unnecessary details before determining what to forward-after-processing to your object storage.

The last step in the pipeline is to decide whether to route log events to a Dynatrace Grail® bucket for retention or drop them.

Overcome vendor lock-in with precise, open forwarding

While other log management solutions provide proprietary log archive formats, you can overcome these challenges with Dynatrace and log forwarding for data archiving.

The Dynatrace OpenPipeline log forwarding capability delivers log events in a standardized and compressed NDJSON (Newline Delimited JSON) (.JSON.GZ) archive format, for which you can define the details, such as segmentation and filename prefixes.

Residing on destinations like AWS S3 or Azure Blob storage, other 3^rd party solutions such as Microsoft Sentinel, Snowflake, or Tibco Spotfire can easily leverage these archives for use cases not covered by your observability platform.

Cost-effective long-term retention for logs

Dynatrace Grail offers cost-effective log retention for up to 10 years, accessible at any time with the same speed as on day one of ingestion, supporting a daily log ingest volume of 1 PB per tenant.

Many of our customers already retain years’ worth of logs today, always accessible without the headaches of re-ingestion, re-indexing, or storage tiering and scaling concepts.

Still, there are use cases where aged logs become low-value, low-touch data.

You may want to keep certain log events available in Grail for the first 60 or 90 days for troubleshooting purposes, and then store a copy in an archive for years.

NDJSON-exported log archives are an ideal file format for further increasing cost-effectiveness and are ready for handover to other teams or parties for investigation, thanks to the standardized JSON.GZ format. These are especially valuable for use cases like:

• Reviewing access logs during a post-mortem security incident
• Providing transaction logs to fulfill a court order
• File access audit-trial review

Tool consolidation with OpenPipeline and log forwarding

Log archiving presents an opportunity and an integration strategy for solutions that don’t yet integrate seamlessly with Dynatrace.

More importantly, OpenPipeline provides you with processing and transformation capabilities, volume control, and a reliable cadence for transmitting your log events.

You no longer have to rely on custom-defined scripts for the splitting and shipping of your logs. And there are no worries about truncating large log events; Dynatrace supports up to 10 MB.

Forget about all the retransmission headaches with custom log shippers when a connection breaks; all this is baked natively into Dynatrace components such as OneAgent, ActiveGate, and others.

Start leveraging OpenPipeline and log forwarding today and retire your third-party forwarding tool stack.

For complete details, go to Log Forwarding in Dynatrace Documentation.

The post Flexible vendor-agnostic log forwarding with OpenPipeline appeared first on Dynatrace news.

Large enterprises such as Western Union, Vodafone, and United Airlines are ditching legacy log solutions in favor of a single, unified observability platform that delivers real-time insights and scalability at the petabyte level, as you’ll hear firsthand from them at Perform 2026.

In this blog post, we’ll look back at the log-focused Dynatrace product releases of 2025, while keeping in mind the three benefits that customers love most about Dynatrace:

1. Fast log onboarding with unified ingestion from any source
2. It’s easy to get started, yet powerful for your daily work
3. Productivity boosts with Davis AI

Boost productivity with Davis AI

The promise of “Logs in Context” is simple:
Find the right log line at the right time, automatically and powered by AI.

The magic of Dynatrace is not a single feature or hyped AI. It’s the sum of many Dynatrace capabilities that comprise the foundation of the Dynatrace platform: Grail®, Smartscape®, Davis® AI, OpenPipeline®, and many others, that come at no extra cost, providing the automation and assistance you need.

If you aren’t yet using Dynatrace for your logs, stop stitching together clues across tools and say goodbye to manual swivel chair ops:

• Logs in context: The right log lines appear automatically within the workflow or Dynatrace app you’re using. Whether that’s troubleshooting a service, reviewing Kubernetes node health, or investigating performance incidents of Infrastructure or cloud native apps.
• Free of charge: Every in-context query, including surrounding logs, is now zero-rated (non-billable) when you view logs inside these Dynatrace core apps: Clouds, Infrastructure & Operations, Services, and Distributed Traces. While these apps don’t generate query consumption, ingestion and retention consumption are billed individually. We’re delivering the logs you need to take action – instantly, efficiently, and automatically correlated.
• Leverage the power of Dynatrace Davis AI: With Dynatrace, features like “Explain logs” dramatically shorten time to action. Our customers report that their teams can more easily understand the possible causes and impacts of incidents without having to manually search for error codes in logs on Google.
• By leveraging Davis AI, Workflow Automation, and integrations such as our ServiceNow partnership, customers can dramatically reduce the number of incidents; one of our customers reported reducing MTTI by 90%.

AI summaries are available across the Dynatrace platform and MCP server.

With Dynatrace, observability is not limited to cloud native apps. These features work seamlessly across cloud native, on-premises, hybrid, and traditional IT stacks. So, whether you’re on Kubernetes, a Mainframe, or an AWS Lambda function, the experience is the same.

Effortless for everyone, powerful for experts

Once your logs are ingested, you need to be able to understand them. This is where our Logs app shines for both new and expert Dynatrace users.

Pre-defined and admin-curated views boost productivity

Earlier this year, we improved the simplicity of applying complex and advanced queries with new data segmentation and advanced filters.

Using segments, admins and power-users can provide reusable and pre-scoped filters. When paired with dynamic variables, users can easily modify filter conditions.

Simultaneously, we continued enhancing the Logs app to provide advanced click-to-filter capabilities in various areas, like pinning frequent queries and filters:

• Filter field: Suggest attributes, operators, and entities
• Facets: Gain a quick understanding of patterns and groups, or build queries
• Advanced filtering: Intuitive click-to-filter side pane, including JSON-structure log support with nesting

JSON‑structured log handling

Log messages aren’t always clean. A field might be hidden inside a nested message attribute or buried three levels deep in nested JSON.

Dynatrace log handling:

• Detects and normalizes JSON.
• Exposes nested fields in the UI without manual mapping.
• Provides human-readable log messages in the results across all apps that use logs.

This way, you and your users can focus on analysis, not plumbing and normalizing logs.

Correlation at scale

With Traces on Grail, your traces are automatically correlated in context with surfaced logs within the Distributed Tracing app, including associated exceptions.

The value you and your teams gain

If you’re accustomed to working with traces, you can continue using your troubleshooting routine and easily navigate from traces to logs and error exception messages. If you prefer to start your work by focusing on logs, you can achieve the same outcome.

Remember, Logs in context are free with the Distributed Tracing app!

Fast log onboarding with unified ingestion from any source

You want all your logs, and you want them fast. You don’t want to wrestle with YAML files, forward scripts, or configure custom collectors.

Centralized configuration, self-service management, and enabling teams with granular permissions to collect and ingest logs—these are what customers asked for:

• OneAgent + Journald – Enhanced capabilities for automatically capturing logs on Linux machines with a single, centralized, configured agent: Dynatrace OneAgent®. Just deploy and watch the logs magically appear in your tenant.

Kubernetes logging made easy – The Dynatrace Kubernetes Logs Module gives you complete visibility without requiring OneAgent to operate in Full-Stack mode or to configure OTel manually.

• Log Onboarding Wizard – To further simplify the onboarding experience, we’ve introduced a new wizard across several apps. When logs are missing, or you manually launch the wizard, it provides guided steps to onboard your logs, including creating an API key.

Scale that never breaks

You can ingest up to 1 PB of logs per day per tenant, which should eliminate most sizing or scaling headaches. This bandwidth is part of the Dynatrace SaaS magic: Dynatrace Grail stores and processes everything in an indexless manner and using schema on-read. At the same time, OpenPipeline® routes the telemetry according to your rules and requirements defined in the pipelines.

• Thousands of pipelines and self-service: Every pipeline has fine-grained permissions, so teams can create isolated pipelines and self-service onboarding, processing, and routing to buckets for retention.
• 120+ parsing processors – From JSON normalization to custom field extraction, you can assign a processor to a pipeline and let OneAgent do the matching magic for you. Are you using OpenTelemetry or Cribl? Matching conditions or technology attribution offers you the same experience, regardless of the log source.

  

With Dynatrace, your log ingestion stays ahead of your growth curve, no matter how many new sources you add.

Keep your costs predictable

Large enterprises often need to charge back to internal business units. We’ve introduced increased flexibility for existing features related to chargebacks:

• Retain with Included Queries: Configurable on the individual bucket level, and seamlessly combinable with the established usage-based IRQ model.
• Cost Allocation: Attribute your logs, metrics, and traces with business‑unit and product labels. This supports your FinOps efforts, as recently discussed in our blog post, Cost Allocation for Logs.

Best Practices: Not everything we delivered in 2025 was a product enhancement. We’ve also delivered a new best practices section in our product documentation, based on field feedback from pre-sales, post-sales, and support teams.

If you prefer to watch a webinar recording instead of reading, we recorded a video that walks you through all the best practices detailed in this blog post.

Dynatrace YouTube Series  | Optimize your logs: Save money and boost performance

Ready to get started?

Let’s make observability effortless, not overwhelming.

Your team can spend less time chasing logs and more time delivering value. Dive in today and experience the power of an observability platform that was built for the future of IT.

If you’re using Dynatrace SaaS with a DPS contract, all the features mentioned in this blog post are available to you. If you’re not, why not start a free trial today and experience the value yourself?

Resources

Dynatrace University – Free training that covers everything from basic log ingestion to advanced analytics.

Dynatrace Playground – Our free sandbox tenant with sample log files, ready to explore log

State of Log Management 2026 – Download the report to explore benchmark data on how AI workloads are exploding log volume and costs, and why unified observability is now essential for reliable, trustworthy AI.

The post How Dynatrace supercharged log observability in 2025 appeared first on Dynatrace news.

Yet many organizations still monitor their IBM z/OS environments with tools that were never built for hybrid, distributed systems, let alone for integrating real-time mainframe insights into solutions like ServiceNow, PagerDuty, or Slack.

This missed opportunity results in a visibility gap: teams are aware when something is slow, but they often don’t know where or why the issue has occurred before it impacts their customers.

In a recent webinar, Provinzial, one of Germany’s largest public insurers, shared how it modernized its mainframe monitoring by upgrading to Dynatrace SaaS, unlocking advanced monitoring capabilities.

In this blog post, we’ll explore why customers like Provinzial choose Dynatrace to complement their traditional mainframe monitoring tool stack.

Why modern observability needs to seamlessly integrate with existing mainframe monitoring tools

IBM Z is foundational! They process millions of transactions per day. But most traditional monitoring tools, and even observability solutions, isolate mainframe deployments from the rest of the IT landscape.

Without true end-to-end visibility and context, troubleshooting becomes expensive, time-consuming, and dependent on scarce z/OS experts.

Dynatrace closed this gap for Provinzial by bringing all mainframe telemetry—regardless of how it is collected—into a unified, AI-powered observability platform that allows seamless and cross-functional collaboration.

Davis® AI helps detect performance issues by automatically identifying anomalies like latency increases, failure spikes, or DB2 bottlenecks, assisting teams in resolving issues proactively, before they cause an outage.

Three ways to integrate your mainframe with Dynatrace

Dynatrace supports three complementary integration methods. Whether your organization uses Dynatrace OneAgent®, OpenTelemetry, IBM-native z/OS tooling, Cribl, or all of the above to collect and ingest data into Dynatrace, the outcome is the same:
All signals are automatically correlated, analyzed, and contextualized.

This freedom of choice allows enterprises to adopt the approach that best fits their operational model, security requirements, and hosting constraints, utilizing OpenPipeline® for ingest processing and data enrichment, which handles all sources equally.

1. OneAgent-based monitoring for CICS, IMS, and Java

Dynatrace z/OS agents provide deep, real-time insights into:

• CICS and IMS transaction behavior
• DB2 workload details
• LPAR and individual processor utilization (GCP and zIIP)
• Automatic log collection and correlation with traces

This creates true end-to-end visibility. When a customer books a trip or completes a payment, Dynatrace captures the full execution path—from the mobile frontend through distributed services to CICS or IMS—highlighting exactly where time is spent and where issues originate. Say goodbye to manual correlation and tool sprawl.

2. OpenTelemetry Collector

When full z/OS agent coverage isn’t possible, or if your organization utilizes open standards, the Dynatrace OpenTelemetry Collector (or any other shipper) provides a flexible solution.

Telemetry is forwarded to Dynatrace via OTLP or the existing transfer mechanisms mentioned. Ingest processing with Dynatrace OpenPipeline allows for transforming and extracting metrics or business events, before signals are retained in Dynatrace Grail® for unified analytics, where data is always accessible, with long-term retention up to 10 years.

3. REXX Framework + IBM z/OS Client Web Enablement Toolkit

For environments with established REXX automation, the fastest path to integration is the IBM z/OS Client Web Enablement Toolkit, available with z/OS 2.5 and later.

This cost-effective approach teams to send events, metrics, and logs directly to Dynatrace via the REST API in essentially no time, as Provinzial shared.

Provinzial chose this method to forward and collect alarms from their BMC AMI Ops mainframe monitor related to business-critical online and batch processes and STCs. The implementation was fast, required no additional firewall permissions, and integrated seamlessly into the existing operational model of their outsourcing and hosting partner.

Real-time alerts and integration into your IT ticket system

Once mainframe telemetry is in Dynatrace, it becomes part of holistic, AI-driven observability:

• Mainframe events trigger real-time alerts.
• Dynatrace Davis® AI correlates all signals holistically.
• Problems include root cause context from both z/OS and distributed systems.
• Teams can move from reactive to preventive operations by integrating with BMC, ServiceNow, Jira ticket creation, or other integrations and custom Workflows.

Provinzial leverages Dynatrace to observe its entire environment, including mainframes. This allows them to consolidate 10 other monitoring tools, while interconnecting and automating IT ticket creation with BMC SmartIT.

The outcome? Reduced complexity, accelerated time to insights, and shorter mean time to resolution (MTTR)

A modern, flexible path to mainframe observability

What makes Dynatrace especially compelling is that you don’t need to replace existing tools or redesign mainframe processes. Instead, Dynatrace integrates seamlessly with:

• Existing AMI Ops monitoring
• IBM Z Automation
• OpenTelemetry instrumentation
• Log forwarders and SMF pipelines
• Custom REXX routines

This allows modern observability without disrupting mission-critical systems  while further enhancing IBM Z mainframe agility.

The future of mainframe observability with Dynatrace

With extended OpenTelemetry support for CICS 6.3, TPF, App Connect Enterprise, and other platforms, organizations can adopt standardized telemetry across their mainframe workloads.

Dynatrace is ready for this future today—processing any mentioned telemetry, SMF records, VSAM statements, or DB2 queries all in a unified, AI-powered platform that helps you to understand logs, database execution plans, or incidents in human-readable language.

Provinzial’s journey demonstrates how Dynatrace SaaS can modernize monitoring without introducing complexity.

By combining AI-driven analytics, open integration options, and seamless cross-platform correlation, Dynatrace allows organizations to

• Leverage hybrid systems with confidence.
• Increase efficiency by automating tasks and actions.
• Reduce time spent on incidents with insights and preventive operations.

Get started

No matter where you are in your observability journey, Dynatrace offers the simplicity, depth, and flexibility required to operate mainframe environments at enterprise scale.

Hungry for more? Ready to significantly improve your mainframe availability by connecting logs with traces? Or is your first step with Dynatrace to analyze query performance: the next level of database optimization?

If you’re interested in learning more details about how Provincial reduced costs and increased service quality, visit our customer success story.

If you aren’t yet a Dynatrace SaaS customer, start a free trial or request a demo to experience the power of true observability yourself.

The post Reimagine IBM Z Observability: How Provinzial modernized z/OS Monitoring with Dynatrace SaaS appeared first on Dynatrace news.

As onboarding and deployment become faster, self-service and automation have become a requirement; more than ever before, costs must remain predictable, attributable, and easy to report.

If you’re working to make log spend visible and fair across teams, you’re not alone—this is a common challenge in modern, and cloud native environments.

Assign cost centers and products

Dynatrace allows precise cost allocation for logs, so you can attribute log ingestion and retention to the right cost centers and cost products. This makes internal showback and chargeback straightforward.

Why this matters

• Cloud-native apps and microservices generate log sources rapidly, while shared platforms can blur ownership. Cost allocation brings clarity.
• Teams want autonomy and instant access, without surprises. Build accountability and trust with simplified and automated cost attribution.
• Service providers require accurate reporting for budgeting, audits, and governance purposes. Cost allocation makes it predictable and repeatable, with more than 60,000 cost allocation combinations out of the box and more available for our large enterprise customers, who are adopting this feature today at scale.

Cost optimization is a team sport

The era of budgets and cost optimization being a concern solely left to finance departments is a thing of the past. Teams are expected to own their budgets. Meaning that cost optimization is not a separate accounting artifact, but rather a shared accountability that each team is expected to contribute to.

Whether your teams offer services, applications, or infrastructure, they will want to leverage logs. Team-level accountability for log management begins with allocating log spend to individual products, owners, or any other method your FinOps practice uses for tracking.

With cost allocation for logs, you can take the non-disruptive route

You can leverage the established and defined annotations and labels of the source, for instance, directly from Kubernetes.

But there might be reasons you want to make that attribution at the processing stage:

• Your source might not be capable of providing annotations and tags.
• You don’t have the resources to configure each source individually to match attribution.
• You might want to take a centralized approach, rather than contacting each team individually.
• Your cost allocation requirements are too complex and require a script or a processing technology.

In Dynatrace OpenPipeline®, you can enrich your logs during processing. What may be tedious manual work elsewhere is now centralized and automated.

Whatever your requirements and expectations are, whether you need simple tagging or complex attribution rules, OpenPipeline is here to help.

Because cost optimization is a team sport, the output aligns perfectly with the most common FinOps formats, providing the exact granularity necessary to support enterprise-wide optimization initiatives.

What’s new with cost allocation

• Billing usage events for logs can now be enriched with dt.cost.costcenter and dt.cost.product.
• Attribution of cost centers and products should best take place at source, but can be dynamically processed with OpenPipeline.
• You can mix and match both attributes or use them individually. This allows you to allocate costs by business unit and product/service, allowing for granular chargeback and showback.

Example: Chargeback and showback with Dynatrace cost attributes

Many organizations use chargebacks to create accountability and transparency for shared costs by charging internal departments for the resources or services they consume, based on actual usage. An effective way to implement this with Dynatrace is to use the dt.cost.costcenter and dt.cost.product attributes together.

Consider a scenario where a central IT team provides observability services to multiple business units, such as Retail, Corporate Banking, and Wealth Management. Each unit runs several applications that generate logs through ingestion channels, such as OneAgent®, Log Ingest API, or OpenTelemetry integrations. To ensure accurate cost attribution, the IT team configures these log sources to automatically enrich each log with the appropriate cost center and product identifiers.

For example, logs generated by the Retail unit’s mobile banking app are enriched with dt.cost.costcenter: retail and dt.cost.product: mobile-app. This dual-tagging approach allows the central IT team to allocate log-related costs to the correct business unit and break down those costs by specific products or services within that unit. When billing usage events are enriched with these attributes, Finance teams can apply direct chargeback methods.

Optimize log spend with granular showback

Using the same attribution, IT teams can generate detailed reports showing how much each cost center is spending on log ingestion and retention, as well as which products drive that spend. These reports can be flexibly incorporated with other costs attributed to the same owners or products, such as query costs, using Lookup data in Grail®.

Now consider that same Retail unit. The granular attribution shows that the mobile app is responsible for 70% of its log spend. The team can now take targeted actions. For instance, it can reduce log verbosity in non-critical flows or adjust retention policies to optimize costs.

Meanwhile, central IT maintains full transparency and control over the shared observability platform. This centrally operated, data-driven chargeback model allows teams to operate autonomously without disruption, while aligning with FinOps principles.

Getting to the numbers

You can report and analyze cost allocation in multiple ways, depending on your audience, business requirements, workflows, and the tools you have.

• Dashboards: Crafting individual dashboards to visualize log ingestion and retention by cost center and product is one of Dynatrace’s key strengths. Individual filters, views, and visuals allow you to slice and dice custom dashboards for your teams.
• Notebooks: Explore and validate enriched billing usage events alongside Grail data for deeper analysis or ad‑hoc investigations. This route allows admins to align consumption data with log query insights of users in a shareable manner, as the results are stored. Non-admin users can view the results this way when the Notebook is shared with them.
• Account Management portal: Create cost management reports to track accrued costs and perform showback/chargeback at scale across business units and products, sent by email and downloadable in CSV format.
• Lookup data: Some organizations may prefer using lookup tables to allocate costs to their owners or products. This is a good fit for customers who already work with organizational structures that link owners with product and their respective cost centers. It can also serve as an additional support to track queries in your environments. Learn more about Lookup data in Grail.

With these views, IT and Finance can align on the same source of truth, driving targeted optimizations such as adjusting log verbosity in non-critical flows or tuning retention policies, while maintaining shared platform governance.

Get started: a guide for cost allocation

Let’s recap the best practices to get started successfully:

1. Inventory your log ingest channels and sources (OneAgent, API, OpenTelemetry, cloud/hyperscaler forwarders, log shippers).
2. Define attributes and assign labels for dt.cost.costcenter and dt.cost.product at the source before ingestion, for example, in Kubernetes, OneAgent, or the API and OpenTelemetry configuration of your apps and services.
3. If updating agents or code changes aren’t feasible, define OpenPipeline rules to enrich during the process.
4. Send sample data, verify attributes configured in Grail, and confirm visibility using the Logs app or your existing dashboards.
5. Iterate by team/product, expand coverage, and standardize reporting in the Account Management portal.

New to Log Management & Analytics in Dynatrace?

Ready to make log costs clear, fair, and easy to report? Define your attributes, turn on enrichment, and give your teams the accountability and insights they need—without slowing them down.

If you’re already using Dynatrace Platform Subscription (DPS), you can instantly get started with logs today! Additional resources and downloads are available in our community examples space on GitHub.

We invite you to explore our Dynatrace Playground tenant at no cost or to start a free trial to ingest your first logs with cost allocation.

The post Cost allocation for logs: Precise, flexible, and non-disruptive appeared first on Dynatrace news.

Dynatrace has redefined what’s possible in log analytics with support for single log events up to 10 MB, setting a new standard in log observability. This eliminates manual workarounds and allows full-fidelity log ingestion at enterprise scale for customers moving their log management and analytics to Dynatrace.

In this blog post, we will examine specific customer examples from different verticals across aviation, retail, cargo, and beyond and examine how they’ve increased the simplicity and reliability of their use cases using Dynatrace.

Large log records are business-critical

While not all businesses generate large log records, those that do consider them operational gold, containing tremendous additional value that can be unlocked. In industries like cargo and transportation, healthcare, or autonomous vehicles, individual events and resulting log records can range in size from a few hundred kilobytes to several megabytes. These records carry essential diagnostic signals, event telemetry, and traces to address compliance requirements.

Yet many traditional log management vendors struggle with records larger than a few MB or require additional manual scaling configuration, scripting skills, or custom tooling upgrades to support enhanced ingestion or increased record retention limits.

With Dynatrace, it’s zero-config and effortless: You can ingest single log records natively up to 10 MB, ensuring full context and accurate analysis. You can take immediate action without trimming, truncation, or fear of ingest failures caused by custom scripts.

Real-world use cases: Built for enterprise workloads at scale

Use case #1: European airline group – Passenger-handling log trails

The aviation industry shares a standard for passenger handling and boarding trails, such as Passenger Name Records (PNR). In some systems, such events, which are crucial to daily operations, can appear as single large log entries and can easily reach sizes larger than 3 MB. In addition, they hold business observability telemetry, which can offer unconsidered value you can unlock using Dynatrace Business Observability.

Previously, our aviation customer used the ELK stack, which required custom tools to ingest and correlate these logs, due to the limitation of 3 MB log event support.

With Dynatrace, logs are ingested seamlessly through OneAgent® and OpenTelemetry (OTel) with no difference in outcome. This way, you can easily scale up and scale out at any time. No pre-/post-processing of large log events is required, and events are available across many Dynatrace® Apps and use cases within the Dynatrace platform.

This tremendously accelerates forensics, reduces operational overhead, and improves business observability or compliance reviews.

Mastering logs with Dynatrace: Unlock log insights without queries (12 min video)

Use case #2: Leading European telecom provider

For telecom providers, retail sales of mobile phones are tied to even more metadata, while business processes are linked to SLA fulfillment. When customers order a mobile phone, the whole lifecycle of delivery, contract activation, SIM card assignment, and unlocking occurs in the provider network.

In this European telecom provider’s previous solution, ingestion failed unless XML tracing log events were split, as the trail event size could easily be larger than 5 MB. Faced with log ingest limits, splitting logs was the only option, resulting in disconnected events and increased manual correlation efforts.

With Dynatrace, full payloads are ingested and visualized with automatic JSON structure recognition and parsing. This allows precise and simple yet powerful filtering in the GUI, with the Dynatrace Query Language (DQL), and Dynatrace Apps, such as Dashboards, Workflow automation, and more.

Download our free eBook to learn how Dynatrace helps its retail customers.

Use case #3: National rail operator

Previously, our customer had to tail and process log event records from their signaling systems using shell scripts. Also, their legacy monitoring system ingested monitoring events into a SQL database, from which our customer had to tail/parse the recent log records. Creating correlation and context between these wasn’t easy.

With Dynatrace, our customer now leverages OneAgent to automatically collect all log records, signals, and metrics. With OneAgent observing their SQL server, they have entered the next level of database observability.

This provides them with the capability to extract events directly from the SQL database, centrally configured on the Dynatrace platform, using the simple yet powerful Dynatrace custom extension creator. If your requirements are more complex than these, a custom extension based on the Dynatrace SQL extension, as shown in the video linked below, will help you address your use cases.

https://youtube.com/watch?v=t9kHPKL9yKY%3Fsi%3DXdlpvJQ3-3KaNN9s

A Practical Guide to SQL Extensions with Dynatrace (53 min video)

Engineers now see full context and experience true observability, without relying on or maintaining fragile ingest logic in custom bash scripts.

Use case #4: Software engineering giant

Our customer faced persistent data loss due to truncation with their previous tool, which occasionally resulted in inconsistent formatting and missing fields. Therefore, they were blocked from accessing certain events, metrics, and meaningful analysis.

With Dynatrace, large JSON log records are parsed as a single record and presented as full events. In addition, our customer can extract metrics and fields from logs during ingestion, and even drop irrelevant records after extraction.

The Dynatrace platform and its apps provide structured views, allowing developers to inspect, filter, and find data in any field/column using the app-specific filter options or performing deep search data exploration.

Dynatrace provides SREs with live forecasting and baselining capabilities that don’t rely on predefined metrics but can use any raw data, such as historic logs, in case an ad-hoc request arises. This is quite common in software engineering when debugging new code or identifying new conditions for setting up alerts or remediation workflows.

Unmatched experience with the Logs app

The Dynatrace Logs app makes working with large, complex logs easier than ever:

• Visual JSON parsing renders nested data in structured, searchable log row results and single events.
• Click-to-filter exploration allows real-time root cause discovery with filter-in and filter-out features.
• Filter → DQL conversion turns applied log filters into Dynatrace Query Language—a true unified and platform-wide query language that addresses your needs and can be used for alerts, dashboards, and automation.

Filters with segments allow you and your teams to predefine reusable views that dynamically filter results by region, cluster, or other dynamic values relevant to your business—cut down search times and promote consistency across departments and your entire organization.

From triage to transformation

These capabilities radically simplify life for site reliability engineers:

• ✅ No more log truncation or splitting
• ✅ No more ingest failure troubleshooting
• ✅ No more duct-taped parsing and log correlation

Instead, SREs get full-fidelity, contextual AI assistance and a unified platform experience across logs, metrics, and traces—all within a single platform.

Ready to go big?

Stop fighting your log tools and start scaling with Dynatrace today:

• Start your free trial and ingest your logs using OneAgent, API, OTLP, Cribl, Fluent, Syslog, Journald, or any other method you choose.
• Alternatively, request a live demo for a tailored walkthrough.

© 2025 Dynatrace LLC

Dynatrace and the Dynatrace logo, are trademarks of the Dynatrace, Inc. group of companies. All other trademarks are the property of their respective owners.

The post Go big with Dynatrace: Native support for large log records appeared first on Dynatrace news.

In this blog post, you’ll learn how Dynatrace OneAgent® automatically identifies Journald and ingests structured logs into Dynatrace while enriching them with topology and infrastructure context.

Why migrate from Syslog to Journald

Journald provides a more modern alternative that addresses the limitations of existing Syslog implementations. It offers structured logging, fast indexing for search, access controls, and signed messages. Thanks to its structured and binary format, Journald is quick and efficient.

Journald provides unified structured logging for systems, services, and applications, eliminating the need for custom parsing for severity or details. It supports multi-line logs, handles log rotation, and even includes mechanisms to check for data corruption.

Overcoming native Journald limitations with Dynatrace

Journald doesn’t natively support easy log collection and centralization. Unlike rsyslog, which requires minimal configuration for centralization, Journald’s approach, using systemd-based operating systems, necessitates more advanced configurations and additional components.

Dynatrace OneAgent solves this problem by effortlessly detecting and centralizing logs while providing unmatched flexibility, thanks to the Dynatrace platform’s centralized and dynamic ingest rule controls.

Dynatrace comes with log ingest rules, including a built-in Journald log rule. When activated, this rule facilitates the OneAgent Journald log detector’s collection and ingestion of Journald-related logs.

The configuration is managed centrally within Dynatrace settings (Settings > Log ingest > Log ingest rules).

The Dynatrace platform and settings also allow you to define custom rules, such as defining which Journald units and logs should additionally be included or excluded.

Why unified observability boosts productivity

While journalctl is a powerful local tool with local filtering capabilities, it doesn’t scale well, especially considering the globally distributed components of today’s hybrid/cloud-hosted environments. When using Dynatrace, in addition to automatic log collection, you gain full infrastructure context and access to powerful, advanced log analytics tools such as the Logs, Notebooks, and Dashboards apps. For forensic log analytics use cases, the Security Investigator app benefits from the scalability and analytics power of Dynatrace Grail™.

Grail, the Dynatrace schema on-read data lakehouse, is at the heart of the Dynatrace platform. It provides unified observability by automatically correlating logs and placing them in the context of traces and metrics.

Unlike traditional setups that require predefined schemas, Grail allows you to store diverse data types without schema definitions at any point, providing greater flexibility in any analytic situation.

The Grail architecture ensures scalability, making log data accessible for detailed analysis regardless of volume. With real-time analysis, you gain faster data-driven decisions and simplified data ingestion. Dynatrace Grail lets you focus on extracting insights rather than managing complex schemas or index and storage concepts.

Davis® AI works with Grail as part of the Dynatrace platform’s integrated automation and AI capabilities.

System health, performance troubleshooting, and debugging situations no longer require manual correlation of logs across multiple disconnected tools or servers. Thanks to Davis AI’s automatic problem detection capabilities with logs in the context of entities, services, applications, and traces, your root cause analysis efforts are greatly reduced.

Why use Dynatrace OneAgent for Journald

Dynatrace OneAgent automatically ingests logs from Journald, eliminating the need for manual configuration and ensuring comprehensive log coverage while offering centralized and granular rules for log collection. In addition, OneAgent handles log rotation, decompression, and parsing automatically, reducing the overhead associated with manual log management.

With built-in log enrichment, OneAgent adds valuable attributes such as trace and span IDs to log records, facilitating deeper analysis and correlation with distributed traces. Journald logs are integrated into the Dynatrace platform this way, providing a unified view of system and application logs alongside other observability data.

Since you can ingest Journald logs using Dynatrace OneAgent, this feature is available to customers of both the latest Dynatrace SaaS and Dynatrace Managed.

Start collecting Journald logs with Dynatrace today

If you haven’t started collecting Journald logs with Dynatrace, install OneAgent on your systemd-based operating systems and consult Dynatrace Documentation for additional details if you want to configure custom rules to include or exclude specific Journald logs.

The post How to observe logs with Journald and Dynatrace appeared first on Dynatrace news.

Fast and efficient log analysis is critical in today’s data-driven IT environments. For enterprises managing complex systems and vast datasets using traditional log management tools, finding specific log entries quickly and efficiently can feel like searching for a needle in a haystack.

Dynatrace segments simplify and streamline data organization in large and complex IT environments, providing pre-scoped data without compromising performance.

In addition to adding segments, we’ve overhauled the filter capabilities of the Logs app, introducing a new advanced filter bar.

In this blog post, we’ll explore how these features boost productivity and accelerate access to the right data sets using shortcuts like segments. We’ll also take a closer look at how the improved filter capabilities can be combined to differentiate segments.

What are Dynatrace Segments?

Segments are reusable, pre-defined filter conditions powered by the Dynatrace Query Language (DQL). Segments can implement variables to dynamically provide, for example, a list of entities to users, such as available Kubernetes clusters, for unmatched flexibility and dynamic segmentation.

Segments focus on specific, relevant data sets across apps and data types as they’re available and applied across the Dynatrace platform. Think of segments as a feature that allows you to create views for teams, departments, or application owners—or just for yourself. With segments, you can isolate particular OpenPipeline™ log sources, resource entities, cloud regions, or even certain buckets your developers use.

Segments allow you to provide a platform-wide pre-filtered and restricted data set on which you can execute and refine additional queries using the advanced filter bar.

Here’s what Dynatrace customers love about using segments in their daily work:

• Time savings with reusability
  Segments save you from manually re-applying filters for every new task, or when switching between Dynatrace Apps. For example, a segment for Service Errors in Azure Region can be applied instantly by selecting it from the dropdown.
• Dynamic and flexible conditions
  Segments can leverage variables (e.g., $bucket, $region, $entity_name) to adjust the context dynamically. This eliminates the need to create multiple static segments or filters for varying conditions using the filter bar.

For example, the Service Errors in Azure Region segments can provide a dynamic list of available regions instead of creating multiple fixed region segments.

Unsure if your team would rather use unique IDs or region names? Don’t worry; with the power of the DQL commands like concat you can easily address this for your users with a single line of code.

Simplified collaboration

Individual users and teams can share segments to ensure consistent filtering logic across apps, dashboards, or even business observability use cases. This fosters collaboration and alignment across departments and teams.

Optimized query performance

Segments narrow the available data scope in real time, improving query speed, reducing overhead, and helping to optimize consumption.

Use case from the field: Data segmentation and filtering Kubernetes cluster logs

Imagine your team frequently troubleshoots errors from specific Kubernetes deployments within your production environment. Instead of defining and applying filters manually each time you launch the Logs app, you can now create segments with predefined filters for:

• Environment: Create and define a segment to narrow the scope to your Production environment using specific cloud subscriptions or tags.
• AWS region: Using a variable, you can dynamically limit the scope to AWS regions selected from the dropdown.
• Cluster: Instead of combining the K8s cluster as an additional DQL filter statement within the AWS region segment, you can create another segment for the Kubernetes cluster.

The obvious benefit is that you can reuse the Kubernetes segment and combine and apply it independently or with other segments and filters.

• Log level: When you and your team only want to see status = error messages for the clusters in question, you can add this to the segment’s DQL statement.

It’s easy to apply the status level as a dynamically applied filter using the filter bar or by selecting the status levels within the histogram.

If you want to learn more about the required steps, Dynatrace Documentation provides a detailed step-by-step guide to getting started with segments and logs.

Advanced filter field: Precision meets simplicity

Dynatrace offers an improved filter bar with powerful operators and dynamic suggestions for available entities. The filter field and entity value suggestions allow you to refine your search precisely, building even the most complex queries quickly.

Why Dynatrace customers love the new filter field

Logical operators (AND, OR, NOT)

You can combine multiple conditions with Boolean logic for complex queries.

For example, if you operate Kubernetes clusters in multiple AWS regions, but don’t want to see the US-based regions:
loglevel = "ERROR" AND log.source = "kubernetes" AND aws.region NOT IN ("us-east-1a","us-west-1b")

Dynamic suggestions

The filter bar provides real-time suggestions for entity parameters such as Kubernetes nodes, virtual machine names, or log pipelines provided by OpenPipeline.

These filters can be added by typing or selecting a field and choosing whether to include or exclude it.

Entity searches

The namespaces of Kubernetes clusters and nodes can be quite lengthy. In addition to automatically suggesting results, the filter bar will suggest a list of results while also acting as a wildcard operator, so you can match multiple entities with a single query.

Field-specific operators

You can use advanced matching options, such as for tags:

host.tag in ("production", "critical", "region-x")

Use case from the Field: Filter field in action

Suppose you’re investigating an issue in a Kubernetes cluster in a dev-staging environment. The dev-staging cluster isn’t monitored regularly or included in an existing segment.

Watch this scenario in action.


^{Figure 5. The filter bar provides instant precise suggestions, helping you find relevant entities and resources and boosting productivity.}

Combining filter fields with suggestions of available entities and values saves time and ensures accuracy in even the most detailed analyses.

Segments vs. filters: Finding the perfect balance

While both segments and filter fields help streamline log analysis, they serve distinct purposes:

Segments are usually pre-defined by power users or Dynatrace administrators and shared across teams. While any individual user can build segments, not all users have permission to share segments.

Segments provide reusable, pre-defined views that can be applied ad hoc when running queries in the Logs app. They’re available across all applications on the Dynatrace platform. On the other hand, individual users apply the Filter field on demand when they model queries.

As a best practice, you can combine segments to initially narrow the search scope to a bucket, cloud platform, region, or cluster, and then use the filter bar to run a focused query within that sliced data set.

Conclusion: Simplified, focused, and faster log analysis

Dynatrace segments and advanced filter fields bring unparalleled efficiency to log analysis while enabling an even broader set of users to make meaningful and powerful use of your data.

Segments help you ensure data consistency across teams and departments, all backed by DQL and Grail.

By enabling predefined, reusable views combined with granular real-time queries, these features empower teams to:

• Spend less time configuring filters when they use the Logs app.
• Focus on identifying and solving critical data faster.
• Ensure consistent analysis across use cases and departments.

Ready to transform how you work with log data?

Start using segments and filters in Dynatrace today to experience faster, smarter, and more collaborative log analysis.

For a deeper dive into these features, visit Dynatrace Documentation.

The post Log filtering made easy: Data segmentation and advanced filters in Dynatrace Logs appeared first on Dynatrace news.

Log ingestion can seem daunting when getting started with Dynatrace, especially when staring at an empty screen in the Logs or Clouds apps.

To address any concerns you and your new-to-Dynatrace teams may have regarding log ingestion, we’ve introduced a new and intuitive log onboarding wizard for Dynatrace® Apps.

The newly introduced step-by-step guidance streamlines the process, while quick data flow validation accelerates the onboarding experience even for power users.

This blog post explains how Dynatrace simplifies log ingestion, whether you’re onboarding logs from your infrastructure using OneAgent®, cloud services using log forwarding, or driving open-source standardization leveraging OpenTelemetry (OTel), Fluent Bit, or any other API-based ingestion methods.

Get started with Logs: The OneAgent advantage

For most scenarios, Dynatrace OneAgent is your best friend for getting started with Dynatrace log ingestion. OneAgent is flexible and dynamic, adapting to new requirements by configuring rule sets centrally within the Dynatrace platform.

A cornerstone of Dynatrace monitoring capabilities, OneAgent boosts your log ingestion experience by automatically detecting and tagging logs based on the detected process technology—even for custom-developed applications.

1. Step-by-step setup

The log ingestion wizard guides you through the prerequisites and provides ready-to-use command examples to start the installation process. This ensures that logs flow into Dynatrace within minutes.

This includes the creation of API access tokens, given the required permissions.

The pre-defined monitoring mode settings, for example, Full-Stack, are pre-selected following your platform administrator’s guidelines. Configuration is fully customizable.

2. Automatic log discovery

Once installed in an operating system or Kubernetes cluster host, OneAgent automatically detects logs from your deployment and ingests them into Dynatrace without requiring additional local configurations. During this process, OneAgent detects and links technologies, such as Java, Docker, or Microsoft IIS, for improved parsing and log analysis.

Tagging is also available when using API-based ingestion methods or later within the platform.
In any case, this ensures that your logs are immediately available for quicker filtering and grouping, and the readability of these log lines is improved.

After successfully installing OneAgent, the log ingestion wizard provides a host selector drop-down to validate the data flow.

3. Preconfigured log rules

Dynatrace provides a set of prepared log ingestion rules, so you don’t need to create custom configurations for common technologies and services. You can refine and modify these preconfigured rules to ingest relevant logs, mask sensitive data, and use advanced options—all configured centrally at scale.

While OneAgent automatically detects technologies and tags log lines accordingly, you can also manually configure OpenPipeline™ with processing rules for other log ingestion methods, such as OpenTelemetry (OTel), Fluent Bit, and any other API-based log ingestion method supported by Dynatrace OpenPipeline.

Hyperscalers and cloud platforms: Effortless log integration

Log ingestion is equally straightforward in cloud environments like AWS, Azure, and GCP.

The log ingestion wizard provides step-by-step instructions like you’ve just seen for OneAgent but also highlights the benefits of each integration, supporting you with additional resources to streamline your real-time log forwarding setup.

One of the main differences between OneAgent and public cloud platforms is that cloud platforms offer differing methods of log forwarding, streaming, and collection. Depending on your current setup and needs, you may want to choose one platform or another.

Following the instructions and recommendations ensures that your logs are available within a few minutes, just like with OneAgent, Otel, and other API-based ingestion methods.

Log forwarding and API-based ingestion: OpenTelemetry, Fluent Bit, and APIs

If you’re already using OpenTelemetry, Fluent Bit, or another log-forwarding solution, integrating with Dynatrace is effortless.

The API-based approach is the most flexible. It’s designed for users familiar with open source standardization looking for support for OpenTelemetry semantic, granular, and advanced log pipeline management capabilities paired with token-based API authentication.

In addition, the Dynatrace Log ingestion API supports additional use cases not covered by the aforementioned methods, such as Edge Computing, IoT, or Point-of-Sales (PoS) use cases.

With any or all of these methods simultaneously, you can build and customize your instrumentation precisely how you need it and benefit from Dynatrace enterprise-ready features without sacrificing open source principles. Using Dynatrace, you can maintain control and increase flexibility in your instrumentation choices.

Why log management is easier with Dynatrace

Dynatrace is the ideal enterprise-wide observability and security platform, as it supports your current and future cloud-native log ingestion methods.

The new log onboarding process is designed with simplicity, scalability, and user experience in mind—for novice users and experts. It provides the same accelerated experience regardless of whether you use OTel, OneAgent, or API-based ingestion.

The onboarding process allows you to create required access tokens, reuse existing tokens, and validate the data flow of selected entities, pipelines, and API tokens.

The post Simplify log onboarding: From zero to observability in minutes appeared first on Dynatrace news.

As cloud complexity increases and security concerns mount, organizations need log analytics to discover and investigate issues and gain critical business intelligence. But exploring the breadth of log analytics scenarios with most log vendors often results in unexpectedly high monthly log bills and aggressive year-over-year costs. To give organizations the freedom to explore log analytics without barriers due to cost concerns, Dynatrace is proud to announce a new Dynatrace Platform Subscription (DPS) pricing model option called Retain with Included Queries.

With this new DPS pricing model option, customers can retain data at a fixed low cost with no additional cost to query for up to 35 days. This model provides a predictable way for customers to manage and analyze logs, drive log management tool consolidation, and reduce costs while gaining maximum value from their log data.

Based on customer feedback, we’re offering the Retain with Included Queries pricing model as an alternative to our existing usage-based plan. Both plans offer the same low ingest price. However, the new all-access plan combines retention and queries into one low price to simplify scoping and budgeting.

Retain with Included-Query pricing simplifies forecasting and annual usage calculation costs. Customers who choose this pricing option get:

• Retention cost: $0.02 per GiB per day
• No cost to query for up to 35 days
• Ingest cost: $0.20 per GiB ingested (no change)

With this approach, the whole team can leverage the power of Grail queries and dashboards without worrying about limiting query usage. Customers can configure the Retain with Included Queries option with retention periods ranging from 10 to 35 days. Customers requiring longer retention periods should opt for our existing usage-based pricing, which supports retention for up to 10 years.

Queries are included

• Predictable pricing: If you know the number of logs you ingest daily, then you’ll know roughly your total annual cost upfront, providing peace of mind and less managerial overhead.
• Simple scoping: Remove the complexity associated with predicting query search volumes. Realize cost savings immediately for high-query usage scenarios. Get started quickly!
• No cost management required: Once your configured retention period ends (a maximum of 35 days), logs are automatically deleted. No oversight is needed over query usage.

Usage-based pricing is still an option

Over time, our existing usage-based pricing is the more cost-optimized option, as you only pay for the queries your users execute, and you benefit from the competitive $0.0007 per GiB per day to retain logs for up to 10 years. Queries are charged at $0.0035 per GiB scanned. Usage-based pricing is ideal for organizations with longer retention requirements and known query patterns. This pricing flexibility allows customers to optimize their log analysis expenses by paying only for what they use.

Cost-efficient:

• Lowest upfront cost
• Charges are strictly based on query execution

Scalability:

• Ideal for businesses with varying query demands
• Adapt dynamically to usage patterns

Retention:

• Supports log storage from 1 day to 10 years
• Optimal for longer-term log analytics needs

Guidance on using both plans

The Retain with Included Queries pricing option is a great way to get started while you learn about your query usage. Dynatrace includes a ready-made cost dashboard that provides insights into query usage and DQL best practices. Once you develop best practices and are confident with your consumption patterns, you can switch to usage-based pricing to maximize the value of your DPS investment.

Innovations on the horizon*

We’re very excited about our new Retain with Included Queries pricing, but we expect to deliver more updates. This pricing model is part of our plan to introduce new features that help customers align the right pricing strategies to their use cases. With these features, customers can easily see, manage, and choose how to align the Retain with Included Queries pricing with the usage-based pricing model.

Customers will soon be able to mix and match log pricing options on a per-bucket basis and provide users with access to both models simultaneously. This flexibility will allow customers to optimize the pricing selection based on the anticipated use case associated with each bucket, yielding even greater savings and value.

Retain with Included Queries: Start here

With the Retain with Included Queries pricing model, Dynatrace now offers a more cost-effective way to get started with log analytics. Drive efficiency and get more value out your logs with this predictable pricing model while you’re building your log analytics practices.

* Disclaimer: This publication may include references to the planned testing, release, and/or availability of Dynatrace products and services. The information provided in this publication is for informational purposes only; its contents are subject to change without notice, and it should not be relied on in making a purchasing decision. The information is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for products remains at the sole discretion of Dynatrace

The post Predictable costs for Log Management & Analytics with new simplified licensing plan appeared first on Dynatrace news.

With the recently released version of the Databases app, Dynatrace allows you to monitor your databases from a query perspective:

• Quickly find all heavy queries that consume your database resources
• Monitor your query metrics and resource utilization
• Understand how your queries are executed through the Query Execution Plan

Optimize application performance: The importance of database analysis

Databases are critical for all (business-critical) applications and significantly affect performance. Monitoring the health and performance of your databases is essential to ensuring optimal application functionality. While infrastructure-level monitoring provides valuable insights, it might not reveal the root causes of database-related slowdowns. To address this, deeper analysis at the query level is necessary. By examining individual queries, you can identify database bottlenecks and implement targeted optimizations.

The Dynatrace Databases app introduces essential query-level monitoring features

The Databases app now offers long-awaited functionality for database monitoring at the query level:

Top Queries view lets you quickly identify queries that take a long time to execute or unnecessarily consume precious database resources. Additionally, it allows you to analyze how resource consumption or performance metrics for a given query have changed over time.

Query execution plans offer even more insights into how a database engine executes queries. Understanding what contributes most to query execution time is essential to query optimization. Instead of experimenting with different optimization techniques, deep analysis of the query execution plan makes pinpointing any possible resource conflicts or service degradations easy.

Top Queries and Execution Plans are available in the Databases app. These functionalities are currently supported for the following database engines:

• Microsoft SQL Server
• Oracle
• PostgreSQL
• MySQL
• MariaDB

To start your analysis, open Databases, go to Instances > Top Queries and select the Statement Performance button for a specific database instance.

After identifying a query for analysis, you can look for interesting metrics that measure various conditions, like execution time, CPU consumption, or I/O utilization.

Expanding the query row allows you to analyze query performance over time from various perspectives.

To better understand how the given query is executed and to identify possible optimizations, you can request a query execution plan. Select the Request button and go to the Execution plan tab.

Execution plans provide a roadmap for how the database engine executes queries. With access to execution plans, you can identify performance bottlenecks, such as inefficient joins, missing indexes, and long-running queries, and optimize them for better efficiency and performance.

Execution plans also allow you to understand the cost and resource usage of the various query operations involved, enabling informed decisions about query re-design and indexing strategies. You can ensure that your databases run efficiently, and ultimately improve application performance.

Get started with the Databases app

Start monitoring your databases at the query level with the Dynatrace Databases app. Ensure that your databases are not a bottleneck for your apps and that you’re efficiently using database queries with the new and improved query execution plans.

If you have specific improvements in mind or would like to share feedback with us, please visit our Dynatrace Community feedback channel.

The post Analyze query performance: The next level of database performance optimization appeared first on Dynatrace news.

Dynatrace introduced numerous powerful features to its Infrastructure & Operations app, addressing the emerging requirement for enhanced end-to-end infrastructure observability. The newly introduced Network devices and Details view within Hosts provide comprehensive health status information, relevant networking signals, and machine metrics—all analyzed and provided by the industry-leading combination of Dynatrace Grail™ data lakehouse and Davis^® AI.

These enhancements are designed to empower IT operations and SRE teams with more comprehensive visibility and increased efficiency at any time.

Let’s explore these exciting new features and see how they elevate infrastructure management.

^{Figure 1. Infrastructure & Operations app showing manual root cause and impact analysis}

Advanced filter components for powerful enterprise-grade search and visualization

One of the standout enhancements of the latest update is the improved filter components, designed to facilitate more complex and advanced searches across cloud, on-premises, or hybrid infrastructure hosting environments, including support for custom tags.

Whether you’re looking to zero in on specific hosts, operating system and application processes, or devices, the improved filters enable high-precision searches without sacrificing the operational performance of the Infrastructure & Operations app.

Specifically, background filter queries and visualizations have been optimized to ensure fast response times for complex and large enterprise datasets. A critical update that further enhances this experience is the increase in visual limits—now, up to 100,000 host elements can be displayed within the resulting filtered view.
You can interactively sort, filter, and reorder these host elements with no query delay. This means quicker access to insights, allowing IT teams to act faster during timely and critical incidents.
Whether you’re looking to swiftly review performance trends across thousands of cloud-native resources or your goal is to investigate a single component within your environment, Infrastructure & Operations provides the details you’re looking for. Speaking of single elements and resources brings us to the next enhancement:

Host detail view: Get the complete picture and deep dive into host health

The recently introduced host detail view has elevated manual health reviews and in-depth analysis to the next level. You can now explore related insights and AI-powered analytics within a single host detail view. This view offers DevOps and engineers metrics related to health, security status, service events, and service level objectives (SLOs), paired with quick access to drill deep into logs or process-level insights.

As shown in Figure 3, the host details view is accessible directly from the list of host resources in Infrastructure & Operations and from within the overview pane.

To increase efficiency and shorten operations cycles, the current filtered host detail view can be shared effortlessly by copying the URL in the browser’s address bar. Teams can share the current view via Teams, Slack, or any other messenger and include the current view and findings as part of a support ticket.

In cases where rich details of process and service level consumption are required, the host details provide you with a Processes page focused on utilization metrics and troubleshooting insights.

Technology-type tags can be added at the process and process group levels, splitting the technology tagging for hosts into main and sub-technologies. This simplifies filtering and automation efforts.

When deep monitoring is activated, drilling into the process detail view shows versioning and release information. Custom tags on the application level allow you to filter more easily while simultaneously enhancing automation capabilities.

A new feature makes it easier for IT teams managing large-scale environments to stay on top of every detail across any infrastructure or region. Now, with more granular insights into process performance and technology versions, teams can quickly review and detect anomalies, prioritize problems reported by Davis AI, and maintain SLOs and compliance—all in one place: Infrastructure & Operations.

Host networking metrics and visualization

Network performance bottlenecks are one of the most common troubleshooting challenges in large infrastructures. The new Host Detail view also introduces improved networking insights to streamline diagnostic efforts, including a summarized view of key network metrics, TCP packet transmission statistics, retransmit errors, and packet loss.

These metrics are critical for understanding network health on the application and host level, isolating issues related to network connectivity or connection outages.

This improvement directly helps DevOps and InfraOps teams by reducing the time spent identifying network-related issues in cross-functional efforts.

Network devices view: Extend end-to-end infrastructure observability

In the daily life of DevOps, InfraOps, or SRE, one of the most common routines and troubleshooting challenges in large infrastructures is staying ahead of network performance bottlenecks.

With SD-WAN appliances, firewalls, load balancers, routers, switches, and other network devices in place, this task is becoming quite challenging yet crucial.

Infrastructure & Operations features a newly introduced Networking Devices section, which provides networking visualizations to streamline these efforts within one place.

With extensions available for Cisco, Palo Alto, NetScaler, Juniper, F5, Fortinet, and HAProxy available within Dynatrace Hub, the Network devices section instantly provides value to teams. Existing, configured extensions provide metrics and Davis AI-powered analytics without additional configuration efforts. Networking devices and appliances not yet onboarded can swiftly be integrated via SNMP integration or technology-specific extensions, as highlighted above.

This new capability dramatically simplifies monitoring signals like ethernet interface availability status and network utilization metrics alongside hardware metrics and consumption. Dynatrace Davis AI plays an integral role in this newly introduced section by automatically identifying and raising problems that might otherwise go unnoticed. It enables IT teams to proactively manage network performance bottlenecks or outages quickly, potentially before these issues impact services and users.

Dynatrace: Powering the next level of end-to-end infrastructure observation

The continued integration of Grail™ data lakehouse and Davis AI into Infrastructure & Operations, alongside new features focusing on networking devices, show the Dynatrace commitment to providing IT teams with unmatched and holistic end-to-end infrastructure observability empowered by AI-driven insights.

Together, these technologies enhance the ability of DevOps and InfraOps teams to manage the growing complexity of modern IT environments within a single platform and with a single app—Dynatrace Infrastructure and Operations.

With AI-driven root cause analysis, IT teams no longer need to manually correlate logs, metrics, and traces. Instead, the platform autonomously identifies the root causes of issues, minimizing downtime and freeing up valuable time for more strategic initiatives.

Networking specialists or architects who desire to surface the corresponding log lines are provided with pre-filtered query views, offering service-specific log information and surrounding logs of related components.

Engineers, developers, and power users are provided with the all-new Logs app, which we introduced recently in our blog post Simplicity meets power: Introducing the all-new Dynatrace Logs app.

Beyond the two apps mentioned, Davis CoPilot integration supports novice Dynatrace users by translating questions from natural language into Dynatrace Query Language (DQL) when requests become more sophisticated

Kickstart the new era of AI-powered end-to-end infrastructure observation with Dynatrace today!

Discover how the new and enhanced Infrastructure & Operations app can transform your daily routines and troubleshoot workflows to drive better business outcomes.

Existing Dynatrace customers with licenses that include Infrastructure Observability have automatic access to the new app and all other features available with Dynatrace SaaS; you don’t need to take any action other than to start using Infrastructure & Operations and its features.

Explore the latest updates on the Dynatrace platform by starting a free trial or requesting a live demonstration and taking a custom-tailored tour.

The post Supercharge your end-to-end infrastructure and operations observability experience appeared first on Dynatrace news.

The new Dynatrace Logs app, fully powered by Grail™ data lakehouse, significantly enhances the experience for novice and seasoned users. Logs delivers unparalleled simplicity with powerful, sharable views and insights to address these needs.

Enhanced log ingestion and seamless out-of-the-box integration

The Logs app is supported by comprehensive log ingestion capabilities provided by the Dynatrace platform and Dynatrace Grail while interwoven with other use-case-specific Dynatrace Apps:

While this is just an overview of the commonly used ingest routes and methods you can leverage, the Dynatrace Hub offers the complete set, currently with 500 supported technologies, 150 additional public extensions, and 65 Dynatrace Apps.

Harness log source sprawl

It’s common for large enterprises to provide hundreds of applications to employees and to host more than a dozen critical line-of-business applications. “Digital workers are now demanding IT support to be more proactive,” is a quote from last year’s Gartner Survey

Understandably, a higher number of log sources and exponentially more log lines would overwhelm any DevOps, SRE, or Software Developer working with traditional log monitoring solutions. The Dynatrace logs in context and surrounding logs features ensure that Dynatrace users are provided with a view that’s tailored to their needs and supports their daily tasks.

The Logs app enables Dynatrace users, whether novice or experienced power users, to rapidly filter and investigate results manually within the app or experience a seamless drill-down handover from use-case-specific apps.

Whether it’s cloud applications, infrastructure, or even security events, this capability accelerates time to value by surfacing logs that provide the crucial context of what occurred just before an error line was logged. For example, if one of your customers unexpectedly uploaded a 1 GB file instead of a 1 MB file, was there an error with the buffer overflowing, or was the network stack unable to handle the unexpected load? Even more importantly, how was the error handled, and did the process end successfully for the customer? With Dynatrace and Surrounding logs, answers to such questions are never more than a click away, with no need to write complex queries or patterns.

Simplicity for novice and power users

For users who seek quick access to relevant logs without the need to write complex queries, easy filtering capabilities are available from within individual log line details or by adding and selecting fields in the menu bar.

For those who aspire to become power users, the new in-app DQL editor (Dynatrace Query Language) translates manually selected filters into the DQL code executed in the backend.

Power users can edit and enhance DQL queries to update results, or they can start directly within the DQL editor view. This allows deep-dive analysis when manual investigation or complex queries are required.

As the screenshot above shows, you can transition back to the filter selection menu bar by selecting Back to previous filters or by sharing the query and results with other teams.

This ensures a smooth user experience for DevOps engineers and SREs, whether they prefer intuitive click-and-filter workflows or fine-grained control through DQL.

Shortening time to value and increasing impact

With the general availability (GA) of the Logs app, another enhancement was introduced that shortens the time to value and increases the positive impact made within organizations.

Filtered log views that were customized by click-to-filter or by leveraging the DQL editor can now be shared using direct links—simply copy and paste the link displayed in your browser’s address bar.

This enables you to

• bookmark and share reoccurring or complex queries.
• share situation or incident-specific views across teams.
• provision new Dynatrace users with relevant queries
• share timeframe-specific and pre-filtered views in a support case

See more with less

During the preview release of the Logs app, many customers provided us with valuable feedback that we incorporated into the design of the app.

User requests, such as the newly added line wrap feature, increase the readability of results and eliminate the extra clicks required to open a detail view or load the results within a notebook. This shortens the time it takes to investigate results and take action.

Speaking of doing more with less, more was released simultaneously with the GA of the Logs app:

• Increased performance of query results returning
• Live search within query results
• Live sorting within query results
• custom column arrangement
• enhanced open-with capabilities to open queries in other platform apps,
  or natively within the Logs app (for example, with Notebooks)

Why Dynatrace

The Dynatrace advantage lies in its ability to set logs in direct context, either automated by Davis® AI, in the context of specific team needs within an application, or as part of the broader observability scope provided by the Logs app itself.

Only Dynatrace provides a comprehensive and accessible log management and analytics experience, helping teams resolve issues faster without compromising on depth. Only Dynatrace Grail is schema-on-read and indexless, built with scaling in mind and built for exabyte scale, leveraging massively parallel processing. With Dynatrace, there is no need to think about schema and indexes, re-hydration, or hot/cold storage concepts.

This architecture also means you’re not required to determine your log data use cases beforehand or while analyzing logs within the new Logs app.

This includes logs collected from your organization’s Hypervisors, Linux and Windows servers, cloud-native logs from Azure, AWS, GCP, or Oracle, alongside the networking signals from Cisco, Juniper, NetScaler, or F5 appliances—to name a few examples.

What’s next

It’s simple, fast, and easy to ingest and gain multi-purpose value from logs—all without the need to be an expert or learn a complex query language first.

Learn how Dynatrace can address your specific needs with a custom live demo. Our observability experts will walk you through our solutions and show you how to deliver excellent customer experiences, foster your application security, and simplify IT operations.

If you’re not yet a Dynatrace customer, start your 15-day free trial.

The post Simplicity meets power: Introducing the all-new Dynatrace Logs app appeared first on Dynatrace news.

Dynatrace enables various teams, such as developers, threat hunters, business analysts, and DevOps, to effortlessly consume advanced log insights within a single platform. Dynatrace Grail™ and Davis^® AI act as the foundation, eliminating the need for manual log correlation or analysis while enabling you to take proactive action.

Dynatrace unified observability and security help enterprises, like our customer, BMO, save time and money, fostering collaboration across business, development, and operations teams.

In this blog post, we’ll provide an overview of the following log-related topics:

• Logs in the context of applications
• Easy yet powerful access to any log with the all-new Logs app
• How logs are ingested
• Dynatrace Application Security with logs
• Dynatrace Davis CoPilot™ integration and AI-powered Application Security
• Schemaless: Instantly gain business event insights from logs

Simplicity is key to success

As IT responsibilities shift left and expand, such as when developers take on application security duties, simplicity in toolsets becomes essential. Existing siloed tools lead to inefficient workflows, fragmented data, and increased troubleshooting times.

Tool consolidation is becoming a priority for C-level decision-makers in 2025. Enterprises are turning to Dynatrace for its unified observability approach for cloud-native, on-premises, and hybrid resources.

Rather than relying on disparate tools for each environment and team, Dynatrace integrates all data into one cohesive platform. Davis AI contextually aligns all relevant data points—such as logs, traces, and metrics—enabling teams to act quickly and accurately while still providing power users with the flexibility and depth they desire and need.

Logs in context with applications

Applications are provided within the Dynatrace platform to address the various needs of different teams and specific use cases. DevOps teams operating, maintaining, and troubleshooting Azure, AWS, GCP, or other cloud environments are provided with an app focused on their daily routines and tasks.

For instance, in a Kubernetes environment, if an application fails, logs in context not only highlight the error alongside corresponding log entries but also provide correlated logs from surrounding services and infrastructure components. This shortens root cause analysis dramatically, as explained in our recent blog post Full Kubernetes logging in context from Fluent Bit to Dynatrace.

The show surrounding logs function provides Dynatrace users with the ability to dive deeper and surface context-specific log lines of the components and services linked to the problem—all without a single line of code or complex query language knowledge. This is explained in detail in our blog post, Unlock log analytics: Seamless insights without writing queries.

For advanced analysis, there is a direct “open with” path, allowing you to load the current view in Notebooks for manual analysis, the Logs app, or other apps capable of visualizing context-specific log lines.

While the way that logs in context are interconnected and provided by Dynatrace is unique, as also Gartner^® recognized for the 14^th consecutive time in their Magic Quadrant™, there are use cases where this is not sufficient and raw access to the logs is required.

Easy yet powerful access to any log with the all-new Logs app

Developers love Dynatrace, without a doubt, and are one of the many teams next to DevOps or SREs making use of our all-new Logs app. The reasons are easy to find, looking at the latest improvements that went live along with the general availability of the Logs app.

In our product news blog post, Simplicity meets power: Introducing the all-new Dynatrace Logs app, we examine these features, which make life easy for new Dynatrace users, along with the newly introduced DQL Editor for power users.

Directly from individual log line results, you can filter simply by selecting corresponding items in the details pane or by loading the surrounding logs when selecting Show surrounding logs.

Keep in mind that Dynatrace Grail is schema-on-read and indexless, built with scaling in mind. There is no need to think about schema and indexes, re-hydration, or hot/cold storage. This architecture also means you are not required to determine your log data use cases beforehand or while analyzing logs within the new logs app.

How logs are ingested

Dynatrace offers OpenPipeline to ingest, process, and persist any data from any source at any scale. OpenPipeline ensures data security and privacy—data is collected and processed securely and compliantly, with high-performance filtering, masking, routing, and encryption—and contextualizes incoming data in real time. Using patent-pending high ingest stream-processing technologies, OpenPipeline currently optimizes data for Dynatrace analytics and AI at 0.5 Petabyte per day and tenant; this will soon increase to one Petabyte per day and tenant.

OpenPipeline’s high-performance filtering and preprocessing provide full ingest and storage control for the Dynatrace platform. As a result, dedicated data pipeline tools are unnecessary for preprocessing data before ingestion.

Dynatrace meets your teams where they are, with your preferred ingest routes and methods—be they Fluent Bit, OpenTelemetry, SysLog, or automatic log collection leveraging OneAgent, to name a few options.

If your team deploys applications cloud-natively, we meet you there, too, as we recently covered in our blog post, Dynatrace log management innovations: Syslog, AWS Firehose. We covered in rich detail how Dynatrace supports log ingestion for cloud-native workloads and simplified log ingestion also for hybrid environments.

In any case, at the heart of the Dynatrace Platform, Grail enables contextual analytics across unified observability, security, and business data. Grail is built for exabyte scale and leverages massively parallel processing (MPP) as well as advanced automated cold/hot data management to ensure that data remains fully accessible at all times, with zero latency, and full hydration.

With no index or schema boundaries in place, paired with long-term data retention ranging from 1 day up to 10 years, you can leverage metrics, logs, and traces for a variety of additional use cases, such as business analytics or security analytics.

Dynatrace Application Security with logs

While most enterprises have Application Firewalls (AppFW), Intrusion Detection Solutions (IDS), and Static Code Analysis (SCA) in place for applications that will be deployed to production, it’s still relevant to understand if anomalies occur during runtime. Monitoring known vulnerabilities within the service hosting the application itself is just another puzzle piece to be considered for full end-to-end observability.

Instead of relying on static patterns, Dynatrace Causal AI understands the desired outcome of the triggered action and the context of the environment hosting the service. For example, deleting the database is not an expected outcome when the function provided is to update a user profile.

With these sophisticated security analytics, simple use cases such as authentication failure anomalies or password spraying attacks, along with more technical HTTP RST statistics, can be visualized in simple and sharable views in Security Investigator, leveraging logs.

Advanced analytics are not limited to use-case-specific apps. Dynatrace offers Notebooks and Dashboards to build views and reports—without the need to write a single line of code or Dynatrace Query Language, all supported by Dynatrace Davis CoPilot™ integration.

Dynatrace Davis CoPilot integration and AI-powered Application Security

Davis CoPilot™ assists occasionally visiting Dynatrace operators throughout the platform in a variety of applications, including Dynatrace Notebooks.

With natural language input in the example displayed in the screenshot below, “Show me the most recurring log lines and add a column with the log source and AWS region,” Davis CoPilot will evaluate the input, translate it into a corresponding DQL Query, and fetch the results accordingly.

Guardrails are in place and can be altered to prevent a high volume of unwanted material from being scanned or returned to Notebooks.

In the same way, Davis CoPilot can recommend remediation strategies and simplify security analysis across all data by translating natural language into the Dynatrace Query Language (DQL) to drive attack protection, security investigations, and forensics.

As mentioned, when ingesting relevant logs into Grail, including firewall and authentication gateway logs, Davis AI can provide end-to-end security insights. At the same time,

Davis CoPilot is integrated into the Dynatrace platform as an intelligent assistant to ease the effort of configuring and finding relevant details and options. By simply asking CoPilot the question stated above, you’re provided with the required configuration steps and product documentation references.

Schemaless: Instantly gain business event insights from logs

A unique picture can be drawn when logs from business-critical LoB applications are collected, which is an underestimated value that can be gained when using Dynatrace.

There are many customer examples and use cases, like room bookings in hotel portals, shopping cart statistics from online shops, or customer metrics from finance platforms, where customers can gain additional business value by translating logs to metrics within Dynatrace.

A pipeline health dashboard showing growth, paired with security-related information, is just one of the many examples of what you can build, either on your own or with the help of the Dynatrace Services team.

In our blog post, Leverage logs for an end-to-end view of your business processes via Dynatrace OpenPipeline, we demonstrated the retail company dashboard example above at a higher granularity, including the necessary steps for JSON log ingestion via Dynatrace OpenPipeline.

Conclusion

It’s simple, fast, and easy to ingest and gain multi-purpose value from logs—all without the need to be an expert or the requirement to first learn a complex query language.

Davis AI and CoPilot make it easy for casual contributors and power users to gain meaningful insights and build notebooks or dashboards while simultaneously increasing application security and reliability.

Learn how Dynatrace can address your specific needs with a custom live demo. Our observability experts will walk you through our solutions and show you how to deliver excellent customer experiences, foster your application security, and simplify IT operations.

If you’re not yet a Dynatrace customer, start your 15-day free trial.

The post Unlock the power of contextual log analytics appeared first on Dynatrace news.