In practice, most organizations attempt to solve this with one of two compromises: lock everything down centrally and become a bottleneck for every pipeline change, or copy rules across pipelines and hope nothing drifts. Neither approach scales. And as data requirements shift faster than ever to address new services, new regulatory mandates, and new object types to observe, the cost of such compromise keeps growing.

Pipeline Groups change that dynamic. Generally available now in Dynatrace SaaS version 1.332, Pipeline Groups let platform engineering teams, the teams with admin permissions over the observability platform’s configuration, mandate and standardize ingest behavior across many pipelines, while safely delegating day-to-day configuration of individual pipelines to the teams best suited to own them.

The problem: Pipeline governance at scale

In large enterprises, dozens of teams may each operate their own custom pipelines. Common requirements, security context enrichment, cost allocation tagging, and storage bucket assignment must be applied consistently across the board. However, without a structured mechanism to enforce such standards, platform teams face a choice that gets harder with every new team, every new data source, and every new regulation:

• Lock everything down and become the gatekeeper for every pipeline change. Every adjustment, no matter how small, becomes a ticket, a review, a delay. Innovation stalls. Teams wait days for changes that should take minutes.
• Copy rules everywhere and accept the risk of configuration drift. What starts as a manageable set of shared rules gradually fragments—one team forgets to apply the latest cost allocation tag, another skips the security enrichment step, a third routes data to the wrong bucket. The inconsistencies compound silently until they surface as compliance gaps or billing surprises.

Customers consistently tell us that this approach doesn’t scale. What they need is a way to separate what must always happen from what teams should be free to decide, and to encode that separation directly into the platform, not just into process documents that team members forget to follow.

What are Pipeline Groups?

A Pipeline Group is a first-class configuration object that separates a global pipeline list into distinct sets; the pipelines in these sets are then members of the group. A Pipeline Group orchestrates a special type of reusable pipeline to define what happens before or after the stages of the member pipelines.

Pipeline Groups allow you to:

• Organize pipelines into groups with clearly defined membership.
• Define execution order so processing happens in a predictable, layered sequence.
• Control stage execution for the member pipelines so that teams can turn on or turn off specific pipeline stages.
• Enforce mandatory global processing that no team can bypass or override.
• Allow team-level customization within boundaries defined by the platform team.

Pipeline Groups determine how data flows through your pipelines. This ensures that governance isn’t an afterthought bolted onto a pipeline configuration, but rather it’s built into the execution model itself.

How Pipeline Group ownership works

The design behind Pipeline Groups draws a deliberate line between group-level configuration and member-pipeline configuration. How organizations map this division to team ownership is up to them. Pipeline Groups provide the mechanism, not a prescriptive policy. The pattern we see most often is straightforward: platform or SRE teams own the pipeline groups, and application teams own their member pipelines within those groups.

To make this more concrete, consider a platform team that’s responsible for observability across multiple business units. They create a Pipeline Group that adds a business segment field to every record for organizational attribution, applies cost allocation tags for accurate chargeback, sets security context so sensitive data is handled consistently, and assigns data to the correct storage bucket based on retention and compliance needs. These are the rules that must always apply, and because they operate at the group level, no member pipeline can bypass or override them.

Member pipeline ownership can be more granular than a simple admin-vs-team split. While a group is always admin-owned, the individual pipelines that make up the Pipeline Group can each have different owners. For example, a pipeline that enriches every record with organizational metadata or classifies data sensitivity might be globally relevant and owned by the platform team, or by a specialist team like the security team. A pipeline that handles domain-specific compliance logic, such as PCI field masking for a financial services division, might be owned by that compliance team because they have the expertise that the platform team lacks. This arrangement reflects how responsibility within that organization is distributed.

Within that group, application teams each get their own member pipeline. One team configures parsing for a specific log format. Another sets up filtering to reduce noise. A third extracts metrics tailored to their services. Each team works independently within its own pipeline without touching the platform-level rules above and without needing to coordinate with other teams or the platform team for routine changes.

The value of this separation becomes clear when things change, as they do frequently in large enterprises. Say that one of those business units onboards a new microservice that generates a completely new log format. Under a centralized-only model, the team would file a request, wait for the platform team to update the pipeline, validate, and iterate. With Pipeline Groups, the team simply adds their parsing and filtering rules to their own member pipeline. The mandatory enrichment, cost tagging, and routing are already guaranteed by the group. The new service is observable in hours, not weeks, and the platform team doesn’t need to be involved at all.

Or picture the reverse direction: a new regulatory requirement lands, say, a mandate that all log data from EU-based services must be assigned to region-specific storage. The platform team updates the group configuration once. Every member pipeline inherits the change immediately.

Why the flexibility of Pipeline Groups matters

The pace at which enterprise data requirements evolve has fundamentally changed. New services are spun up in days, not months. Regulatory landscapes shift across jurisdictions. The types of software entities that organizations need to observe, from traditional infrastructure to AI model outputs, edge devices, and third-party SaaS telemetry, keep expanding. In this environment, a rigid, centralized-only pipeline configuration becomes a constraint rather than a benefit.

Pipeline Groups are built for this reality. They give enterprises a mechanism that:

1. Creates room for innovation. Application teams can iterate on their pipeline configurations independently, experimenting with new parsing rules, extracting new metrics, and adapting to new data shapes without waiting for central approval on every change.
2. Eliminates bottlenecks. Platform teams define the rules once and let the system enforce them. Teams are freed from serving as gatekeepers for routine changes and can focus on architecture, standards, and strategy.
3. Ensures that ever-changing guardrails are applied. As compliance requirements, security policies, or cost structures evolve, platform teams can update them all in one place. The changes propagate automatically to every pipeline in the Pipeline Group.

Pipeline Groups give platform teams the governance controls and flexibility they need

Pipeline Groups give enterprise platform teams the governance controls they’ve been asking for, mandatory processing, centralized ownership of sensitive stages, and structured delegation, all without taking away the flexibility that makes Dynatrace OpenPipeline® valuable to individual teams in the first place.

Ready to get started with OpenPipeline and Pipeline Groups?

If your organization manages observability across multiple teams and struggles with consistency, Pipeline Groups are now generally available.

Learn how to define global guardrails once and empower your application teams to build their own pipelines confidently within standardized boundaries.

The post Pipeline Groups in Dynatrace OpenPipeline: Enterprise-grade governance explained appeared first on Dynatrace news.

Observability data can tell you a lot, but often not in a language that your business understands. Traces fire, business events flow, and dashboards light up, yet P1 incidents still break out. The first question from the war room is typically, “Who’s impacted?” Too often, the honest answer is, “Let me check the database and get back to you.” This gap between raw telemetry and human-readable business context is exactly what lookup tables in Dynatrace are designed to close.

In our introductory blog post on Dynatrace lookup tables, we explained the fundamentals: what lookup tables are, how to ingest them, and initial use cases spanning security to business context enrichment.

In this blog, we’ll go deeper, with real-world examples of lookup tables, practical patterns, and how teams are already putting them to work.

The problem: Dashboards display cryptic identifiers, not insights

Daniel Adams, Observability Engineer at FreedomPay, put it plainly in a recent Dynatrace Tips & Tricks session:

“A lot of our apps are not coded to show customer names or plain text human-readable stuff. It’s mostly IDs being passed back and forth.”

The result is that dashboards are filled with cryptic identifiers that operations teams and business stakeholders can’t mentally decode. And splitting data by customer, country, or store requires trips back to the SQL database for further contextual configuration.

Many organizations that run distributed systems at scale face the same challenge: telemetry is rich, but the business context is missing.

The solution: Use lookup tables to make your reference data available in Grail

Lookup tables let you upload reference data (CSV, JSON, or XML files) directly into Dynatrace Grail®, where it lives alongside your logs, traces, metrics, and business events. Once uploaded, two DQL commands unlock the enrichment from your lookup tables:

• Load fetches the contents of a lookup table, working just like querying any other data type in Grail.
• lookup joins records from your observability data with matching rows in the lookup table, using a defined key field.

In FreedomPay’s case, the lookup table holds customer names, customer codes, store IDs, and countries. The join key is the store ID, the same cryptic identifier that was previously displayed on their dashboards. One DQL lookup command later, those IDs become readable names, and the data can be split by country, grouped by customer, and surfaced in a dashboard that anyone on the team can use.

Lookup tables reduce user cognitive load

The impact of lookup tables is not just cosmetic. Daniel described the before-and-after in P1 situations:

“Previously, everyone was asking: who’s impacted, which customer? Now it’s just very apparent and real-time. People can share screenshots directly from Dynatrace.”

The shift from “let me go check” to “it’s right here” compresses triage time and reduces the cognitive load on everyone in the room.

Beyond incident response, lookup tables also power dashboard variables. FreedomPay uses dashboard variables to populate dropdown filters (for example, clients and countries) directly from the lookup table itself, keeping the UI dynamic without manual maintenance. The load command behaves like any other fetch command but operates on reference data rather than telemetry, making it a natural fit for driving interactive dashboards.

And the enrichment isn’t limited to business events. The load and lookup commands work across all data types in Grail.

In practice: cost allocation with lookup tables

Cost allocation is a good example of how lookup tables bridge the gap between operational data and business reality. Many organizations already maintain mappings that associate users, teams, or services with products or cost centers, but that context rarely lives inside observability data. Lookup tables allow you to introduce such context.

Consider DPS (Dynatrace Platform Subscription) consumption driven by user activity such as running queries, triggering automation, or executing functions. On its own, such usage is hard to attribute to a specific part of the organization. By associating users with products or cost centers in a lookup table, you can attribute usage to organizational ownership rather than just technical activity, align cost views with existing financial or product structures, and keep that attribution logic centralized and reusable rather than embedding it in individual analyses.

What makes this pattern scale is that business mappings change over time, while usage data continues to flow. Lookup tables allow you to evolve the mapping without rewriting your analyses. The same mechanism used for cost allocation works equally well for mapping users to teams, services to portfolios, or environments to internal programs. Cost allocation simply highlights how powerful this becomes when operational data is interpreted through a business lens.

Importantly, this is just one way to approach cost allocation, not the only way. Some teams rely on host-based attribution, others on pipeline metadata or external financial systems. Lookup tables complement these models by making it easy to incorporate existing reference data wherever it adds clarity.

Keeping lookup data fresh

For lookup tables to deliver lasting value, their data must remain current. FreedomPay currently uses Postman to push updated CSVs via the upload API, but they’re building automation to pull this data from their SQL backend daily and sync changes using the overwrite: true parameter. The lookup table path stays the same; the data underneath gets refreshed. Downstream dashboards and queries are updated automatically.

More broadly, there are several approaches to keeping lookup data up to date:

• Dynatrace Workflows automate extraction and upload from SQL databases or APIs on a schedule.
• Periodic refresh updates your lookup data programmatically as part of CI/CD or data sync processes.
• Manual refresh uses Postman, cURL, or other API-based uploads with the overwrite flag when data changes infrequently.

Dynatrace lookup tables are now generally available

With the release of Dynatrace SaaS version 1.334, lookup tables are generally available for all customers running Dynatrace SaaS with an active Dynatrace Platform Subscription (DPS). GA brings production-readiness, improved query performance for the lookup command, and full integration with Grail’s scalability and access controls.

To get started

1. Identify a data enrichment use case: cost allocation, customer context, error code mapping, or security allow lists.
2. Prepare your reference data as a CSV, JSON, or JSONL (JSON Lines) file.
3. Upload the file to Grail using the Dynatrace API, Workflows, or a custom app.
4. Use the load and lookup commands in your DQL queries, notebooks, and dashboards.

For detailed instructions, visit our lookup data documentation. And to see how FreedomPay built its implementation from end to end, watch the full demo on YouTube.

The post How lookup tables turn observability data into business insight appeared first on Dynatrace news.

As onboarding and deployment become faster, self-service and automation have become a requirement; more than ever before, costs must remain predictable, attributable, and easy to report.

If you’re working to make log spend visible and fair across teams, you’re not alone—this is a common challenge in modern, and cloud native environments.

Assign cost centers and products

Dynatrace allows precise cost allocation for logs, so you can attribute log ingestion and retention to the right cost centers and cost products. This makes internal showback and chargeback straightforward.

Why this matters

• Cloud-native apps and microservices generate log sources rapidly, while shared platforms can blur ownership. Cost allocation brings clarity.
• Teams want autonomy and instant access, without surprises. Build accountability and trust with simplified and automated cost attribution.
• Service providers require accurate reporting for budgeting, audits, and governance purposes. Cost allocation makes it predictable and repeatable, with more than 60,000 cost allocation combinations out of the box and more available for our large enterprise customers, who are adopting this feature today at scale.

Cost optimization is a team sport

The era of budgets and cost optimization being a concern solely left to finance departments is a thing of the past. Teams are expected to own their budgets. Meaning that cost optimization is not a separate accounting artifact, but rather a shared accountability that each team is expected to contribute to.

Whether your teams offer services, applications, or infrastructure, they will want to leverage logs. Team-level accountability for log management begins with allocating log spend to individual products, owners, or any other method your FinOps practice uses for tracking.

With cost allocation for logs, you can take the non-disruptive route

You can leverage the established and defined annotations and labels of the source, for instance, directly from Kubernetes.

But there might be reasons you want to make that attribution at the processing stage:

• Your source might not be capable of providing annotations and tags.
• You don’t have the resources to configure each source individually to match attribution.
• You might want to take a centralized approach, rather than contacting each team individually.
• Your cost allocation requirements are too complex and require a script or a processing technology.

In Dynatrace OpenPipeline®, you can enrich your logs during processing. What may be tedious manual work elsewhere is now centralized and automated.

Whatever your requirements and expectations are, whether you need simple tagging or complex attribution rules, OpenPipeline is here to help.

Because cost optimization is a team sport, the output aligns perfectly with the most common FinOps formats, providing the exact granularity necessary to support enterprise-wide optimization initiatives.

What’s new with cost allocation

• Billing usage events for logs can now be enriched with dt.cost.costcenter and dt.cost.product.
• Attribution of cost centers and products should best take place at source, but can be dynamically processed with OpenPipeline.
• You can mix and match both attributes or use them individually. This allows you to allocate costs by business unit and product/service, allowing for granular chargeback and showback.

Example: Chargeback and showback with Dynatrace cost attributes

Many organizations use chargebacks to create accountability and transparency for shared costs by charging internal departments for the resources or services they consume, based on actual usage. An effective way to implement this with Dynatrace is to use the dt.cost.costcenter and dt.cost.product attributes together.

Consider a scenario where a central IT team provides observability services to multiple business units, such as Retail, Corporate Banking, and Wealth Management. Each unit runs several applications that generate logs through ingestion channels, such as OneAgent®, Log Ingest API, or OpenTelemetry integrations. To ensure accurate cost attribution, the IT team configures these log sources to automatically enrich each log with the appropriate cost center and product identifiers.

For example, logs generated by the Retail unit’s mobile banking app are enriched with dt.cost.costcenter: retail and dt.cost.product: mobile-app. This dual-tagging approach allows the central IT team to allocate log-related costs to the correct business unit and break down those costs by specific products or services within that unit. When billing usage events are enriched with these attributes, Finance teams can apply direct chargeback methods.

Optimize log spend with granular showback

Using the same attribution, IT teams can generate detailed reports showing how much each cost center is spending on log ingestion and retention, as well as which products drive that spend. These reports can be flexibly incorporated with other costs attributed to the same owners or products, such as query costs, using Lookup data in Grail®.

Now consider that same Retail unit. The granular attribution shows that the mobile app is responsible for 70% of its log spend. The team can now take targeted actions. For instance, it can reduce log verbosity in non-critical flows or adjust retention policies to optimize costs.

Meanwhile, central IT maintains full transparency and control over the shared observability platform. This centrally operated, data-driven chargeback model allows teams to operate autonomously without disruption, while aligning with FinOps principles.

Getting to the numbers

You can report and analyze cost allocation in multiple ways, depending on your audience, business requirements, workflows, and the tools you have.

• Dashboards: Crafting individual dashboards to visualize log ingestion and retention by cost center and product is one of Dynatrace’s key strengths. Individual filters, views, and visuals allow you to slice and dice custom dashboards for your teams.
• Notebooks: Explore and validate enriched billing usage events alongside Grail data for deeper analysis or ad‑hoc investigations. This route allows admins to align consumption data with log query insights of users in a shareable manner, as the results are stored. Non-admin users can view the results this way when the Notebook is shared with them.
• Account Management portal: Create cost management reports to track accrued costs and perform showback/chargeback at scale across business units and products, sent by email and downloadable in CSV format.
• Lookup data: Some organizations may prefer using lookup tables to allocate costs to their owners or products. This is a good fit for customers who already work with organizational structures that link owners with product and their respective cost centers. It can also serve as an additional support to track queries in your environments. Learn more about Lookup data in Grail.

With these views, IT and Finance can align on the same source of truth, driving targeted optimizations such as adjusting log verbosity in non-critical flows or tuning retention policies, while maintaining shared platform governance.

Get started: a guide for cost allocation

Let’s recap the best practices to get started successfully:

1. Inventory your log ingest channels and sources (OneAgent, API, OpenTelemetry, cloud/hyperscaler forwarders, log shippers).
2. Define attributes and assign labels for dt.cost.costcenter and dt.cost.product at the source before ingestion, for example, in Kubernetes, OneAgent, or the API and OpenTelemetry configuration of your apps and services.
3. If updating agents or code changes aren’t feasible, define OpenPipeline rules to enrich during the process.
4. Send sample data, verify attributes configured in Grail, and confirm visibility using the Logs app or your existing dashboards.
5. Iterate by team/product, expand coverage, and standardize reporting in the Account Management portal.

New to Log Management & Analytics in Dynatrace?

Ready to make log costs clear, fair, and easy to report? Define your attributes, turn on enrichment, and give your teams the accountability and insights they need—without slowing them down.

If you’re already using Dynatrace Platform Subscription (DPS), you can instantly get started with logs today! Additional resources and downloads are available in our community examples space on GitHub.

We invite you to explore our Dynatrace Playground tenant at no cost or to start a free trial to ingest your first logs with cost allocation.

The post Cost allocation for logs: Precise, flexible, and non-disruptive appeared first on Dynatrace news.

In large enterprises, attributing IT costs to various cost centers, teams, or departments can be cumbersome and is typically only possible through significant manual overhead. Sometimes, introducing new IT solutions is delayed or canceled because a single business unit can’t manage the operating costs alone, and per-department cost insights that could facilitate cost sharing aren’t available.

In scenarios like these, automated and precise cost allocation can make a huge difference. Cost Allocation also unlocks new possibilities for strategic IT management, empowering you to align IT spending with your business priorities and serving as a fundamental prerequisite for adopting FinOps practices.

FinOps, short for Financial Operations, is a methodology combining finance, technology, and business teams to optimize cloud spending and maximize value in cloud environments. Costs and their origin are transparent, and teams are fully accountable for the efficient usage of cloud resources.

With the addition of the new Cost Allocation feature, the Dynatrace Platform Subscription now enables the application of FinOps, providing detailed cost transparency in near real-time (data is updated every 15 minutes), which is paramount to taking your strategic IT management to the next level.

Automatically allocate costs to teams, departments, or apps for full cost-transparency

In recent years, the Dynatrace platform expanded with many innovative features covering various use cases, from business insights to software delivery. These enhancements enable you to extract more value from your data, leading to wider adoption across enterprise departments. As Dynatrace now powers many different teams, the Cost Allocation feature helps you better control and prioritize your internal spending.

Incurred usage can be tagged at its origin based on your unique company structure. Also known as “chargeback” or “showback,” this functionality enables you to align every aspect of IT expenditure with your organizational framework, as you can now pinpoint exactly where and when costs occur within your organization.

This gives you a better understanding of financial impact and allows for granular strategic decision-making.

New insights into cloud spend enable strategic prioritization and business alignment

Dynatrace Cost Allocation is a groundbreaking upgrade for your cost management that provides many benefits:

• Business alignment: Cost Allocation ensures that every dollar you spend on IT resources is directly linked to your business priorities.
• Enhanced cost transparency: Enabling detailed tracking and reporting of expenses across various departments and products gives unparalleled visibility into IT costs. This granular level of transparency helps identify cost drivers, monitor usage patterns, and uncover opportunities for cost savings.
• Increased budget control: Cost Allocation empowers organizations to clearly understand their current costs and resource usage for different cost centers and products.
• Better planning and forecasting: By analyzing historical data, organizations can forecast future spending and adjust their budgets, promoting a disciplined approach to IT financial planning.
• Easier Dynatrace rollout across organizations: DPS enhanced by Cost Allocation allows departments to extract value from the Dynatrace platform while only paying for what they need. By leveraging improvements in Identity and Access Management, admins can ensure that Dynatrace users only have access to the data they need to do their jobs.

Explore and visualize your cost data

Allocated costs are stored in the Dynatrace Grail™ data lakehouse, which enables you to utilize the entirety of the Dynatrace platform to analyze, explore, and visualize your data. Start with our downloadable dashboard and customize it to your needs.

Use Davis® AI for accurate forecasting or to automatically catch unexpected spending deviations. You can also set up tailored and automated alerts utilizing the Davis Anomaly Detection app. Our comprehensive suite of tools ensures that you can extract maximum value from your billing data, efficiently turning insights into action.

You can also create individual reports using Notebooks—or export your data as CSV—and share it with your financial teams for further processing.

Set up Cost Allocation

Implementing Dynatrace Cost Allocation is straightforward and can be tailored to fit the unique needs of any organization. The process involves configuring cost center and product fields, setting up an allow list for valid values within account management, and using Dynatrace’s powerful API to extract and analyze cost data.

Best practices include regularly reviewing cost allocation reports, ensuring all relevant expenses are captured accurately, and refining budget limits based on usage trends.

Head over to Dynatrace Documentation to learn more about how to set up cost allocation in your environment.

Conclusion

Dynatrace Cost Allocation is essential for enterprises that seek to align IT spending with business goals, achieve cost transparency, and maintain strict budget control. By leveraging cost allocation, organizations can optimize their IT investments, drive financial efficiency, and support their overarching business strategy.

With regular updates and comprehensive dashboards, businesses can maintain a clear view of their IT spending, ensuring accountability and fostering a culture of cost consciousness.

Get started with Cost Allocation

Existing Dynatrace customers with a Dynatrace Platform Subscription can integrate Cost Allocation into their IT management strategy anytime, achieving unparalleled transparency and budget control across critical areas. Read more to learn how to activate Cost Allocation in your environment, then download the ready-made Cost Allocation dashboard from our dedicated community user group and start monitoring your costs.

With the release of Dynatrace SaaS version 1.303, Cost Allocation is available for host monitoring, security protection, and security analytics. Support for additional capabilities will be added in the future. Our documentation provides more details and will help you better understand the existing limitations.

If you want to learn more about Cost Allocation and experience the functionality live, have a look at our Observability Lab episode with Andreas Grabner and Sophie Mayerwieser: Cloud Cost Transparency with Dynatrace fine-grained Cost Allocation.

The post Level up your strategic IT management with fully cost-transparent, fine-grained Dynatrace Cost Allocation appeared first on Dynatrace news.