In practice, most organizations attempt to solve this with one of two compromises: lock everything down centrally and become a bottleneck for every pipeline change, or copy rules across pipelines and hope nothing drifts. Neither approach scales. And as data requirements shift faster than ever to address new services, new regulatory mandates, and new object types to observe, the cost of such compromise keeps growing.

Pipeline Groups change that dynamic. Generally available now in Dynatrace SaaS version 1.332, Pipeline Groups let platform engineering teams, the teams with admin permissions over the observability platform’s configuration, mandate and standardize ingest behavior across many pipelines, while safely delegating day-to-day configuration of individual pipelines to the teams best suited to own them.

The problem: Pipeline governance at scale

In large enterprises, dozens of teams may each operate their own custom pipelines. Common requirements, security context enrichment, cost allocation tagging, and storage bucket assignment must be applied consistently across the board. However, without a structured mechanism to enforce such standards, platform teams face a choice that gets harder with every new team, every new data source, and every new regulation:

• Lock everything down and become the gatekeeper for every pipeline change. Every adjustment, no matter how small, becomes a ticket, a review, a delay. Innovation stalls. Teams wait days for changes that should take minutes.
• Copy rules everywhere and accept the risk of configuration drift. What starts as a manageable set of shared rules gradually fragments—one team forgets to apply the latest cost allocation tag, another skips the security enrichment step, a third routes data to the wrong bucket. The inconsistencies compound silently until they surface as compliance gaps or billing surprises.

Customers consistently tell us that this approach doesn’t scale. What they need is a way to separate what must always happen from what teams should be free to decide, and to encode that separation directly into the platform, not just into process documents that team members forget to follow.

What are Pipeline Groups?

A Pipeline Group is a first-class configuration object that separates a global pipeline list into distinct sets; the pipelines in these sets are then members of the group. A Pipeline Group orchestrates a special type of reusable pipeline to define what happens before or after the stages of the member pipelines.

Pipeline Groups allow you to:

• Organize pipelines into groups with clearly defined membership.
• Define execution order so processing happens in a predictable, layered sequence.
• Control stage execution for the member pipelines so that teams can turn on or turn off specific pipeline stages.
• Enforce mandatory global processing that no team can bypass or override.
• Allow team-level customization within boundaries defined by the platform team.

Pipeline Groups determine how data flows through your pipelines. This ensures that governance isn’t an afterthought bolted onto a pipeline configuration, but rather it’s built into the execution model itself.

How Pipeline Group ownership works

The design behind Pipeline Groups draws a deliberate line between group-level configuration and member-pipeline configuration. How organizations map this division to team ownership is up to them. Pipeline Groups provide the mechanism, not a prescriptive policy. The pattern we see most often is straightforward: platform or SRE teams own the pipeline groups, and application teams own their member pipelines within those groups.

To make this more concrete, consider a platform team that’s responsible for observability across multiple business units. They create a Pipeline Group that adds a business segment field to every record for organizational attribution, applies cost allocation tags for accurate chargeback, sets security context so sensitive data is handled consistently, and assigns data to the correct storage bucket based on retention and compliance needs. These are the rules that must always apply, and because they operate at the group level, no member pipeline can bypass or override them.

Member pipeline ownership can be more granular than a simple admin-vs-team split. While a group is always admin-owned, the individual pipelines that make up the Pipeline Group can each have different owners. For example, a pipeline that enriches every record with organizational metadata or classifies data sensitivity might be globally relevant and owned by the platform team, or by a specialist team like the security team. A pipeline that handles domain-specific compliance logic, such as PCI field masking for a financial services division, might be owned by that compliance team because they have the expertise that the platform team lacks. This arrangement reflects how responsibility within that organization is distributed.

Within that group, application teams each get their own member pipeline. One team configures parsing for a specific log format. Another sets up filtering to reduce noise. A third extracts metrics tailored to their services. Each team works independently within its own pipeline without touching the platform-level rules above and without needing to coordinate with other teams or the platform team for routine changes.

The value of this separation becomes clear when things change, as they do frequently in large enterprises. Say that one of those business units onboards a new microservice that generates a completely new log format. Under a centralized-only model, the team would file a request, wait for the platform team to update the pipeline, validate, and iterate. With Pipeline Groups, the team simply adds their parsing and filtering rules to their own member pipeline. The mandatory enrichment, cost tagging, and routing are already guaranteed by the group. The new service is observable in hours, not weeks, and the platform team doesn’t need to be involved at all.

Or picture the reverse direction: a new regulatory requirement lands, say, a mandate that all log data from EU-based services must be assigned to region-specific storage. The platform team updates the group configuration once. Every member pipeline inherits the change immediately.

Why the flexibility of Pipeline Groups matters

The pace at which enterprise data requirements evolve has fundamentally changed. New services are spun up in days, not months. Regulatory landscapes shift across jurisdictions. The types of software entities that organizations need to observe, from traditional infrastructure to AI model outputs, edge devices, and third-party SaaS telemetry, keep expanding. In this environment, a rigid, centralized-only pipeline configuration becomes a constraint rather than a benefit.

Pipeline Groups are built for this reality. They give enterprises a mechanism that:

1. Creates room for innovation. Application teams can iterate on their pipeline configurations independently, experimenting with new parsing rules, extracting new metrics, and adapting to new data shapes without waiting for central approval on every change.
2. Eliminates bottlenecks. Platform teams define the rules once and let the system enforce them. Teams are freed from serving as gatekeepers for routine changes and can focus on architecture, standards, and strategy.
3. Ensures that ever-changing guardrails are applied. As compliance requirements, security policies, or cost structures evolve, platform teams can update them all in one place. The changes propagate automatically to every pipeline in the Pipeline Group.

Pipeline Groups give platform teams the governance controls and flexibility they need

Pipeline Groups give enterprise platform teams the governance controls they’ve been asking for, mandatory processing, centralized ownership of sensitive stages, and structured delegation, all without taking away the flexibility that makes Dynatrace OpenPipeline® valuable to individual teams in the first place.

Ready to get started with OpenPipeline and Pipeline Groups?

If your organization manages observability across multiple teams and struggles with consistency, Pipeline Groups are now generally available.

Learn how to define global guardrails once and empower your application teams to build their own pipelines confidently within standardized boundaries.

The post Pipeline Groups in Dynatrace OpenPipeline: Enterprise-grade governance explained appeared first on Dynatrace news.

• Your AI investments can’t reach their potential without effective AI governance.
• AI governance is a challenge that demands unprecedented agility, proactive measures, and comprehensive oversight to manage complexity.
• With Dynatrace, you’re prepared for whatever comes next.

Stay compliant and build trust in your AI systems

AI regulation is tightening, and non-compliance is becoming a huge risk to broader, production-scale AI adoption. Penalties are only part of the impact: reputational damage, customer mistrust, and stalled innovation can cripple even forward-looking organizations. That’s why we’re introducing data governance and audit trails for AI observability: a scalable way to manage, monitor, and secure the AI data lifecycle with end-to-end lineage, retention controls, and evidentiary records of model and user interactions.

Our platform helps you turn governance into a competitive advantage. Built-in audit support helps with emerging regulations like the EU AI Act, and alignment to industry standard frameworks such as NIST AI and ISO/IEC 42001:2023.

The hidden challenges of AI data governance

The complexity of compliance

AI regulations are becoming stricter, and new regulations are on the horizon. Organizations must maintain detailed records of AI activities for years, ensure transparency of data and processes, and align retention policies with legal requirements. These measures are imperative for trust and safety, but they introduce significant challenges. For instance, AI-related events are often scattered across multiple systems, applications, and teams, complicating efforts to create a unified audit trail. Default retention periods can fall short of regulatory needs, and manual governance processes are error-prone and infeasible at scale.

The risk of non-compliance

Failing to meet regulatory standards risks hefty fines and penalties, but the market consequences, reputational damage, and loss of customer trust are even worse. Without the right tools, organizations will struggle to manage the growing complexity of AI data governance and reap the full benefits of AI investments.

Introducing Dynatrace data governance and audit trails

Dynatrace has a long history of empowering organizations to tackle complex challenges with AI-driven solutions. Building on this expertise, we’re introducing a new set of capabilities designed to simplify compliance, enhance transparency, and streamline data management. With Dynatrace, you can:

• Automatically retain AI-related events for up to 10 years in Grail®, our secure data lakehouse.
• Monitor and capture events from platforms like Amazon Bedrock, tracking everything from model deployments to fine-tuning activities.
• Leverage OpenTelemetry to collect real-time traces and metrics of AI workloads, along with every AI user interaction, giving you a complete picture of your AI ecosystem.

Close the compliance gap with embedded oversight

What sets Dynatrace apart is seamless integration with your existing workflows. With OpenPipeline® on Grail, you can route AI-related events to custom storage buckets with extended retention, automatically, and without forcing teams to change tools or processes. This allows long-term auditability and helps meet sector-specific compliance requirements that might require special retention and auditability measures.
Once configured, Dynatrace can automatically route and store events, creating a reliable and transparent audit trail. This helps to reduce fragmentation, tool sprawl, and manual effort traditionally associated with data governance.

Imagine being able to trace every user interaction, model training session, or deployment event with just a few clicks. Dynatrace makes this possible by consolidating fragmented data into a single, coherent view. Whether you’re responding to a regulatory inquiry or optimizing your AI models, you’ll have the insights you need, when you need them.

Simplified and instant data filtering with Dynatrace segments

Not all audit data carries the same compliance weight. For global enterprises with complex IT environments, the ability to instantly filter data by precise criteria is essential for accelerating compliance across diverse regulations, from strict local regulatory transparency obligations to lighter regimes elsewhere.

Dynatrace segments make it simple to break down and filter data to match your analysis needs and regulatory requirements:

• Targeted compliance views: Instantly filter audit data by region, environment, platform, model, or custom criteria to align with diverse regulatory requirements.
• Dynamic adaptability: Segments automatically update, for example, when new LLM models or environments are introduced, minimizing manual maintenance and keeping governance current.
• Reusable assets: Leverage a single dashboard or notebook across multiple use cases by simply applying different segments, reducing duplication of effort.
• Noise reduction: Exclude irrelevant data such as development or test logs to keep compliance and observability focused on what truly matters.
• Custom team-context: Provide different teams (for example, compliance, data science, operations) with clear, filtered views of their audit data, ensuring ownership and audit-readiness across departments.

From observability to trusted automation

The future of AI governance lies in proactive, automated solutions that not only meet today’s regulations but also anticipate tomorrow’s challenges. With Dynatrace, you’re not just complying—you’re building a foundation of trust and reliability that scales with your business. By capturing and integrating AI events into a unified platform, Dynatrace transforms compliance from a burden into a strategic advantage.

Get started today

Ready to simplify your AI data governance?

Try it out yourself on the Dynatrace playground. Or, learn how to configure AI governance in our documentation.

Read more

• Part one of the Rise of Agentic AI blog series covers the fundamentals of AI agents, models, and emerging communication standards such as Agent2Agent (A2A) and MCP.
• Part two of the Rise of Agentic AI blog series explores AI agent observability and monitoring, A2A and MCP communications, and how to scale and monitor Amazon Bedrock Agents.
• Part three explains how to monitor Amazon Bedrock Agents and how observability optimizes AI agents at scale.
• Part four covers full-stack observability for AI with NVIDIA Blackwell and NVIDIA NIM.
• Part five demonstrates how to build a simple agentic application using the OpenAI Agents SDK and instrument the data with Dynatrace.
• Part six explores AI Model Versioning and A/B testing for smarter LLM services.

The post The rise of agentic AI part 7: introducing data governance and audit trails for AI services appeared first on Dynatrace news.