An agent that can call tools, hand work to another agent, and run for hours is no longer just a prompt plus a model call. It is a small distributed system with identities, credentials, budgets, sessions, provider choices, and policies. Those concerns need somewhere to live that is not the model’s temporary context and not every individual application service.

That “somewhere” is increasingly described as an Agent control plane: a management and governance layer separate from the code that performs an agent’s task. The phrase is useful, but it is not a fully standardized architectural term. Different teams use it for different scopes: a centralized multi-tenant administration layer, a runtime policy gateway, or broader fleet management. The durable idea is the boundary, not a prescribed product or component size.

Plain English: An agent control plane keeps the rules and records for running agents. The agent execution path does the work; the control plane decides the conditions under which that work is allowed and operated.

This boundary is becoming more concrete. Langfuse’s September 11 releases added organization-level gateway control work, user-triggered session revocation, evaluator controls and history, session-timeline improvements, trusted-execution resolution, and provider-connection attribution. Those are operational capabilities that persist across individual model calls rather than features of one prompt or one Tool invocation. Langfuse v4.34.0 and v4.35.0 are examples of this boundary taking shape, not proof of one universal implementation.

What an agent control plane controls

An agent system has an execution plane: the path that receives a request, assembles Context, invokes a model, calls Tools, and returns or persists a result. This is where domain behavior happens. A support agent might look up an account; a coding agent might inspect a repository; a background agent might prepare a report.

The control plane sits beside that path. It owns durable management state: which organization owns a request, which agent configuration is active, which Provider connections are eligible, which policies apply, whether a session remains valid, and which evaluator version should judge a result. It can also retain usage and metering records for capacity planning or internal chargeback.

A Tenant is an organization or customer whose data, configuration, and permissions must remain separate from another’s. AWS describes an agentic control plane as distinct from the application plane, with responsibilities including configuration, tenant and agent Observability, consumption and metering, and tenant policy. AWS also identifies tenant identity, per-tenant resource provisioning, route and scope application, and agent-to-agent authentication and authorization as onboarding concerns. AWS Prescriptive Guidance uses this framing for multi-tenant environments.

That does not mean every agent needs a centralized platform on day one. A single internal Workflow may only need a configuration record, an access check, and a clean audit trail. The distinction matters when the same decisions must be consistent across multiple agents, workers, model providers, and tenants.

Why now: transient Context is the wrong database

Models operate on temporary Context: the instructions, messages, retrieved material, and Tool outputs provided for one inference. Context is powerful for reasoning, but it is a poor home for durable operational state. It can be truncated, assembled differently on retries, and cannot itself reliably enforce a tenant boundary or revoke an already-issued session.

Consider a request entering an interactive agent. Before execution, the platform may need to establish tenant identity; resolve the allowed agent version; select an approved model Provider; load a policy set; issue short-lived credentials; and decide whether the requested session is still active. The execution worker then uses those decisions to carry out the task. Afterward, the system may attach usage and evaluation results to the same identities and configuration versions.

Plain English: Keep long-lived facts—who may run what, for whom, and under which rules—outside the chat history. Pass only the necessary decisions into each run.

This separation also makes change safer. You can alter a Provider connection or an organization policy without mixing that change into repository-specific business logic. You can revoke a user’s session without asking a model to honor a sentence in its Context. And you can compare results from two evaluator versions without treating old scores as if they came from the same measurement.

How the boundary works, step by step

A practical control-plane design normally begins at request admission. An API gateway or service receives a request and authenticates its caller. The control plane maps that identity to a tenant, roles, scopes, and an allowed agent or Workflow. It resolves configuration: perhaps a particular model connection, an approved Tool set, a spending limit, and a policy version.

The execution plane receives a narrow execution contract rather than a vague global configuration. That contract can name the tenant, the selected agent version, permitted resources, session identifier, policy references, and correlation identifiers. The worker still performs model inference and Tool calls. Crucially, it should not silently decide that it belongs to another tenant or substitute an unapproved Provider connection.

Provider routing is a good example. It means selecting the model service or endpoint used for an execution based on managed rules and metadata. A model gateway primarily normalizes or routes inference requests. It can be part of a control plane, but it is not the whole thing: a broader plane also handles sessions, identities, policies, evaluations, and administrative state. Langfuse’s release notes specifically mention trusted-execution resolution and Provider-connection attribution in its AI gateway. Langfuse v4.35.0

Session lifecycle deserves the same treatment. Session creation, revocation, recovery, timeout, and termination are operational transitions, not generated text. If a session is revoked, every execution path that accepts its credential needs a consistent answer. A long-running background task also needs defined behavior: finish safely, checkpoint, stop, or move to an operator review state. User-triggered session revocation in Langfuse v4.34.0 illustrates why this belongs in durable platform state. Release notes

Finally, the plane records decisions and outcomes. Semantic observability means observing not only latency and errors, but also what an agent attempted, which Tools it called, and whether it met a task-oriented quality measure. OpenTelemetry’s GenAI semantic conventions provide a standardized telemetry vocabulary for AI and agent systems; they are an interoperability aid, not a control-plane architecture. OpenTelemetry The control plane can associate such records with tenant, configuration, session, Provider, and policy data.

What it is not

A control plane is not an Agent Harness. A Harness normally runs one agent loop: it assembles Context, calls a model, handles retries, and invokes Tools. A control plane governs shared, cross-run state around many such loops. You can have a strong Harness without a broad control plane, especially early on.

It is also not only capability mediation. Capability mediation is the security decision over whether an agent may invoke a particular resource or Tool. That decision may be configured and audited by a control plane, but sessions, evaluator administration, metering, and Provider attribution are broader concerns.

Nor is it Evaluation itself. Evaluation is the systematic measurement of an agent’s behavior against defined criteria. A control plane can version evaluator definitions, preserve score history, and decide where evaluations apply. It cannot make an agent correct merely by storing more scores. Semantic quality remains an application responsibility with test cases, review, and safeguards.

Plain English: The control plane does not reason for the agent. It makes agent operations governable and makes the decisions around a run inspectable.

A practical example: an internal coding platform

Imagine a company with an interactive coding agent and a background agent that reviews pull requests. Both can read repositories; only the review agent may publish a comment. Several business units use the platform, and different projects have different model budgets and approved integrations.

Without a control plane, each agent service may contain its own copy of project mapping, credential lookup, model selection, and budget logic. A session revocation could reach the interactive service but not a queued review job. A Provider switch could be changed in one code path and missed in another. Evaluator scores might be stored without recording which rubric created them.

With a control plane, onboarding creates a tenant identity, assigns projects and scopes, registers approved provider connections, and attaches policies. When a review job starts, it obtains an execution contract: the tenant, repository scope, agent version, permitted Tool capabilities, provider route, and budget. The review worker does the actual task. Its trace and usage records return with the resolved identifiers; an evaluator can then compare its output with the correct versioned criteria.

This design does not imply a single database or a single runtime. It means the authority for operational facts is explicit. For a small platform, that might be a few services and carefully versioned records. For a multi-provider environment, it may involve federating several Provider-owned control planes. AWS cautions that such environments can contain separate control planes with incomplete authority over one another, so trust, identity, policy, and lifecycle contracts must be explicit. AWS Prescriptive Guidance

Where the design breaks

Centralization creates a valuable target and a dependency. If the control plane is unavailable, can a safe in-flight job finish? Can a read-only request proceed from a cached contract? Those are product and security choices, not implementation details. Authorization bugs there can affect every agent at once.

State consistency is equally awkward. Revocation must race correctly with work already queued or running. Provider failover can change behavior and cost. A session timeline may not perfectly match event delivery during failures. Design these as explicit state transitions with identifiers, ownership, retries, and audit records—not as inferred conclusions from a model transcript.

Privacy is another limit. Centralized traces, scores, usage, and tenant metadata can create retention and data-residency obligations. Collect enough to operate safely, set clear retention rules, and avoid treating observability as permission to retain arbitrary user content.

What a senior engineer can do this week

First, draw the boundary in one existing system. Make two columns: execution decisions made during a run, and durable operational decisions that should survive it. Identity, policy assignment, session status, provider connections, evaluator versions, and usage accounting generally belong in the second column.

Second, define an execution contract for one Workflow. Do not start with a giant platform. Decide what a worker must receive, who resolves it, and which fields the worker may not override. Include tenant and scope identifiers from the beginning.

Third, make session revocation testable. Write down the expected result for an active interactive run, a queued task, and a long-running worker after revocation. The important outcome is not one universal behavior; it is a behavior that is deliberate, enforceable, and visible to operators.

Fourth, version your evaluation configuration alongside the agent configuration. Preserve the evaluator identity with every score. That small discipline prevents a common operational mistake: comparing results produced by different standards as though they were one metric.

Finally, instrument the boundary with stable telemetry vocabulary where it helps. OpenTelemetry’s GenAI conventions can support shared correlation across components, while leaving architecture and policy choices to your system. OpenTelemetry The goal is not to declare that every agent needs a control plane. It is to stop hiding cross-cutting operational authority inside prompts, workers, and scattered configuration.