Chapter 17 · Systems

Execution API — Calling the Harness over HTTP

The Execution API is how another system drives the harness over ordinary HTTP. It does three things: it runs an agent you hand it as a zip file (bundles), it runs a graph of steps you define (workflows), and it tells you which tools your credential is actually allowed to invoke (tools). Nothing gets deployed and nothing gets installed. This chapter is written for whoever has to call that API, and it assumes you have never seen this codebase. Most of it covers bundles, which is the deepest of the three; workflows and tool discovery have their own sections near the end.

Start here: what this actually does

Every other way of using the harness assumes the agent is already there. Someone put an AGENT.md file in a folder, the host found it when it started up, and from then on you can talk to it. That works when you own both sides.

The Bundle API is for when you don't. Your customer wrote the agent. Your CI job generates one per build. A tenant uploads their own. In all of those cases you can't redeploy the host every time the agent changes — so instead, the agent travels with the request.

i
The one-sentence version

You upload a zip containing an agent, the harness gives you a receipt, you use that receipt to start a run, and then you either poll for the answer or watch it stream in live.

Here is the whole flow. Four HTTP calls, and only the first two are mandatory:

the shape of every integration
  YOUR SYSTEM                                  THE HARNESS
      │
      │  1. POST /api/bundles                    stage the zip on disk,
      │     (your bundle.zip)          ────────▶  check it for nasties
      │  ◀────────────  { handle: "abc123" }     ← your receipt
      │
      │  2. POST /api/bundles/abc123/runs         work out what this caller
      │     { userMessages: [...] }     ────────▶  is allowed to do, then
      │  ◀────────────  { jobId: "job789" }        queue the conversation
      │                                          ← "accepted", NOT "done"
      │
      │  3. GET  .../runs/job789                  ...agent is thinking...
      │     (poll until it's finished) ────────▶
      │  ◀────────────  { status, result }       ← the answer
      │
      │  4. DELETE /api/bundles/abc123 ────────▶  throw the zip away
      │     (optional — it expires anyway)

Four words appear over and over in this chapter. Learn them now and the rest reads easily:

Bundle
The zip file you upload. Inside it is an AGENT.md describing the agent, plus any skill files it needs. It is just a folder, zipped.
Handle
The receipt you get back after uploading. A short opaque string like "a3f9c1…". It means "the copy of your zip that I unpacked and am holding for you." You pass it in the URL of every later call. One handle can start as many runs as you like.
Run (and its job id)
One conversation with the uploaded agent. Starting a run gives you a job id. The handle is the agent sitting on the shelf; the job is one time you actually used it.
Capability envelope
The list of things the harness will let this particular caller's agent do — which tools it may touch, how much freedom it has. You do not send this. The host works it out from who you are. There is a whole section on it below.

Is this the surface you want?

The harness exposes three HTTP-shaped front doors and they are not interchangeable. Pick by who authored the agent:

SurfaceHostUse it when
Bundle API Presentation.ExecutionApi The agent is authored outside the host and shipped in with the request. One-off runs, tenant-supplied agents, CI jobs that carry their own agent definition.
AgentHub Presentation.AgentHub The agent is already installed in the host and you want an interactive, stateful conversation (SignalR + AG-UI streaming, chat history). See its README.md.
MCP server Infrastructure.AI.MCPServer You want to hand the harness's tools to someone else's agent, not run an agent here. See Chapter 08.
!
Off by default, and it will not start half-configured

The subsystem is gated on AppConfig:AI:BundleExecution:Enabled (default false) — while it is off, every endpoint answers 403. Separately, the host is fail-closed on authentication: it refuses to boot unless you configure an Entra scheme (both TenantId and ClientId) or consciously set Auth:AllowAnonymous: true. Setting exactly one of tenant/client is treated as a mistake, not as an implicit request to serve openly, and also refuses to boot.

What each call sends and gets back

Here are the real payloads, so you know exactly what to expect before you write any code.

1 — Upload the bundle

A multipart/form-data POST, the same thing an HTML file-upload form sends. The form field must be named file — any other name is rejected.

http · POST /api/bundles → 201 Created
{
  "handle":    "9f2c1d4a8b6e47f0913c2d5e7a8b0c1d",
  "expiresAt": "2026-07-24T15:42:11.000+00:00"
}

expiresAt is the earliest the handle can disappear, not a fixed deadline — see Lifetimes below for why the clock keeps getting pushed back.

2 — Start a run

You send the messages the agent should respond to. userMessages is a list because you can seed a multi-turn conversation in one go.

http · POST /api/bundles/{handle}/runs — request body
{
  "userMessages": [ "Hi, I am Sam. What can you do?" ],
  "maxTurns": 10,
  "stream": false
}

Only userMessages is required (1–100 entries, none of them empty). maxTurns defaults to 10 and must be between 1 and 100; stream defaults to false and is explained under Two ways to get the output. A fourth field, conversationId, turns the run into a continuing session — see Continuing a conversation.

Continuing a conversation across runs

By default a run is self-contained: it answers the messages you send and forgets them. For a one-off question that is exactly what you want. For anything that runs as a session — a voice channel, a chat widget, a long-lived assistant — it means the agent starts every turn with no memory, so the only way to be understood is to resend the entire transcript each time. That gets more expensive and slower with every turn, and it stops working altogether once the conversation exceeds the 100-message cap.

Send a conversationId instead and the harness keeps the transcript for you. The agent is given the conversation's recent history before the first turn, and this run's turns are saved for the next one — so each request carries only what is new.

http · POST /api/bundles/{handle}/runs — continuing a session
{
  "userMessages": [ "And what about the second clause?" ],
  "conversationId": "voice-session-8f14e45f"
}

The id is yours to choose — a GUID is the obvious pick — and it is created the first time you use it. A few things worth knowing:

  • The conversation belongs to whoever created it. Naming a conversation that belongs to another caller returns 404, exactly as an unknown handle does, so nobody can probe for other people's session ids.
  • Turns never interleave. If a second run names a conversation that is mid-turn, it waits its turn rather than talking over it — including when the two runs land on different servers.
  • maxTurns and the 100-message cap bound one run, not the conversation. A session can run far longer than either. What bounds its total length is the conversation's lifetime token budget, which is cumulative across every run; when it is exhausted the run stops gracefully with budgetExhausted: true rather than failing.
  • Only recent history is replayed — the last AppConfig:AI:Conversations:MaxHistoryMessages messages (50 by default), so the prompt cannot grow without limit even though the stored transcript does.
http · → 202 Accepted
{
  "jobId":     "1a2b3c4d5e6f7081",
  "statusUrl": "/api/bundles/9f2c1d…/runs/1a2b3c4d5e6f7081",
  "streamUrl": null
}

statusUrl and streamUrl are relative — prepend your API base URL before using them. streamUrl is null unless you asked for stream: true.

!
202 means "accepted", not "finished"

This is the single most common misreading of this API. HTTP 200 OK would mean "here is your answer." 202 Accepted means "I have written your request down and I will get to it." The agent has not said anything yet. The response body contains no answer and never will — you have to go and fetch it in step 3.

3 — Get the result

Call statusUrl repeatedly (say, every second or two) until status stops being "Queued" or "Running". While it is still working you get:

http · GET statusUrl → 200 (still working)
{
  "jobId":       "1a2b3c4d5e6f7081",
  "status":      "Running",
  "error":       null,
  "createdAt":   "2026-07-24T15:12:03.000+00:00",
  "startedAt":   "2026-07-24T15:12:04.100+00:00",
  "completedAt": null,
  "result":      null
}

And when it is done, the same shape gains a result:

http · GET statusUrl → 200 (finished)
{
  "jobId":       "1a2b3c4d5e6f7081",
  "status":      "Succeeded",
  "error":       null,
  "createdAt":   "2026-07-24T15:12:03.000+00:00",
  "startedAt":   "2026-07-24T15:12:04.100+00:00",
  "completedAt": "2026-07-24T15:12:09.780+00:00",
  "result": {
    "conversationSucceeded": true,
    "finalResponse":         "Hello Sam — I can answer short questions concisely.",
    "turnCount":             1,
    "totalToolInvocations":  0,
    "budgetExhausted":       false,
    "conversationError":     null
  }
}

finalResponse is the agent's answer — that string is usually the only thing your application cares about. The rest is diagnostics. Note what is not here: the messages you sent, and the permissions the run executed under. The endpoint returns a fixed, deliberately narrow subset of the run's internal record, so polling can never be used to read back what went into the run — only what came out.

4 — Clean up (optional)

DELETE /api/bundles/{handle} returns 204 No Content and no body. You can skip it entirely — unused handles are swept automatically — but deleting promptly frees disk on the host and is good manners.

i
Handles and runs are separate on purpose

One handle can start any number of runs — upload once, ask ten different questions. The two also expire on separate clocks. Most importantly, the permissions a run executes under are decided when the run starts, from the credential that starts it — not when the bundle was uploaded, and not from whoever uploaded it. That one design decision is what makes a leaked handle harmless, and it is explained in full below.

Quickstart

This uses the anonymous development mode. Read the security section before pointing it at anything shared. The commands below need curl, zip, jq, and a bash-compatible shell — on Windows, Git Bash or WSL.

bash · build a minimal bundle
mkdir -p my-bundle/skills/greeter

cat > my-bundle/AGENT.md <<'EOF'
---
id: quickstart-agent
name: Quickstart Agent
description: Minimal externally-authored agent used to prove the bundle API end to end.
version: 1.0.0
skills: ["greeter"]
---

You are a concise assistant. Answer in at most three sentences.
EOF

cat > my-bundle/skills/greeter/SKILL.md <<'EOF'
---
name: "greeter"
description: "Greets the user and answers short questions."
version: "1.0.0"
---

Greet the user by name when they give one, then answer their question directly.
EOF

cd my-bundle && zip -r ../bundle.zip . && cd ..
bash · register, run, poll
# The host ships no launch profile, so pick the address explicitly:
#   ASPNETCORE_URLS=http://localhost:5000 dotnet run --project src/Content/Presentation/Presentation.ExecutionApi
API=http://localhost:5000

# Anonymous dev mode needs no token. Against a real Entra host, add
#   -H "authorization: Bearer $TOKEN"
# to every call below, where $TOKEN targets api://{ClientId}.

# 1 — register. The multipart field name MUST be "file".
HANDLE=$(curl -s -X POST "$API/api/bundles" -F "file=@bundle.zip" | jq -r .handle)

# 2 — start a run. 202 Accepted; the conversation has not happened yet.
JOB=$(curl -s -X POST "$API/api/bundles/$HANDLE/runs" \
  -H 'content-type: application/json' \
  -d '{"userMessages":["Hi, I am Sam. What can you do?"],"maxTurns":2}' | jq -r .jobId)

# 3 — poll until terminal.
curl -s "$API/api/bundles/$HANDLE/runs/$JOB" | jq '{status, result}'

# 4 — clean up (optional; the TTL sweeper would do it anyway).
curl -s -X DELETE "$API/api/bundles/$HANDLE" -o /dev/null -w '%{http_code}\n'

With a real Entra scheme configured, every call above additionally carries -H "Authorization: Bearer $TOKEN", where the token targets this API's own audienceapi://{ClientId}. The bundle API never shares an audience with the agent hub or the MCP server, because it runs code the host did not write.

What goes in the zip

A bundle is the same on-disk shape the harness already understands for an installed agent, so anything you learned in Chapter 05 applies unchanged:

bundle.zip
AGENT.md                  ← REQUIRED, at the archive root
skills/
  research/SKILL.md       ← optional; nested skills, owned by this agent alone
  summarise/SKILL.md
plugins/
  my-plugin/plugin.json   ← optional; parsed and carried, not yet wired into the run
plugin.json               ← optional; a root-level manifest is also read

Two hard requirements, both enforced at registration:

  • AGENT.md must sit at the archive root. Not in a subfolder. A zip built from a directory containing your bundle folder will be rejected — zip the folder's contents, as the quickstart does.
  • It must resolve to a non-empty id. The parser takes id: from the YAML front matter, falls back to name:, and falls back again to the folder name. A manifest with no front matter at all yields no id and is rejected.

Everything else is optional. A malformed SKILL.md is skipped with a warning rather than failing the whole bundle, and duplicate skill ids keep the first occurrence. Skills declared in a bundle are owned by that bundle's agent: they are resolved only for its own runs and never enter the host's global skill pool.

The security model: the bundle asks, the host grants

This is the most important section on the page. If you only properly read one, read this one.

Start with the problem. You are about to run a file that somebody else wrote, on your server. Inside that file, the author gets to write whatever they want — including a line that says which tools the agent is allowed to use:

AGENT.md · written by someone who is not you
---
id: totally-innocent-agent
allowed-tools: ["file_system", "shell", "http_client"]
---

If the harness simply believed that line, the security model would be "attackers, please declare only the permissions you ought to have." Obviously that cannot work. So the harness does the only sane thing:

!
Everything inside the bundle is a request, never a grant

allowed-tools in an uploaded AGENT.md is the bundle asking. What it actually gets is decided entirely on the host side, from configuration the bundle author cannot see or influence. The final permission set is the overlap of the two — asking for more than you were granted gets you nothing extra, and the request can only ever narrow the grant, never widen it.

The host-side half of that overlap is called the capability envelope. It is a block of config on the server that maps who is calling to what their agent may do:

json · AppConfig:AI:BundleExecution:Envelopes
{
  "Envelopes": {
    "Default": {
      "AllowedTools": [],
      "AllowedMcpServers": [],
      "AutonomyCeiling": "Restricted"
    },
    "BySubject": {
      "11111111-2222-3333-4444-555555555555": {
        "AllowedTools": [ "file_system", "calculation_engine", "docs_search" ],
        "AllowedMcpServers": [ "internal-docs" ],
        "AutonomyCeiling": "Autonomous"
      }
    },
    "ByRole": {
      "BundleRunner.ReadOnly": {
        "AllowedTools": [ "file_system" ],
        "AllowedMcpServers": [],
        "AutonomyCeiling": "Autonomous"
      }
    }
  }
}

Resolution precedence, in order:

  1. Exact subject match (BySubject) wins outright.
  2. Otherwise the caller's matching roles are combined to the least-privilege result — the intersection of tool and MCP allowlists and the minimum autonomy ceiling. Holding more roles can only ever narrow a grant, never widen it.
  3. Otherwise the Default applies — which grants nothing unless an operator widened it.

There is no code path on which "no envelope" means "no restriction". An unrecognised AutonomyCeiling string degrades to the most restrictive tier with a warning rather than opening up, and a numeric or comma-composite value is rejected outright for the same reason.

The two allowlists gate different things, and you almost always need both: AllowedMcpServers controls which servers may be contacted; AllowedTools controls which tools may be invoked — including the tools those servers publish. Granting internal-docs lets the run reach that server and put its tool schemas in front of the model; actually calling docs_search still requires docs_search in AllowedTools, which is why it appears in the example above alongside the two local tools.

!
Granting MCP servers but no tools grants nothing usable

An envelope with a non-empty AllowedMcpServers and an empty AllowedTools is the one combination that looks permissive and behaves inert. The run contacts every granted server, pays the round trip, publishes the fetched schemas to the model — and then denies every resulting call, because none of those tool names is on the invocation allowlist. The host logs a warning naming the granted servers when it resolves an envelope in that shape. It warns rather than refusing to start: the configuration fails closed, and an empty tool list is legitimate for a caller who is meant to have no tool access at all.

Worked through end to end, with the config shown above, a run looks like this:

how one run's permissions get decided
The uploaded AGENT.md asks for:   file_system, shell, http_client
The caller's token says:          sub = 1111…5555

  → host looks up BySubject["1111…5555"]
  → that envelope grants:         file_system, calculation_engine, docs_search
                                  (+ the internal-docs MCP server)

  overlap (ask ∩ grant)  =  file_system
  shell        → DENIED  (asked for, not granted)
  http_client  → DENIED  (asked for, not granted)
  calculation_engine     → granted, but the bundle never asked, so it is
                           simply never wired into the agent
  docs_search            → granted, and reachable via the granted MCP server,
                           but this bundle never asked for it either

The agent runs with exactly one tool: file_system.

The denials there are bypass-immune: no auto-approve mode, no configuration flag, and nothing the bundle can say will turn them back on. That is what makes it safe to run an agent whose author you do not trust.

!
A non-Autonomous ceiling suspends tool use entirely — it does not queue approvals

Live mid-tool-call approval routing is deferred, so the governor currently treats "this action requires human sign-off" as a fail-closed block. In practice that means a Restricted or Supervised ceiling stops the bundle from using tools at all, rather than gating each call for approval. A bundle that must actually do work therefore runs with AutonomyCeiling: "Autonomous" and is confined by AllowedTools / AllowedMcpServers plus the host's own risk and capability gates. This matches how the harness's plugin and tier baselines behave today — see Security · Autonomy & Governance for the tier model and Security · Tools & Permissions for the gates that still apply inside an Autonomous ceiling.

Which claim identifies you: oid vs sub

A JWT carries several different ids for the same person, and this API deliberately uses two different ones for two different jobs.

The symptom, when you get it wrong, is very specific: your BySubject grant looks correct in config but behaves as if it isn't there, and every run falls through to Default. That is almost always because the config was keyed on the wrong claim.

Two claims identify you, for two different jobs

Ownership of handles and runs keys on the first available of oid → object-identifier → nameidentifiersub → name. Envelope lookup (BySubject) keys on nameidentifier or sub only. For most Entra tokens these differ: oid is the directory object id, sub is a per-application pairwise identifier. If your BySubject grants appear to be ignored, you have almost certainly keyed them on the object id. Key them on the sub claim, or use ByRole with an app role — role values are read from the standard role claim and from the short roles/role claim names Entra emits.

Three more properties worth designing around:

  • MCP is allowlist-only and by name. A bundle references host-registered MCP servers by name and can never define an endpoint, so its entire outbound MCP surface is the list you granted. That closes SSRF by construction.
  • Ownership is enforced on every operation. Run, poll, stream, and delete all check that the caller owns the handle. A foreign or unknown resource is reported identically — 404 for reads, a silent 204 for delete — so the API never confirms that someone else's handle exists.
  • The grant follows the invoker, not the handle. A stolen handle runs under the thief's envelope, which is the point: it cannot be used to borrow the registrant's privileges.

Anonymous mode is a development mode

With Auth:AllowAnonymous: true the host authenticates every request as one synthetic principal and logs a prominent warning at startup for as long as it runs. The consequences are specific, and both matter:

  • Every caller shares one owner. Ownership checks still run, but they all pass — so there is no isolation between clients. Anyone can run, poll, and delete anyone's handle.
  • Every run gets the Default envelope. The synthetic principal carries no subject or role claims, so neither BySubject nor ByRole can match. If Default is unconfigured, anonymous runs are granted nothing — the door is open, the room is empty.

Endpoint reference

The full machine-readable contract — schemas, examples, every status code — is checked in as assets/openapi/bundle-api.yaml (OpenAPI 3.1). Generate a client from it. This host serves no runtime Swagger endpoint, unlike the agent hub.

!
This page and the spec are hand-maintained

This page and the spec are written by hand, not generated from the controller, so they can drift. If anything here disagrees with the running server, the server is right — please file an issue. (Changing the contract rather than consuming it? The canonical update list is Change the wire contract in the host's README.md.)

EndpointSuccessNotes
POST /api/bundles 201 Field name file. Location points at /api/bundles/{handle}, which answers only DELETE — there is no GET-a-bundle endpoint. 10 req/min.
POST /api/bundles/{handle}/runs 202 Body: userMessages (1–100, none empty), maxTurns (1–100, default 10), stream (default false), conversationId (optional; continues a stored conversation).
GET /api/bundles/{handle}/runs/{jobId} 200 Returns a fixed subset of fields — never echoes the envelope or your seed messages.
GET …/runs/{jobId}/stream 200 text/event-stream. Only valid for a run started with stream: true that is still Queued; otherwise 409.
DELETE /api/bundles/{handle} 204 Idempotent and non-disclosing — always 204.
Tools — discovery, and (opt-in) direct invocation
GET /api/tools 200 Lists the tools your credential may invoke — the intersection of what the host registers and what your envelope's AllowedTools grants. An empty list is a valid answer, not an error.
GET /api/tools/{name} 200 Describes one granted tool. A tool that doesn't exist and one you aren't granted both answer 404 — indistinguishably, by design.
POST /api/tools/{name}/invoke 200 Runs one operation of one tool and returns its result. Body: operation, optional parameters, optional timeoutSeconds. Needs the Harness.Tools.Invoke role and the tool in your envelope. Off by default403 until an operator enables it.
Workflows — submit a DAG, then run it
POST /api/workflows 201 Body: name, steps, edges, optional configuration. Every id is minted server-side. Cycles and unreachable steps are rejected here, not at first run.
POST /api/workflows/{id}/runs 202 No body. Returns a jobId and a statusUrl. 409 if this workflow already has a live run.
GET /api/workflows/{id}/runs/{jobId} 200 A projection — never echoes the envelope or tenant the run holds.
GET …/runs/{jobId}/stream 200 text/event-stream. First frame is always a SNAPSHOT. 503 when the host or you are at the stream ceiling.
DELETE /api/workflows/{id}/runs/{jobId} 200 Cancels the run and withdraws any approval it was waiting on. stopped: false means signalled, not yet stopped. 409 if already terminal.
Evals — run the host's own test suites
GET /api/evals/datasets 200 The names you are allowed to run. Empty list on a host with no dataset roots configured — that is the answer, not an error.
POST /api/evals/runs 202 Body: datasets (names, never paths), optional repeats, parallelism, tagFilter, failRateThreshold. Returns a jobId and a statusUrl. 404 if a name is not one this host serves.
GET /api/evals/runs/{jobId} 200 Status, plus counts/verdict/duration/cost once finished. Not the per-case transcripts.
DELETE /api/evals/runs/{jobId} 200 Cancels a run that has not started. stopped: false means it was already executing and will finish — an eval in flight cannot be interrupted. 409 if already terminal.

Finding out what you may actually invoke

The envelope section above explains that the host decides what you may use. It does not tell you what the host decided. GET /api/tools answers that directly, and it is the first call worth making before you write a workflow with a ToolUse step.

curl -sS -H "Authorization: Bearer $TOKEN" http://localhost:5000/api/tools
{
  "tools": [
    {
      "name": "document_search",
      "description": "Searches ingested documents.",
      "operations": ["search"],
      "riskTier": "Low",
      "isReadOnly": true,
      "isConcurrencySafe": true,
      "isDirectlyInvocable": true
    }
  ]
}

The name is the exact string a ToolUse step must supply, and operations tells you which values that step's operation field will accept — an invocation naming anything else is rejected by the tool itself.

i
An empty list is an answer, not a failure

The shipped default envelope grants nothing, so an operator who has not configured a grant for your credential leaves you with {"tools": []} and a 200. That is the fail-closed default working as intended. Ask your operator for an Envelopes entry rather than assuming the host is broken. Equally, this listing is yours — it is not the host's inventory, and another credential will legitimately see a different set.

Two absences you may notice and should not chase. Tools that drive an interactive client — dashboard_control and the render_* family — cannot be constructed in this host at all, because there is no client on the other end to render into; they are omitted rather than advertised as callable. And /api/tools is not the same thing as the agent hub's /api/mcp/tools: that one lists tools published by external MCP servers, which additionally require their server to be granted.

Running a single tool directly

Everything else in this chapter hands the host your work — a bundle, a workflow, a suite — and lets an agent decide which tools it needs along the way. This endpoint is the exception: you name the tool and the operation, and the host runs it. That is genuinely useful when you want one answer and a whole workflow would be ceremony. It is also the most privileged thing this API can do, so it is deliberately the hardest to switch on.

!
Off by default — and unlike everything else here, it stays off

Bundle execution and workflow submission ship enabled in this host, because serving them is what it is for. Direct invocation ships disabled, and the difference is the point: those surfaces run an agent that chooses its own tools, whereas this one lets a caller point at host-side code and pull the trigger. An operator has to set AppConfig:AI:DirectToolInvocation:Enabled deliberately. Until then every call answers 403, including yours with the right role.

Three separate things must all be true before a tool runs:

  1. You are authenticated.
  2. You hold the Harness.Tools.Invoke role. Listing tools does not need it — seeing what the host could do and making it do it are different grants, and an operator can hand out the first without the second.
  3. Your envelope's AllowedTools names the tool. This one is checked twice, by two independent mechanisms.
curl -sS -X POST http://localhost:5000/api/tools/file_system/invoke \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"operation": "read", "parameters": {"path": "notes/todo.md"}}'
{
  "tool": "file_system",
  "operation": "read",
  "succeeded": true,
  "output": "- ship T2",
  "outputTruncated": false,
  "durationMs": 12
}

The status code and succeeded mean different things

This trips people up, so it is worth being blunt about. The HTTP status describes the invocation; succeeded describes the tool's verdict. A tool that ran perfectly and decided the answer was no returns 200 with "succeeded": false. Every 4xx and 5xx below means the tool never ran at all.

  • 400 — the request is malformed, or names an operation the tool does not declare. The error names the operations it does accept.
  • 401 — no credential, or a credential carrying nothing usable as an identity. Some identity providers emit sub values with characters the harness cannot use as a permission subject; the remedy is the same either way, which is to present a different token.
  • 403 — you lack the role, or the host has this surface off, or governance refused an invocation of a tool you genuinely are granted.
  • 404 — no such tool, or not granted to you, or not offered on this surface. Deliberately indistinguishable, for the same reason GET /api/tools/{name} is.
  • 413 — your request body exceeded the host's cap (64 KiB by default), refused before it was read.
  • 429 — you already have the maximum number of invocations executing at once (4 per caller). This endpoint is capped by concurrency rather than request rate, because an invocation holds a server thread for its whole duration and a per-minute cap would not stop calls that are still running when the window rolls. Retry when one of yours finishes.
  • 504 — the tool outran its deadline (30 s by default) and was cancelled. That deadline covers the whole invocation, not just the tool: authorization and the data-classification check happen under it too.

The most confusing 403 is the third kind. If your envelope's autonomyCeiling is Supervised or Restricted, tool execution is suspended entirely — mid-run approval routing is not built yet, so anything needing approval is refused rather than queued. Only Autonomous permits invocation today. That is a property of the envelope, not of this endpoint, and it applies to bundle and workflow runs identically.

Some tools are listed but not invocable here

Check isDirectlyInvocable in the catalog before you build against a tool. When it is false it is not saying you lack permission — it is saying the tool's result would be meaningless to you. The render_* family and dashboard_control return drawing instructions for a browser attached to a live agent run, and there is no browser on your end of an HTTP call. delegate_task is excluded for a different reason: it starts whole agent turns, so one apparently-simple synchronous call would expand into open-ended work that outlives your request. Those tools stay listed because a ToolUse step in a workflow can still use them perfectly well.

What comes back has been through a filter

Output is scrubbed by the host's response sanitizers before it reaches you, and so is a failing tool's error message — error text is the likeliest place a filesystem path or a connection string leaks out. Long output is cut at the host's ceiling and outputTruncated is set to true; check that field, because a prefix that does not announce itself will parse as a complete answer. Output is never compressed: the harness's compression exists to squeeze results into a model's context window and does it by summarising and leaving pointers an agent can expand, and you have no way to follow those.

Workflows: running a DAG instead of a conversation

A bundle run is a conversation. A workflow is a directed graph of steps — model calls, tool calls, retrieval, conditional branches, and human approval gates — that the harness executes for you. Use a workflow when the shape of the work is known in advance and you want it enforced; use a bundle when you want an agent to decide.

Submission and execution are separate calls. Submitting stores a definition under your ownership and costs nothing; running it spends the host's model and tool credentials.

# 1. Submit — the id comes back from the server, you do not choose it
curl -sS -X POST http://localhost:5000/api/workflows \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{
        "name": "draft-and-review",
        "steps": [
          {"name": "draft",  "type": "LlmCall",
           "configuration": {"type": "LlmCall", "prompt": "Draft a summary."}},
          {"name": "review", "type": "LlmCall",
           "configuration": {"type": "LlmCall", "prompt": "Check it for errors."}}
        ],
        "edges": [{"from": "draft", "to": "review", "type": "ControlFlow"}]
      }'

# 2. Run it — returns 202 and a job id
curl -sS -X POST http://localhost:5000/api/workflows/$WORKFLOW_ID/runs \
  -H "Authorization: Bearer $TOKEN"

# 3. Poll, or stream
curl -sS -H "Authorization: Bearer $TOKEN" \
  http://localhost:5000/api/workflows/$WORKFLOW_ID/runs/$JOB_ID
!
One live run per workflow, and Blocked counts as live

Starting a second run while one is queued, running, or parked on a human gate returns 409. This is not a throttle you should retry past — two runs of one workflow would share a single execution state machine, so the second would re-execute the first's live steps and adopt its outputs. Wait for the first to finish, or cancel it.

That makes status worth reading carefully. Treat only Succeeded, Failed, and Cancelled as final. Blocked means the run is parked on a human approval gate: it keeps its job id, keeps its hold on the workflow, and resumes on that same id once someone answers the gate. A client that treats Blocked as an ending will give up on a run that was about to continue.

Streaming a workflow run

GET …/runs/{jobId}/stream opens a text/event-stream. Unlike the bundle stream, you may attach at any time — the first frame is always a SNAPSHOT of the run as it stands, so there is no window in which you have to reconcile what you missed.

FrameMeaning
SNAPSHOTAlways first. The run's state when you attached; isTerminal: true means it had already finished and nothing further follows.
STEPA step started or completed. sequence is monotonic within the run.
FINISHEDTerminal. The stream closes after it.
PARKEDThe run hit a human gate. The stream closes but the run has not ended — re-attach after the gate is answered.
GAPYou fell behind and droppedCount frames were discarded. Reported rather than hidden, so a slow client never mistakes a truncated stream for a complete one.

One property to design around: authorization is checked when the stream opens, not continuously. A token revoked mid-run keeps receiving until the run ends. The exposure is bounded by the run's duration, and the frames carry nothing you could not already read from the status endpoint — but it is a deliberate trade, not an oversight.

Evals: running the host's own test suites

The other three surfaces run your work — your bundle, your workflow, your tool call. This one runs the host's. An evaluation suite is a file of test cases an operator has put on the server, and running it asks the harness to answer every case and score itself. You use it to check that a change to prompts, models, or tools did not make things worse.

It is off unless someone turned it on, and gated on a role. Every endpoint below 403s until AppConfig:AI:Evaluation:Enabled is set, and the host refuses to start if that is set without at least one dataset root. On top of that you need the Harness.Evals.Execute role — this is the only surface in this chapter that asks for one. The reason is that every other route runs your work on your grant, whereas this one spends the host's model budget on the operator's suites. Holding a valid token is not the same authority as being allowed to do that.

The role does not change ownership: it decides whether you may evaluate at all, and you still only ever see your own runs.

You name a dataset; you never give a path

This is the part worth understanding, because it looks like an inconvenience and is actually the whole security design. The command underneath takes file paths — which is right for the command-line runner, where a developer points it at a file on their own machine. Over HTTP that same shape would mean every request carried a filesystem reference, and the only thing standing between a caller and reading /etc/shadow would be a check remembering to say no.

So the wire contract has no path field at all. You send a name; the server finds the file by listing the directories an operator configured and matching what is actually there. A name cannot express “outside those directories”, so the dangerous request is not rejected — it cannot be written down. Ask for what is available first:

curl -sS -H "Authorization: Bearer $TOKEN" http://localhost:5000/api/evals/datasets
# { "datasets": ["router-accuracy", "safety-refusals"] }

Those names are the files sitting directly in a configured root, without their extensions. Subdirectories are not searched: a name with a slash in it would be a path again, just wearing a different hat.

Start a run, then poll it

JOB_ID=$(curl -sS -X POST http://localhost:5000/api/evals/runs \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"datasets":["router-accuracy"],"repeats":3}' | jq -r .jobId)

curl -sS -H "Authorization: Bearer $TOKEN" \
  http://localhost:5000/api/evals/runs/$JOB_ID

The 202 means the run was accepted, not that it finished — a suite takes as long as it takes. Once it is done the poll carries a report: how many cases passed, failed, warned, or errored, the overall verdict, how long it took, and what it cost in dollars.

The report is also filed into the host's own evaluation store automatically, so it shows up in the dashboard without you posting anything back — that is the main thing running a suite server-side buys you over running the CLI.

The poll does not carry the per-case transcripts. Those contain every test input and the agent's full response, which is a lot of data to put on a status check and more of the agent's output than any other endpoint here exposes. If you need transcripts, read the report files the evaluation framework writes on the server.

Succeeded does not mean everything passed. It means the evaluation ran. A suite where every case failed is still a succeeded run — one whose report says "verdict": "Fail". The distinction matters: if the two were collapsed you could not tell “my prompt change broke things” from “the host is broken and never answered me”.

Two limits to know about

Cost is capped before anything runs, not interrupted afterwards. The host multiplies the number of cases by your repeats and refuses the whole run if that exceeds MaxCaseExecutionsPerRun. It refuses rather than quietly running a subset, because a pass rate for a suite that only half-ran is worse than no answer.

Cancelling only works before the run starts. DELETE on a queued run stops it. On a run already executing you get 200 with "stopped": false, and it finishes anyway. That is deliberate honesty rather than a gap: a workflow can be signalled to stop mid-flight, an evaluation cannot, and telling you it had stopped when the spend was still accruing would be the worse answer. The real protection is the cost cap above.

Two ways to get the output

Background + poll (the default)

Your run joins a queue, and a background worker — the dispatcher — picks runs off that queue and executes them. You find out how it went by polling.

One property of the dispatcher matters a great deal when you set your poll interval: it drains the queue one run at a time. Your run waits behind every other queued run on that host. So the time to a result is not just how long the model takes to think — it is that plus however long the queue ahead of you takes. Budget for both.

Poll until status is Succeeded or Failed, then read result. The distinction that catches people:

i
Succeeded describes the run, not the answer

A conversation that completed but whose agent reported a failed turn is status: "Succeeded" with result.conversationSucceeded: false and the reason in result.conversationError. Only an unhandled exception, a handle that expired before the run started, or cancellation produces status: "Failed" — and then the reason is in the top-level error, always a stable scrubbed string, never a raw exception. A correct client checks both fields.

Live stream (opt in with stream: true)

Instead of polling, you hold one long-lived HTTP connection open and the server pushes output down it as the agent produces it. The format is Server-Sent Events (SSE) — a plain-text streaming format with the content type text/event-stream, supported by every browser and most HTTP clients.

A streaming run behaves differently from a background one in a way that surprises people: it is reserved, not queued. Nothing runs when you POST it. The run sits waiting, and opening streamUrl is what actually starts the agent. Closing that connection cancels the run. If you never connect at all, the reservation is thrown away after StreamReservationTtl (default 5 minutes).

Frames are data: {json}\n\n, camelCase, with a type discriminator taken from the AG-UI wire vocabulary — a hand-written client that switches on type works unchanged. A strict AG-UI client library is not guaranteed to: tool-call frames can arrive while a TEXT_MESSAGE is still open (see below, some AG-UI client libraries reject that), and TOOL_CALL_RESULT's fields have no AG-UI precedent to match against.

Frame typePayloadWhen
RUN_STARTEDthreadId (the handle), runId (the job id)Once, first.
TEXT_MESSAGE_STARTmessageId, role: "assistant"Lazily, on the first real token. A run that emits no text emits no message frames at all.
TEXT_MESSAGE_CONTENTmessageId, deltaPer token batch. Append to your buffer.
TEXT_MESSAGE_ENDmessageIdOnce, if the message was opened.
TOOL_CALL_STARTtoolCallId, toolCallNameWhen the agent decides to call a tool. May arrive before the first TEXT_MESSAGE_START.
TOOL_CALL_ARGStoolCallId, delta, withheld (only present when true)Immediately after TOOL_CALL_START — the complete arguments JSON in one frame, not an incremental delta.
TOOL_CALL_ENDtoolCallIdImmediately after TOOL_CALL_ARGS.
TOOL_CALL_RESULTtoolCallId, result (redacted preview)Once the tool has actually run.
RUN_FINISHEDthreadId, runIdTerminal, on success.
RUN_ERRORmessage (caller-safe)Terminal, on failure. Never both this and RUN_FINISHED.

Two things about the tool-call frames that are easy to assume wrong: the delta field on TOOL_CALL_ARGS is named to match the AG-UI wire vocabulary, but it is not incremental — the model-provider integration only ever exposes a tool call's arguments once fully assembled, so all three of TOOL_CALL_START, TOOL_CALL_ARGS, and TOOL_CALL_END arrive back-to-back in one burst. And tool-call frames freely interleave with text frames, including arriving before the assistant message ever opens — a client cannot assume text comes first.

TOOL_CALL_ARGS.delta and TOOL_CALL_RESULT.result are never the raw payload — both pass through the same secret-redaction the harness applies before persisting tool activity to its observability store, since a live stream to a browser is just as much an exposure point as disk. They differ in how they bound size: result is truncated to a fixed preview length, which is safe for free text. delta is never truncated — cutting JSON mid-token would hand the client invalid data — so above a size ceiling the real arguments are withheld instead: delta becomes the fixed placeholder "{}" and withheld is true. A client should treat a withheld frame as "arguments unavailable," not attempt to interpret "{}" as the tool's real (empty) arguments. If a tool call fails, result is a generic failure message, not the underlying exception text.

ts · consuming the stream
async function runStreamed(API: string, handle: string, token: string): Promise<string> {
  const start = await fetch(`${API}/api/bundles/${handle}/runs`, {
    method: 'POST',
    headers: { 'content-type': 'application/json', authorization: `Bearer ${token}` },
    body: JSON.stringify({ userMessages: ['Walk me through it.'], maxTurns: 5, stream: true }),
  });
  const { streamUrl } = await start.json();

  // Opening this is what runs the agent. EventSource cannot send an Authorization
  // header, so use fetch + a stream reader when the host requires a bearer token.
  const res = await fetch(`${API}${streamUrl}`, { headers: { authorization: `Bearer ${token}` } });
  if (!res.body) throw new Error(`Stream failed: ${res.status}`);
  const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();

  let buffer = '', answer = '';
  for (;;) {
    const { value, done } = await reader.read();
    if (done) break;
    buffer += value;

    // Frames are separated by a blank line.
    let cut;
    while ((cut = buffer.indexOf('\n\n')) !== -1) {
      const frame = buffer.slice(0, cut).replace(/^data: /, '');
      buffer = buffer.slice(cut + 2);
      const evt = JSON.parse(frame);
      if (evt.type === 'TEXT_MESSAGE_CONTENT') answer += evt.delta;
      if (evt.type === 'TOOL_CALL_START') console.log(`calling ${evt.toolCallName}…`);
      if (evt.type === 'RUN_ERROR') throw new Error(evt.message);
      if (evt.type === 'RUN_FINISHED') return answer;
    }
  }

  // The stream ended without a terminal frame — the connection dropped mid-run.
  throw new Error('Stream ended before RUN_FINISHED or RUN_ERROR.');
}

Streams are capped by concurrency, not rate: each open connection holds a permit for its whole lifetime, and a caller may hold MaxConcurrentStreamsPerCaller (default 4) at once. The excess is rejected with 429 immediately rather than queued, so a client that opens streams in a loop must bound itself.

What the host does with your archive

Nothing in the zip is parsed until every structural guard has passed, and any failure deletes the partial extraction before returning. Each rejection is a 400 whose detail names the guard — never the archive's contents.

GuardDefault limitSetting
Archive size on the wire10 MiBMaxArchiveBytes (also caps the multipart body)
Entry count2 000MaxEntryCount
Total uncompressed size50 MiBMaxTotalUncompressedBytes
Compression ratio100×MaxCompressionRatio — only applied once a bundle expands past 1 MiB, so small, highly-compressible markdown bundles are never caught by it
Zip-slip (../ or absolute entry paths)rejected
Symlinks resolving outside the staging directoryrejected
Not a zip / empty archiverejected

The decompression-bomb check runs twice: once cheaply against the archive's declared header sizes, then again against actual bytes written during extraction — so an archive that lies about its entry lengths still trips it.

Accepted bundles are extracted into their own unique subdirectory of TempRoot. There is one rule about where that root may live, and it matters:

!
TempRoot must not sit inside a skill or agent discovery path

The host scans its discovery folders recursively to find skills. So if you stage uploaded bundles inside one of those folders, the host will happily walk into the staging area, find a customer's private skills, and publish them globally to every agent on the box. That is a serious leak, so the staging service checks for the overlap up front and simply refuses to stage rather than letting it happen.

Errors you should handle

StatusMeansDo
400Archive guard rejection, or an invalid run request (no messages, empty message, maxTurns out of range, malformed conversationId — letters, digits, hyphens and underscores only, up to 200 characters).Read detail; fix the payload. Not retryable as-is.
401No/invalid token, or a principal with no usable identity to own resources under.Re-authenticate against api://{ClientId}.
403Bundle execution is disabled on this host.Operator action — Enabled: true. Do not retry.
404Handle, run, or named conversation is unknown, expired, or not yours. These are deliberately indistinguishable, so the API never confirms that someone else's resource exists.Re-register the bundle to get a fresh handle; check the conversationId is one you created.
409The run is not awaiting a stream — it is a background run, already being streamed, or already finished.Poll the status endpoint instead.
42910/min register; 60/min run, poll, and delete; the stream endpoint is capped by open connections instead.Back off; for streams, close one first.
500Unexpected server error. The body is deliberately generic.The detail is in the host's logs. Retry idempotently.

Lifetimes

ClockDefaultBehaviour
HandleTtl30 minSliding — refreshed on every lookup and every run start. expiresAt is a floor, not a deadline.
RunRecordTtl30 minHow long a completed run stays pollable before it is swept. Read your result inside this window.
StreamReservationTtl5 minHow long an unclaimed streaming reservation waits for you to connect. Independent of RunRecordTtl on purpose.
CleanupInterval60 sHow often the sweeper deletes expired staging directories and run records.

An in-flight run holds a lease on its staged bundle, so the sweeper can never delete the directory out from under a running agent.

!
Nothing here is durable, and the host is not clustered

Handles, run records, and results live in memory on one host. Two consequences follow, and both will bite you in production if you don't plan for them. Restart the host and every in-flight and completed run is gone. Put a load balancer in front of two instances and your poll will eventually land on a machine that has never heard of your job.

So until these stores are made durable, your client has to be the system of record. Concretely, that means two things: save the final response yourself the moment you read it, and keep one caller pinned to one instance — or just run a single instance.

Configuration reference

All of it lives under AppConfig:AI:BundleExecution.

KeyDefaultPurpose
EnabledfalseMaster toggle. Off ⇒ every endpoint 403s.
TempRoot"" (system temp)Staging root. Must be disjoint from all skill/agent discovery paths.
MaxArchiveBytes10485760Wire size cap; also bounds the multipart body.
MaxEntryCount2000Entry-count cap.
MaxTotalUncompressedBytes52428800Absolute expansion cap.
MaxCompressionRatio100Expansion-factor cap (above a 1 MiB floor).
HandleTtl00:30:00Sliding handle lifetime.
RunRecordTtl00:30:00How long a terminal run stays pollable.
StreamReservationTtl00:05:00How long an unclaimed stream reservation survives.
MaxConcurrentStreamsPerCaller4Open streams one caller may hold.
CleanupInterval00:01:00Sweeper period.
Envelopesfail-closedThe per-caller grant table (see above).
Auth:TenantId / Auth:ClientIdunsetThis API's own Entra audience. Supply both or neither.
Auth:AllowAnonymousfalseExplicit local-development opt-in. Cannot be combined with a configured scheme.

The eval endpoints read a separate section, AppConfig:AI:Evaluation.

KeyDefaultPurpose
EnabledfalseMaster toggle for /api/evals. Off ⇒ every endpoint 403s.
DatasetRoots[]Directories datasets may be read from. Use absolute paths. Setting Enabled without any root refuses to start.
MaxDatasetsPerRun10Datasets one run may name.
MaxCaseExecutionsPerRun500Cases × repeats — what the run actually costs. Refuses, never truncates.
MaxDatasetBytes5242880Checked before the file is opened, so an oversized dataset is never parsed.
MaxRepeats50Ceiling on repeats.
MaxParallelism128Ceiling on parallelism.

Running the host

bash
ASPNETCORE_URLS=http://localhost:5000 \
  dotnet run --project src/Content/Presentation/Presentation.ExecutionApi

The shipped appsettings.json enables bundle execution but leaves Auth empty, so a fresh clone boots only in Development, where the shipped appsettings.Development.json opts into anonymous auth. Every other environment trips the fail-closed guard above at startup until you configure Entra. Outside Development the host also enables HSTS and HTTPS redirection.

Troubleshooting

SymptomCause
Host throws at startup about authenticationFail-closed guard. You configured neither a scheme nor AllowAnonymous, set exactly one of TenantId/ClientId, or set both a scheme and AllowAnonymous (contradictory).
Every call returns 403Enabled is false.
Register returns 400 "Bundle has no AGENT.md at its root."You zipped the parent folder. Zip the bundle directory's contents.
Register returns 400 about the staging rootTempRoot overlaps a skill or agent discovery path.
Runs succeed but the agent never uses a toolThe envelope. Either the tool is not in AllowedTools, or the ceiling is not Autonomous — see the autonomy callout.
BySubject grant appears ignoredKeyed on oid instead of sub/name-identifier.
Stream returns 409The run was not started with stream: true, or something already claimed it.
Poll returns 404 for a job you just startedDifferent caller identity than the one that registered the handle, a restarted host, or the record was swept.

Where to go from here