What is this thing?

You've probably chatted with an AI assistant. You type a question, it thinks, it answers. Simple, right? Under the hood there's a whole orchestration system making that work — and most people have no idea it exists.

Why should you care? When your agent goes off the rails, knowing what the harness does helps you diagnose where things broke — not just that something did.

What happens when you send a message?

You hit Send. Your message takes a journey through five actors before a reply comes back. Step through the flow below.

It's not just a chatbot

Anyone can glue a text box to an API call. A real harness earns its name by handling five hard problems that a raw API call ignores.

The conversation loop, in plain English

Every one of those capabilities hangs off one short piece of code — the outer loop that runs a conversation turn by turn.

public async Task<ConversationResult> Handle(RunConversationCommand request, CancellationToken cancellationToken)
{
    _logger.LogInformation("Starting conversation with {AgentName}, {MessageCount} messages, max {MaxTurns} turns",
        request.AgentName, request.UserMessages.Count, request.MaxTurns);
 
    var sw = Stopwatch.StartNew();
    var turns = new List<TurnSummary>();
    var totalToolInvocations = 0;
    AgentTurnResult? lastResult = null;
 
    foreach (var (userMessage, index) in request.UserMessages.Select((m, i) => (m, i)))
    {
        if (index >= request.MaxTurns)
        {
            _logger.LogWarning("Max turns ({MaxTurns}) reached for {AgentName}", request.MaxTurns, request.AgentName);
            break;
        }
 
        var turnCommand = new ExecuteAgentTurnCommand
        {
            AgentName = request.AgentName,
            UserMessage = userMessage,
            ConversationHistory = lastResult?.UpdatedHistory ?? [],
            ConversationId = request.ConversationId,
            TurnNumber = index + 1
        };
 
        lastResult = await _mediator.Send(turnCommand, cancellationToken);
    }

The tech stack at a glance

Five technologies do most of the heavy lifting. You don't need to know them inside-out — just recognize the names when they come up.

Quick check: do you get it?

Meet the Cast

In the last module, you watched a message travel through the whole system. Now let's zoom in and meet each of the characters who made that journey happen.

How Actors Collaborate

When you ask the agent to read a file, a whole cast of characters springs into action. Watch them talk to each other in real time.

Notice how the LLM doesn't touch the file system directly. It asks for a tool, and the handler checks permissions before executing it. Every actor stays in its lane.

The Domain Layer: Blueprints

The Domain layer defines what things are without knowing how they work. Here are two of its most important blueprints.

AgentManifest — The Character Sheet

public class AgentManifest
{
    public string Id { get; set; } = string.Empty;
    public string Name { get; set; } = string.Empty;
    public string Description { get; set; } = string.Empty;
    public string Instructions { get; set; } = string.Empty;
    public IList<SkillReference> Skills { get; set; } = new List<SkillReference>();
    public IList<string> AllowedTools { get; set; } = new List<string>();
    public IList<ToolDeclaration> ToolDeclarations { get; set; } = new List<ToolDeclaration>();
    public StateConfiguration? StateConfiguration { get; set; }
    public DecisionFramework? DecisionFramework { get; set; }
    // plus Version, Author, Domain, Category, Tags, file-system metadata
}
        

SkillDefinition — Three Tiers of Detail

Skills use progressive disclosure across three levels: Level 1 (Index Card, ~50-100 tokens), Level 2 (Folder, ~5,000-token instructions body), and Level 3 (Filing Cabinet, on-demand resource files). The agent sees Level 1 in a menu, opens Level 2 only for the skill it picks, and never auto-loads Level 3 — it reads those files only via file-system tools.

public class SkillDefinition
{
    #region Level 1: Index Card (Metadata — Always Loaded)

    public string Id { get; set; } = string.Empty;
    public string Name { get; set; } = string.Empty;
    public string Description { get; set; } = string.Empty;
    public string Category { get; set; } = string.Empty;
    public IList<string> Tags { get; set; } = new List<string>();

    #endregion
        

The Application Layer: The Director

The Application layer is where the action happens. It reads the Domain blueprints, calls Infrastructure services, and orchestrates everything. The AgentFactory is the director who assembles the full cast.

public async Task<AIAgent> CreateAgentAsync(AgentExecutionContext agentContext, CancellationToken cancellationToken = default)
{
    var chatClient = await _chatClientFactory.GetChatClientAsync(clientType, deploymentOrAgentId, cancellationToken);

    var chatClientBuilder = chatClient.AsBuilder()
        .UseOpenTelemetry(configure: c => c.EnableSensitiveData = true)
        .UseFunctionInvocation(configure: c => { c.AllowConcurrentInvocation = true; c.MaximumIterationsPerRequest = 5; })
        .Use(inner => new Middleware.ObservabilityMiddleware(inner, …))
        .Use(inner => new Middleware.ToolDiagnosticsMiddleware(inner, …))
        .UseDistributedCache(_distributedCache);

    var middlewareEnabledChatClient = chatClientBuilder.Build();
    var agent = new ChatClientAgent(middlewareEnabledChatClient, agentOptions);

    _logger.LogInformation("Creating agent {AgentName} using {ClientType} with {Deployment}",
        agentContext.Name, clientType, deploymentOrAgentId);

    return agent.AsBuilder().UseOpenTelemetry(…).Build();
}
        

Infrastructure & Presentation

Two outer layers remain. Infrastructure does the heavy lifting with real services. Presentation is the front door.

Check Your Understanding

Now that you know who's who, let's watch them in action — follow a real conversation from the first message to the final answer.

The Conversation Loop

You know that moment when an AI assistant pauses, calls a tool, gets results, and then keeps going? That is not one step — it is an entire loop running behind the scenes, and every iteration goes through a gauntlet of checks.

Think of it like a chess game with a coach. The agent (the player) studies the board and picks a move. The coach (the harness) checks the move is legal. Then the opponent (reality, via a tool call) responds. The game continues until checkmate — or the clock runs out.

The Gauntlet: 15 Pipeline Behaviors

Before the handler ever touches your message, it must survive a gauntlet of fifteen checkpoints. These are called pipeline behaviors, registered across two layers: ten AI-specific behaviors (outermost) wrap five common behaviors (innermost). Together they run in this exact order every time.

This pattern is called CQRS with MediatR. Every request is a “command” that gets routed through behaviors before reaching a handler. The order matters — UnhandledException is outermost so it can catch errors from anything inside; Timeout is innermost so it only times the actual work, not the safety checks.

ExecuteAgentTurn — One Round

This is the single most important piece of code in the harness. It handles one turn — send messages to the LLM, check if it asked for tools, and report back whether we are done.

public async Task<AgentTurnResult> Handle(ExecuteAgentTurnCommand request, CancellationToken cancellationToken)
{
    var skillId = _agentRegistry.TryGet(request.AgentName)?.Skill ?? request.AgentName;

    var agent = await _agentCache.GetOrCreateAsync(
        request.ConversationId, skillId,
        new SkillAgentOptions {
            AdditionalContext = request.SystemPromptOverride,
            DeploymentName = request.DeploymentOverride,
            Temperature = request.Temperature
        }, cancellationToken);

    var messages = new List<ChatMessage>(request.ConversationHistory)
    {
        new(ChatRole.User, request.UserMessage)
    };

    _usageCapture.TakeSnapshot();
    LlmUsageCapture.Current = _usageCapture;

    var response = await agent.RunAsync(messages, cancellationToken: cancellationToken);

    var usage = _usageCapture.TakeSnapshot();
    var responseText = ExtractResponseText(response);

    return new AgentTurnResult {
        Success = true, Response = responseText,
        UpdatedHistory = [..messages, ..response.Messages],
        ToolsInvoked = usage.ToolsInvoked,
        InputTokens = usage.InputTokens, OutputTokens = usage.OutputTokens,
        CostUsd = usage.CostUsd, Model = usage.Model
    };
}
        

RunConversation — The Full Loop

The previous screen showed one turn. This is the loop that chains turns together — keep going until the agent finishes or we hit the turn limit.

foreach (var (userMessage, index) in request.UserMessages.Select((m, i) => (m, i)))
{
    if (index >= request.MaxTurns)
    {
        _logger.LogWarning("Max turns ({MaxTurns}) reached", request.MaxTurns);
        break;
    }

    var turnCommand = new ExecuteAgentTurnCommand
    {
        AgentName = request.AgentName,
        UserMessage = userMessage,
        ConversationHistory = lastResult?.UpdatedHistory ?? [],
        TurnNumber = index + 1,
        ObservabilitySessionId = dbSessionId
    };

    lastResult = await _mediator.Send(turnCommand, cancellationToken);

    if (!lastResult.Success) break;
}
        

Going Multi-Agent: The Orchestrator

One agent running a conversation loop is powerful. But what if the task is too big for one agent? The orchestrator breaks it into subtasks, hands each one to a specialist sub-agent, runs them one after another, then synthesizes their results into a final answer.

// Phase 1: Build the orchestrator and ask it to plan
var orchestrator = await _agentFactory.CreateAgentFromSkillAsync(
    request.OrchestratorName,
    new SkillAgentOptions { AdditionalContext = BuildAgentCatalogPrompt(request.AvailableAgents) },
    cancellationToken);

var planResponse = await orchestrator.RunAsync(planMessages, cancellationToken: cancellationToken);
var subtasks = ParseSubtasks(ExtractResponseText(planResponse), request.AvailableAgents);

// Phase 2: Run sub-agents sequentially, sharing one budget
foreach (var (agentName, subtask) in subtasks)
{
    await using var scope = _scopeFactory.CreateAsyncScope();
    var mediator = scope.ServiceProvider.GetRequiredService<IMediator>();

    var conversationResult = await mediator.Send(new RunConversationCommand
    {
        AgentName = agentName, UserMessages = [subtask],
        MaxTurns = Math.Min(5, request.MaxTotalTurns - totalTurns)
    }, cancellationToken);

    subAgentResults.Add(new SubAgentResult(agentName, subtask, conversationResult.Response));
}

// Phase 3: Synthesize — orchestrator reads all sub-agent answers and writes the final response
var synthesis = await orchestrator.RunAsync(BuildSynthesisMessages(subAgentResults), cancellationToken: cancellationToken);
        

Test Your Understanding

Skills — What Agents Know

An LLM has a finite memory called a context window. Dump everything into it and the agent drowns. Load too little and it doesn't know how to help.

Think of the context window as a glass jar. Every piece of information the agent needs is a colored section that takes up space. When the jar overflows, older information gets pushed out and the agent forgets.

The ContextBudgetTracker is the person watching the jar. It tracks how many tokens each section uses and raises a warning when the jar is almost full.

The Spy Briefcase — Three Compartments

Imagine a spy's briefcase with three compartments. Each one holds a different level of detail about a mission. You never open Compartment 3 during planning — that would overwhelm you.

In the code, these three compartments are defined in a single class called SkillDefinition. Each tier is marked with a region comment so developers know which level it belongs to.

public class SkillDefinition
{
    #region Level 1: Index Card (Metadata — Always Loaded)
    public string Id { get; set; } = string.Empty;
    public string Name { get; set; } = string.Empty;
    public string Description { get; set; } = string.Empty;
    #endregion

    #region Level 2: Folder (Instructions — On Demand)
    public string? Instructions { get; set; }
    public string? Objectives { get; set; }
    public string? TraceFormat { get; set; }
    #endregion

    #region Categorization
    public string? Category { get; set; }
    public IList<string> Tags { get; set; } = new List<string>();
    public string? SkillType { get; set; }
    #endregion

    #region Runtime Configuration
    public IList<string> AllowedTools { get; set; } = new List<string>();
    #endregion

    #region Level 3: Filing Cabinet (Resources — On Demand)
    public IList<SkillResource> Templates { get; set; } = new List<SkillResource>();
    public IList<SkillResource> References { get; set; } = new List<SkillResource>();
    public IList<SkillResource> Scripts { get; set; } = new List<SkillResource>();
    public IList<SkillResource> Assets { get; set; } = new List<SkillResource>();
    #endregion
    // Plus Extensibility (ToolDeclarations), Hierarchy, Loading State, Computed Properties regions
}
        

SKILL.md — A Skill's Mission File

Each skill is written as a Markdown file with a special header section. The header uses YAML frontmatter (the structured bit between the --- lines), and the body contains the actual instructions.

---
name: "orchestrator-agent"
description: "Coordinates specialized agents to accomplish complex, multi-step tasks."
category: "orchestration"
skill_type: "orchestration"
version: "1.0.0"
tags: ["orchestrator", "multi-agent", "coordination"]
allowed-tools: ["file_system", "web_search"]
---

# Your Role

You are a task orchestrator. When given a complex task:
1. Break it into independent subtasks
2. Assign each subtask to the most appropriate agent
3. Synthesize results into a coherent answer

## Task Decomposition Format
- Each subtask must have: description, assigned agent, expected output
- Prefer parallel subtasks over sequential when possible

## Guidelines
- Never execute subtasks yourself — always delegate
- If a subtask fails, retry with a different agent or approach
        

The Budget Tracker — Loading the Right Compartment

Something has to decide which compartment of the briefcase to open, based on what the budget allows. We'll call that logic the TieredContextAssembler — a teaching name for the tier-loading decision. (In today's code the tiering is enforced by per-tier allocations recorded in the real ContextBudgetTracker and composed by the agent factory; there's no single class by this name. The Tier 1 / 2 / 3 idea below is exactly what happens.)

public sealed class TieredContextAssembler
{
    private const int DefaultTier1MaxTokens = 3000;
    private const int DefaultTier2MaxTokens = 8000;

    public Task<AssembledContext> AssembleContextAsync(
        SkillDefinition skill,
        string? basePath = null,
        CancellationToken cancellationToken = default)
    {
        var agentName = skill.Id;

        // Tier 1: org-level files declared in context_loading frontmatter
        var tier1 = LoadTier1(skill, agentName);   // truncates oversized files at budget boundary
        _budgetTracker.RecordAllocation(agentName, "tier1_context", tier1.TotalTokens);

        // Tier 2: domain/activity files — always loaded, also bounded by budget
        var tier2 = LoadTier2(skill, agentName);
        _budgetTracker.RecordAllocation(agentName, "tier2_context", tier2.TotalTokens);

        // Tier 3: NEVER preloaded — just return the lookup config
        var tier3 = BuildTier3Config(skill);  // returns Tier3AccessConfig with paths + fallback prompt

        return Task.FromResult(new AssembledContext(tier1, tier2, tier3, FormatPromptSection(tier1, tier2, tier3)));
    }
        

Behind the Scenes: The Budget Negotiation

Here's what happens inside the system when the agent activates a skill. Three components "talk" to each other to decide what fits in the jar.

Compaction — When Memory Gets Full

When the budget tracker fires a warning, the system needs to make room. Compaction summarizes older messages into shorter versions — like replacing a 30-minute meeting recording with a half-page of notes.

The CompactionBoundaryMessage records what happened — how many tokens were saved, what strategy was used, and a summary of the compacted content. This creates an audit trail so nothing is silently lost.

Check Your Understanding

How Tools Get to the Agent

Skills tell the agent what it knows. Tools are what it can actually do. Think of the harness like a Swiss Army knife with a safety lock — each blade is sharp and useful, but there is a mechanism that prevents you from opening the dangerous ones without explicit permission.

Watch each step below to see how a tool travels from a skill declaration all the way into the agent’s hands.

The Tool Resolution Chain

Here is the real C# method that decides where each tool comes from. The pattern is: try MCP first, fall back to keyed DI, then try a fallback tool if one is declared.

private async Task<IEnumerable<AITool>?> ProvisionToolAsync(ToolDeclaration declaration)
{
    // Try MCP first
    if (_mcpToolProvider != null)
    {
        try
        {
            var mcpTools = await _mcpToolProvider.GetToolsAsync(declaration.Name);
            if (mcpTools?.Count > 0)
            {
                _logger.LogDebug("Resolved tool {ToolName} from MCP server", declaration.Name);
                return mcpTools;
            }
        }
        catch (Exception ex)
        {
            _logger.LogDebug(ex, "MCP resolution failed for {ToolName}, trying keyed DI", declaration.Name);
        }
    }

    // Fallback to keyed DI
    var resolved = ResolveToolByName(declaration.Name);
    if (resolved != null)
        return resolved;

    // Try fallback tool
    if (declaration.HasFallback && !declaration.FallbackIsManual)
    {
        resolved = ResolveToolByName(declaration.Fallback!);
        if (resolved != null)
        {
            _logger.LogInformation("Using fallback tool {Fallback} for {ToolName}",
                declaration.Fallback, declaration.Name);
            return resolved;
        }
    }

    if (!declaration.Optional)
        _logger.LogWarning("Required tool {ToolName} could not be resolved", declaration.Name);

    return null;
}
        

The Sandbox — Keeping Tools on a Leash

Giving an AI agent unrestricted access to your file system is a terrible idea. The harness puts a sandbox around dangerous tools so they can only operate inside allowed base paths.

public class ToolPermissionBehavior<TRequest, TResponse>
    : IPipelineBehavior<TRequest, TResponse>
    where TRequest : notnull
{
    public async Task<TResponse> Handle(TRequest request, RequestHandlerDelegate<TResponse> next, CancellationToken ct)
    {
        // Runtime check — only requests that opt in get gated
        if (request is not IToolRequest toolRequest) return await next();

        // 3-phase resolution via ThreePhasePermissionResolver:
        // Phase 1: Deny gates — absolute blocks (%WINDIR%, Program Files, System32)
        // Phase 2: Ask rules — requires user confirmation
        // Phase 3: Allow rules — permitted operations
        var decision = await _resolver.ResolveAsync(toolRequest, ct);
        return decision.Allowed ? await next() : CreateRejection(decision);
    }
}
        

MCP — Plugging Into the World

MCP is the universal adapter for agent tools. Instead of hardcoding every capability, the agent can discover tools on remote servers at runtime — like plugging a USB device into your laptop and having it just work.

Watch the conversation below to see how an MCP client discovers and invokes a tool on a remote MCP server.

A2A — When Tools Are Not Enough

MCP lets an agent borrow tools from other servers. But what if the task is too complex for a single tool call? A2A goes further — instead of borrowing a tool, the agent delegates an entire task to another agent.

public record AgentCard

{
    public required string Name { get; init; }
    public required string Description { get; init; }
    public string? Url { get; init; }
    public IReadOnlyList<string> Capabilities { get; init; } = [];
    public IReadOnlyList<string> Skills { get; init; } = [];
    public string? Version { get; init; }
}
        

Check Your Understanding

Seeing Inside & Staying Safe

AI agents are black boxes by default. A conversation goes sideways and you have no idea why — was it the prompt? A bad tool result? A context overflow? The harness solves this by instrumenting everything with OpenTelemetry, so you can trace any conversation turn-by-turn, tool call by tool call.

Think of a hospital monitoring room. Every patient (agent conversation) has vital signs on a screen — heart rate is token usage, blood pressure is response latency, temperature is error rate. When something spikes, alarms fire.

The Three Pillars of Observability

Every production monitoring system rests on three pillars. The harness implements all three out of the box.

Custom LLM Span Processor

The harness adds a custom span processor that enriches every trace with agentic context — agent name, turn index, tool calls, and token counts. This means when you open Jaeger, you see the agent’s perspective, not just raw HTTP calls.

public static class OrchestrationMetrics
{
    public static Histogram<double> ConversationDuration { get; } =
        AppInstrument.Meter.CreateHistogram<double>(
            OrchestrationConventions.ConversationDuration, "{ms}");

    public static Histogram<int> TurnsPerConversation { get; } =
        AppInstrument.Meter.CreateHistogram<int>(
            OrchestrationConventions.TurnsPerConversation, "{turn}");

    public static Counter<long> SubagentSpawns { get; } =
        AppInstrument.Meter.CreateCounter<long>(
            OrchestrationConventions.SubagentSpawns, "{spawn}");

    public static Counter<long> ToolCalls { get; } =
        AppInstrument.Meter.CreateCounter<long>(
            OrchestrationConventions.ToolCalls, "{call}");

    public static Histogram<double> TurnDuration { get; } =
        AppInstrument.Meter.CreateHistogram<double>(
            OrchestrationConventions.TurnDuration, "{ms}");

    public static Counter<long> TurnsTotal { get; } =
        AppInstrument.Meter.CreateCounter<long>(
            OrchestrationConventions.TurnsTotal, "{turn}");

    public static Counter<long> TurnErrors { get; } =
        AppInstrument.Meter.CreateCounter<long>(
            OrchestrationConventions.TurnErrors, "{turn}");
}
        

Content Safety: The Bouncer at the Door

Before any message reaches the agent, it passes through a content safety gate. Think of a nightclub bouncer — every person (message) gets checked at the door. If something looks wrong, they never get inside. The agent never even knows the message existed.

Watch both scenarios play out: a flagged message that gets blocked, then a clean message that sails through.

The Code Behind the Bouncer

ContentSafetyBehavior is a pipeline behavior — it sits in the MediatR pipeline and inspects every command that implements IContentScreenable.

public class ContentSafetyBehavior<TRequest, TResponse>
    : IPipelineBehavior<TRequest, TResponse>
    where TRequest : notnull
{
    public ContentSafetyBehavior(ITextContentSafetyService safety, IObservabilityStore observabilityStore)
    { _safety = safety; _observabilityStore = observabilityStore; }

    public async Task<TResponse> Handle(TRequest request, RequestHandlerDelegate<TResponse> next, CancellationToken ct)
    {
        // Runtime opt-in: only requests that implement IContentScreenable get screened
        if (request is not IContentScreenable screenable) return await next();

        var content = screenable.ContentToScreen;
        ContentSafetyMetrics.Evaluations.Add(1);  // every screen, pass or fail

        if (!string.IsNullOrWhiteSpace(content))
        {
            var result = await _safety.ScreenContentAsync(content, ct);

            if (!result.IsSafe)
            {
                _logger.LogWarning("Content safety block: {Categories}", string.Join(", ", result.Categories));
                await _observabilityStore.RecordSafetyEventAsync(screenable, result, ct);
                ContentSafetyMetrics.Blocks.Add(1);

                // Prefer returning a structured Result.ContentBlocked rather than throwing
                if (ResultHelper.TryCreateFailure<TResponse>(result, out var failure)) return failure;
                throw new ContentSafetyException(result.Reason, result.Category);
            }
        }

        return await next();
    }
}
        

The Permission System: Three Layers of Defense

Content safety catches harmful content. But what about harmful actions? An agent with access to a file deletion tool needs guardrails beyond “is this text safe?” The permission system controls what tools an agent is allowed to use, using a 3-phase resolution: Deny first, then Ask, then Allow.

The order matters. Deny rules always win. This is a security pattern called “default deny” — nothing gets through unless it is explicitly allowed, and certain things can never get through.

public sealed record SafetyGate(string PathPattern, string Description)
{
    public bool IsBypassImmune => true;
}

public sealed record ToolPermissionRule(
    string ToolPattern,
    string? OperationPattern,  // nullable — omit to match any operation
    PermissionBehaviorType Behavior,
    PermissionRuleSource Source,
    int Priority,
    bool IsBypassImmune = false);
        

The Meta-Harness: An Agent That Optimizes Other Agents

Remember how skills (Module 4) are instruction files that shape agent behavior? The meta-harness reads the primary agent’s causal traces, finds patterns in what worked and what did not, and proposes improvements to those skill files automatically.

It is a four-phase optimization loop — like a coach reviewing game tape and updating the playbook.

// Phase 1: Load benchmark tasks and prior learnings from disk
var evalTasks = await LoadEvalTasksAsync(cfg.EvalTasksPath);
var priorLearnings = await LoadPriorLearningsAsync(cfg.LearningsPath);

// Phase 2: Build a proposer context, ask the proposer for a candidate
var proposerCtx = new HarnessProposerContext {
    CurrentCandidate = currentBest,
    OptimizationRunDirectoryPath = runDir,
    PriorCandidateIds = candidateHistory, Iteration = iteration,
    PriorLearnings = priorLearnings
};
var candidate = await _proposer.ProposeAsync(proposerCtx, cancellationToken);

// Phase 3: Evaluate the candidate against the eval task set
var evaluated = await _evaluationService.EvaluateAsync(candidate, evalTasks, cancellationToken);

// Phase 4: Accept only if better AND passes regression check
if (IsBetter(evaluated, currentBest, cfg.ScoreImprovementThreshold)
    && _regressionService.Check(regressionSuite, evaluated))
{
    await _candidateRepository.SaveAsync(evaluated, cancellationToken);
    _logger.LogInformation("New best candidate: pass rate {Rate} (was {Previous})",
        evaluated.PassRate, currentBest.PassRate);
}
        

The regression suite is the key safeguard here. The meta-harness never ships a change that makes the agent worse — it only keeps strict improvements. This is how autonomous agents can evolve without human babysitting.

The Full Picture

You have now seen every layer of the harness. A user message enters the system and passes through content safety, gets routed to an agent, which loads its skills, picks its tools, calls the LLM, executes tool calls, and returns a response — all while traces, metrics, and logs record every step. And above it all, the meta-harness watches, learns, and improves.

That is the Microsoft Agentic Harness. Not just an agent — a complete system for building, running, monitoring, and continuously improving AI agents.

Final Quiz: Test Your Understanding

Five questions covering concepts from across the course. These are scenario-based — there is no “scroll up and find the answer.” You need to think about what you learned.

Welcome to the Tour Bus

The previous six modules gave you the map. This one is a guided ride — four real requests, traced through the actual files, line by line, with every stop labeled.

Until now we have talked about the harness — the cast, the pipeline, the skills system. From here on we follow real requests through real files. Each walkthrough is one route. At every stop we will tell you the exact file path, what is happening, and why it matters.

Think of it like riding the bus through a city you have only seen on a map. You already know that the courthouse is downtown and the library is on Main Street. Now you actually pass each building, the driver points it out, and suddenly the map makes sense.

Before we leave the station — meet the cast

Each walkthrough involves a recurring cast of components talking to each other. Here is a quick refresher in their own voice.

Walkthrough A — "Summarize My README"

You type a sentence and hit Send. Nine files run before you see the first token of the response. Let's follow it.

The whole journey happens inside one roundtrip: browser POSTs one AG-UI run request, harness streams events back over Server-Sent Events until the run finishes. Tool calls inside that roundtrip do not become new roundtrips — they are looped invisibly by the framework middleware. By the time the SSE stream closes, all reasoning, all tool calls, and all evaluations are already done.

The nine stops

The heart of stop 4 — in code

Below is the actual handler that fired at stop 4. It is the single most important method in the WebUI request path: build the agent, run it, capture token usage, return the result.

public async Task<AgentTurnResult> Handle(ExecuteAgentTurnCommand request, CancellationToken cancellationToken)
{
    var skillId = _agentRegistry.TryGet(request.AgentName)?.Skill ?? request.AgentName;

    var agent = await _agentCache.GetOrCreateAsync(
        request.ConversationId, skillId,
        new SkillAgentOptions { AdditionalContext = request.SystemPromptOverride,
            DeploymentName = request.DeploymentOverride, Temperature = request.Temperature },
        cancellationToken);

    var messages = new List<ChatMessage>(request.ConversationHistory)
        { new(ChatRole.User, request.UserMessage) };

    _usageCapture.TakeSnapshot();
    LlmUsageCapture.Current = _usageCapture;

    response = await agent.RunAsync(messages, cancellationToken: cancellationToken);

    var usage = _usageCapture.TakeSnapshot();
    var responseText = ExtractResponseText(response);
}
        

Test your understanding

Walkthrough B — Loading a Skill

A folder of markdown files becomes a tool the LLM can pick from a menu — with strict token budgets enforced at every level.

Skills are how the harness teaches the LLM new abilities without ballooning the prompt. The trick is progressive disclosure: three tiers of detail, only the cheapest tier loads up front.

Where skills live on disk

The eight stops

The parser, in code

The parser at stop 3 is small but does the heavy lifting. Notice how everything maps directly to fields on SkillDefinition — no fancy YAML library, just careful string slicing.

public SkillDefinition ParseFromFile(string skillFilePath, string sourcePath)
{
    var raw = File.ReadAllText(skillFilePath);
    var frontmatter = ExtractFrontmatter(raw);
    var body = ExtractBody(raw, frontmatter);

    var (objectives, traceFormat, instructions) = ExtractStructuredSections(body);

    return new SkillDefinition
    {
        Id = name,  Name = name,  Description = description,
        Instructions = instructions,
        Category = ParseString(frontmatter, "category"),
        Tags = ParseList(frontmatter, "tags"),
        AllowedTools = ParseList(frontmatter, "allowed-tools"),
        FilePath = skillFilePath, IsFullyLoaded = true
    };
}

private static string? ExtractFrontmatter(string raw)
{
    if (!raw.StartsWith("---", StringComparison.Ordinal)) return null;
    var end = raw.IndexOf("---", 3, StringComparison.Ordinal);
    return end < 0 ? null : raw[3..end];
}
        

Test your understanding

Walkthrough C — The LLM Calls file_system

The LLM says "I need to read README.md." Eight stops later, the file content is back in the conversation history. Most of the stops exist to keep the LLM from escaping the sandbox.

This walkthrough is the most "computer-sciencey" of the four because tools rely on keyed dependency injection and reflection-based schema generation. But the path is short and the sandbox is the prize.

The eight stops

The sandbox check, in code

The single most security-critical method in the entire harness lives at stop 7. Every read, write, list, and search calls it first. The order of operations matters — resolve symlinks before validating the prefix.

private string ResolveAndValidate(string path, bool write = false)
{
    ArgumentException.ThrowIfNullOrWhiteSpace(path);

    // Defense-in-depth: reject traversal patterns, null bytes, shell injection
    if (!SecureInputValidatorHelper.ValidateFilePath(path))
        throw new ArgumentException("Path contains invalid characters or traversal patterns.");

    var fullPath = Path.IsPathRooted(path)
        ? Path.GetFullPath(path)
        : ResolveRelative(path);

    // Resolve symlinks/junctions to real target, then re-validate
    fullPath = ResolveSymlinks(fullPath);

    if (!IsPathAllowed(fullPath))
    {
        _logger.LogWarning("Blocked access to path outside sandbox: {Path}", fullPath);
        throw new UnauthorizedAccessException("Path is outside the allowed sandbox.");
    }

    if (write) ValidateWriteTarget(fullPath);
    return fullPath;
}
        

Test your understanding

Walkthrough D — "What Does the README Say About RAG?"

Ten stops. Eight services. Two searches running in parallel. One math formula that does the magic at the end.

RAG is the harness's most layered subsystem. Most production RAG fails because someone wires up one piece (vector search) and calls it done. The harness layers eight pieces to recover from each one's weaknesses.

The flow visualized

The ten stops

The RRF math, in code

Stop 7 is where the magic happens. The whole function is twenty lines and gives you provably-better retrieval than either method alone.

// chunkScores: Dictionary<string, (Chunk, DenseScore, SparseScore, FusedScore)>

for (var rank = 0; rank < denseResults.Count; rank++)
{
    var result = denseResults[rank];
    var rrfScore = 1.0 / (rrfK + rank + 1);
    chunkScores[result.Chunk.Id] = (result.Chunk, result.DenseScore, 0.0, rrfScore);
}

for (var rank = 0; rank < sparseResults.Count; rank++)
{
    var result = sparseResults[rank];
    var rrfScore = 1.0 / (rrfK + rank + 1);
    if (chunkScores.TryGetValue(result.Chunk.Id, out var existing))
        chunkScores[result.Chunk.Id] = (existing.Chunk, existing.DenseScore,
            result.SparseScore, existing.FusedScore + rrfScore);
    else
        chunkScores[result.Chunk.Id] = (result.Chunk, 0.0, result.SparseScore, rrfScore);
}

return chunkScores.Values
    .OrderByDescending(x => x.FusedScore)
    .Take(topK)
    .Select(x => new RetrievalResult(x.Chunk, x.DenseScore, x.SparseScore, x.FusedScore))
    .ToList();
        

Test your understanding

The Patterns You Keep Seeing

Four walkthroughs, dozens of files, but only a handful of patterns. Now that you have seen them in action, here is the synthesis.

Every codebase has its own dialect, but mature C# codebases tend to settle on the same handful of structural patterns. The harness uses five. Spot any of them in another project and you will know roughly where to look.

Final scenario quiz — combine what you've seen

The Quality Loop

Five new systems shipped in the last work cycle. Individually they each fix one problem. Together they form a feedback loop that lets the harness watch itself, catch itself getting worse, and learn from corrections.

So far the course has shown a harness that runs agents well. This module is about a harness that improves agents over time — without you sitting there grading every output.

Think of an autoimmune system. White blood cells (governance) check every request. When something looks risky, they flag it for a doctor (escalation). Long-term blood panels (drift detection) catch slow degradation that no single test would notice. The body remembers what made it sick before (learnings). And the autonomic nervous system (autonomy tiers) decides how much you have to think about any of this consciously. The five systems below are the harness's version of that.

Meet the loop — in their own voice

The five systems talk to each other on every interesting turn. Here is roughly what that conversation looks like when an agent does something governance doesn’t love.

Autonomy Tiers — The Trust Dial

Every agent has one knob that governs how much freedom it gets. Three positions: Restricted, Supervised, Autonomous.

Without autonomy tiers, you have two bad options: every agent asks for permission on every action (annoying, slow) or every agent runs unsupervised (terrifying). The tier is the per-agent compromise — a new internal-tools agent might start Restricted, a production support agent might be Supervised with a handful of pre-approved tools, and a fully-vetted research agent can be Autonomous.

In code it’s a single enum — deliberately small so it can be compared with >= for “requires at least tier X” checks elsewhere in the harness.

public enum AutonomyLevel
{
    /// Read-only tier. Default behavior is Ask, forcing approval for every action.
    /// Safety gates handle true Deny scenarios.
    Restricted = 0,

    /// Recommend-and-wait tier. Default is also Ask, but Supervised agents
    /// can have specific tool Allow overrides via ToolOverrides in tier policy.
    Supervised = 1,

    /// Act-within-guardrails tier. Default is Allow. Safety gates and AGT
    /// policies still apply as a ceiling above the tier's baseline.
    Autonomous = 2
}
        

Governance — The Rulebook Engine

Six decisions, one pipeline behavior, a YAML policy file. Sits at position 7 of the pipeline — between tool permission and the hook layer.

Governance answers: “Should this specific tool call, by this specific agent, on these specific arguments, proceed?” The answer is one of six verdicts.

Inside GovernancePolicyBehavior.Handle

The pipeline behavior at position 7 of the gauntlet. Note the early bail-out if the request isn’t a tool call, and the fail-closed behavior when the engine isn’t available.

public async Task<TResponse> Handle(TRequest request,
    RequestHandlerDelegate<TResponse> next, CancellationToken cancellationToken)
{
    if (request is not IToolRequest toolRequest)
        return await next();

    if (!_config.CurrentValue.Enabled || !_policyEngine.HasPolicies)
        return await next();

    var agentId = _executionContext.AgentId ?? "unknown";

    var decision = _policyEngine.EvaluateToolCall(agentId, toolRequest.ToolName);

    if (_config.CurrentValue.EnableAudit)
        _auditService.Log(agentId, toolRequest.ToolName, decision.Action.ToString());

    if (decision.IsAllowed) return await next();

    if (decision.Action == GovernancePolicyAction.RequireApproval)
        return await HandleRequireApprovalAsync(agentId, toolRequest, decision, next, cancellationToken);

    if (ResultHelper.TryCreateFailure<TResponse>(nameof(Result.GovernanceBlocked),
            decision.Reason, out var blocked)) return blocked;

    throw new InvalidOperationException($"Governance policy denied: {decision.Reason}");
}
        

Quick check

Escalation — Pause and Ask a Human

When governance returns RequireApproval, the agent pauses. The UI receives a EscalationRequestedEvent over Server-Sent Events. A human votes. The agent resumes — or fails closed.

This is the single most concrete benefit of the whole governance stack: human-in-the-loop without the agent permanently blocking on every call. The cost of being safe is one extra SSE event when something risky happens.

The shape of an escalation event

This is what the browser actually receives when an agent triggers approval. It is a JSON object with a polymorphic type discriminator so multiple event types can travel the same stream.

public sealed record EscalationRequestedEvent : AgUiEvent
{
    [JsonPropertyName("escalationId")]
    public required string EscalationId { get; init; }

    [JsonPropertyName("agentId")]
    public required string AgentId { get; init; }

    [JsonPropertyName("toolName")]
    public required string ToolName { get; init; }

    [JsonPropertyName("description")]
    public required string Description { get; init; }

    [JsonPropertyName("priority")]
    public required string Priority { get; init; }

    [JsonPropertyName("approvers")]
    public required IReadOnlyList<string> Approvers { get; init; }

    [JsonPropertyName("timeoutSeconds")]
    public required int TimeoutSeconds { get; init; }

    [JsonPropertyName("arguments")]
    public IReadOnlyDictionary<string, string>? Arguments { get; init; }
}
        

Quick check

Drift Detection & Learnings — The Long Memory

Two systems that work as a pair: drift catches slow quality regressions; learnings remember the corrections so the same mistake isn’t repeated.

Drift Detection — the canary

After each evaluation run, quality scores (faithfulness, relevance, structural correctness, …) feed into an EWMA running average. Each new score updates the EWMA via the formula EWMAt = λ · xt + (1 - λ) · EWMAt-1. The current EWMA is compared to a stored baseline; if it has drifted by more than N sigma, an alert fires.

var lambda = config.EwmaLambda;
var previousEwma = existingState?.CurrentEwma ?? baselineMean;

var newEwma = lambda * currentValue + (1 - lambda) * previousEwma;
var deviation = sigma > 0
    ? Math.Abs(newEwma - baselineMean) / sigma
    : 0.0;

var updatedState = new EwmaState
{
    Scope = baseline.Scope,
    ScopeIdentifier = baseline.ScopeIdentifier,
    Dimension = dimension,
    CurrentEwma = newEwma,
    SampleCount = newSampleCount,
    LastUpdatedAt = _timeProvider.GetUtcNow()
};

var saveResult = await _stateStore.SaveStateAsync(updatedState, ct);
        

The resulting deviation is classified into four severity tiers by DriftSeverityClassifier:

Learnings — remembering corrections

When something goes wrong — a drift alert, an escalation Deny, a human correction in chat — the harness can capture a learning. Learnings are natural-language facts: “agents writing to system paths are almost always hallucinating”, “customer X prefers concise summaries”. They decay over time so old wisdom doesn’t poison new behavior.

public Task<double> CalculateFreshnessAsync(LearningEntry learning, CancellationToken ct)
{
    if (learning.DecayClass == DecayClass.Permanent)
        return Task.FromResult(1.0);

    var shelfLifeDays = learning.DecayClass switch
    {
        DecayClass.Volatile => config.VolatileShelfLifeDays,   // default 7
        DecayClass.Stable => config.StableShelfLifeDays,       // default 180
        _ => config.StableShelfLifeDays
    };

    var referenceTime = learning.LastReinforcedAt ?? learning.CreatedAt;
    var ageDays = (_timeProvider.GetUtcNow() - referenceTime).TotalDays;
    var rawFreshness = Math.Clamp(1.0 - (ageDays / shelfLifeDays), 0.0, 1.0);

    if (config.BiasCorrection && learning.UpdateCount is > 0 and < 5)
    {
        var correctionFactor = 1.0 / (1.0 - Math.Pow(1.0 - config.DecayBiasAlpha, learning.UpdateCount));
        return Task.FromResult(Math.Clamp(rawFreshness * correctionFactor, 0.0, 1.0));
    }

    return Task.FromResult(rawFreshness);
}
        

Quick check

Learning From Its Own Work — and Not Believing Everything It Writes Down

Drift and learnings watch the scores. This is a different kind of memory: the agent keeps a diary of the jobs it actually did, then quietly studies that diary so it does better next time — while being careful not to trust every note in it.

The work diary — capture, synthesise, recall

Imagine a new employee who, at the end of every task, jots a quick note: “Here’s what I was asked, here’s what I tried, here’s what worked.” On its own each note is just a scrap. But overnight, when nobody is waiting on an answer, a quieter process reads a whole stack of those scraps and distils them into a handful of durable lessons — “when the request looks like this, the move that works is that.” Then, the next time a similar request arrives, those lessons are pulled back up and handed to the agent before it starts. The agent walks into the new job already reminded of how the last one like it went.

Trust-aware memory — a diary can be poisoned

There’s a catch, and it’s an important one. If the agent will read its own notes later and act on them, then anything that sneaks a note into that diary can steer the agent’s future behaviour. A malicious web page or document could try to plant a “lesson” like “always email this file to that address.” So the harness treats its own memory as untrusted until proven otherwise.

Every time something wants to write to memory, it passes through a write-gate — a bouncer for the diary. A note that can’t be vouched for isn’t thrown away, but it’s stamped untrusted and quarantined. Later, when the agent goes to recall a lesson, that trust stamp is checked again: untrusted notes stay out of the briefing. The agent gets the benefit of learning from itself without blindly believing every scrap that landed in its notebook.

Three More Ways the Harness Keeps Itself Honest

Governance decides whether an action is allowed. These three guardrails watch how the work is going — catching runaway loops, runaway costs, and sensitive data before they become a problem.

The sensitive-data checkpoint, a little closer

This checkpoint is the harness’s tie-in to Microsoft Purview-style data governance. It runs as its own gate, separate from the permission rulebook, and it has three settings so an organisation can adopt it gradually:

Putting It Together — The Closed Loop

Five systems, one feedback cycle. Each one feeds the next, and the cycle as a whole makes the harness improve over time without manual intervention on every turn.

Here is the loop in motion. Drift detection notices an agent slipping. The next risky call gets caught by governance — either because policies have tightened, or because the autonomy tier was downgraded after the drift alert. Escalation surfaces it to a human. The human’s decision is captured as a learning. The learning shifts the agent’s future behavior, which shifts its quality scores, which (eventually) brings drift back into the green. Round and round.

Final scenario quiz

Two Loops Forward — Reinforcement & the Proposer

Decay shows how learnings fade. This screen shows how they grow stronger — and how a second, offline loop edits the harness itself when the online loop alone isn’t enough.

Screen 5 covered one half of the learnings story: freshness decay, the age-based score that lets old wisdom expire. The other half is reinforcement: every time a learning is used and rated, its FeedbackWeight moves toward the new signal via an exponential moving average. Decay is what time does to a learning if nobody touches it; reinforcement is what feedback does when somebody does.

And one level above the online loop, a second loop runs offline: the harness-proposer meta-agent reads execution traces from prior runs and proposes structural edits — to skill files, prompts, or config — that an evaluator then scores against a benchmark. The online loop tunes weights. The offline loop edits the harness.

Online loop — ImproveLearningCommandHandler

When a turn finishes and the user (or an evaluator) supplies a feedback score from 1 to 5, the harness sends an ImproveLearningCommand for each learning that influenced the answer. The handler normalizes the score to [0, 1] and folds it into the existing weight with an EMA.

var config = _options.CurrentValue.AI.Learnings;
var alpha = config.FeedbackAlpha;
var normalized = (request.FeedbackScore - 1.0) / 4.0;
var newWeight = alpha * normalized + (1 - alpha) * learning.FeedbackWeight;

if (config.BiasCorrection && learning.UpdateCount < 5)
{
    var correctionFactor = 1.0 / (1.0 - Math.Pow(1.0 - alpha, learning.UpdateCount + 1));
    newWeight = Math.Clamp(newWeight * correctionFactor, 0.0, 1.0);
}

var updated = learning with
{
    FeedbackWeight = newWeight,
    UpdateCount = learning.UpdateCount + 1,
    LastReinforcedAt = _timeProvider.GetUtcNow(),
    Content = request.ReinforcementContent ?? learning.Content
};

var updateResult = await _store.UpdateAsync(updated, ct);
var bridgeResult = await _driftBridge.CheckAndAdjustBaselineAsync(updated, ct);
LearningsMetrics.Improved.Add(1);
        

Worked example — four 5-star updates on a fresh learning

Take alpha = 0.2, starting weight 0.0, and feed four perfect (5-star ⇒ normalized 1.0) ratings. With bias correction, here is the trajectory:

Now imagine the fifth turn produces a 2-star rating (normalized = 0.25). Past UpdateCount = 5 the correction stops; the EMA cleanly absorbs the signal: 0.2 · 0.25 + 0.8 · 1.0 = 0.85. The learning is still trusted — but a sustained run of low scores would erode the weight smoothly. No single bad turn flips the bit.

The drift bridge — reinforcement feeds the canary

After every successful weight update, ImproveLearningCommandHandler calls ILearningsDriftBridge.CheckAndAdjustBaselineAsync. The bridge asks: does this newly reinforced learning belong to a domain that has an EWMA baseline? If yes, and if the cumulative reinforcement crosses a threshold, the baseline mean is shifted. Drift detection (Screen 5) keeps comparing the live EWMA against a baseline that now reflects the team’s accumulated corrections — rather than a baseline frozen at launch day.

Bridge failure is non-critical. The learning update always succeeds independently; a failed bridge call is logged at Warning level and the loop continues. The bridge is an enrichment, not a gate.

Offline loop — the harness-proposer meta-agent

Reinforcement tunes weights. It cannot rewrite a bad system prompt or fix a misaligned skill. That is what the offline loop is for. The harness-proposer agent (in agents/harness-proposer/AGENT.md, mirrored by OptimizeExample in Presentation.ConsoleUI) is a meta-agent: it reads what other agents did, and proposes targeted edits to their configuration.

It expects three artifacts from a completed evaluation run, all newline-delimited JSON:

From these three inputs, it produces a list of proposed edits — targeted, small, and reviewable. A separate evaluator agent (or human) scores each proposal: apply, modify, or reject. The winner becomes the new baseline; traces from the next run feed the next iteration. Four phases per cycle: trace → propose → evaluate → apply.

Online vs offline — side by side

Quick check

The Agent Acts & Shows Its Work

Everything so far pictured the agent as something that talks — you ask, it answers in words. This last module is about the two ways it steps out of the chat box: it can show you things you can look at and click, and it can act on the screen in front of you.

Think of the difference between a travel agent who reads you a list of flights over the phone, and one sitting next to you who pulls up the booking screen, lays the options out as a table, hands you a form to pick your seat, and then clicks through the booking while you watch. Same job — but the second one is showing their work and doing the work, right in front of you. That’s what this module is about.

Showing Its Work — Widgets Instead of Walls of Text

In Module 5 you met these as ordinary tools. Here’s what they actually put on your screen.

When the agent decides that a picture would say more than a paragraph, it reaches for a special kind of tool. To the agent it’s just another tool call — but instead of returning text, the harness renders the result as a live element right there in the chat. There are four flavours:

The form is the interesting one, because it turns a one-way announcement into a back-and-forth. The agent hands you a form, you fill it in and hit submit, and your answers become the agent’s next piece of information — a genuine conversation conducted partly through the interface rather than only through typing.

Doing the Work — An Agent That Drives the Dashboard

Showing you a chart is helpful. Changing the screen for you is a bigger leap — and it’s where the agent stops being a narrator and becomes a co-pilot.

Picture one of the harness’s monitoring dashboards — charts, filters, a date picker, several pages. Now picture a small chat panel tucked into the corner of that same dashboard. You can ask that panel things in plain language, and instead of just answering, it can operate the dashboard for you. That’s the acting agent.

And it can still show at the same time: ask it a question about the data and it can draw a fresh chart right in the chat panel, on top of driving the page. So in one exchange it can read the current view, change the time range, and hand you a new chart of what changed — all from a sentence you typed in plain language.

Quick check