Operations & Cost

From a $50/month proof-of-concept to an $800+/month production deployment. Cost tiers, scaling knobs, CI/CD pipelines, governance runtime, and the operational playbook for keeping the harness healthy.

Cost tiers

The harness runs on a flexible Azure footprint that ranges from nearly-free development to a high-availability production deployment. The tiers below represent natural breakpoints where cost, reliability, and performance trade off.

Cost breakdown by category

Where the money goes at each tier. Use this to identify your largest cost levers and optimize accordingly.

Scaling patterns

The harness has three independent scaling dimensions: compute (Container Apps), AI throughput (Azure OpenAI), and search capacity (AI Search). Each has its own throttles and tuning knobs.

Container Apps autoscaling

Container Apps supports HTTP-based, CPU-based, and custom (queue-depth) scaling rules. Combine them for responsive scaling under mixed workloads.

• HTTP concurrent requests: scale when >50 concurrent requests per replica
• CPU threshold: scale when >70% CPU utilization
• Custom: scale on Azure Service Bus queue depth (for async agent jobs)
• Cold start: always keep minReplicas=1 for the API to avoid cold start on first request

scale: {
  minReplicas: 1
  maxReplicas: 20
  rules: [
    {
      name: 'http-requests'
      http: { metadata: { concurrentRequests: '50' } }
    }
    {
      name: 'cpu-utilization'
      custom: {
        type: 'cpu'
        metadata: { type: 'Utilization', value: '70' }
      }
    }
  ]
}

Azure OpenAI scaling

TPM (Tokens Per Minute) is the primary throttle for Azure OpenAI. When you hit the limit, requests return HTTP 429 with a Retry-After header. Strategies for managing throughput:

• Start with pay-as-you-go, move to PTU when you have predictable load and want guaranteed latency.
• Isolate quotas: use different deployments for agent chat vs. embeddings so one workload doesn’t starve the other.
• Multi-region fallback: configure a secondary Azure OpenAI resource in a different region. The harness Polly circuit breaker chain handles failover automatically.
• Monitor 429s: the harness has no dedicated throttle counter — watch the HTTP client instrumentation (http.client.request.duration with http.response.status_code = 429) or the Azure Monitor dependencies failure rate for the OpenAI endpoint. Sustained 429s mean you need more quota or a PTU commitment.

AI Search scaling

• Replicas: 1 (dev), 3 (prod HA with SLA). Each replica can handle ~15 QPS.
• Partitions: add when index size exceeds tier limits.
• Semantic ranker: adds ~200ms per query. Enable only when quality improvement justifies the latency.

Governance runtime

The harness includes in-app governance systems that directly affect operational behavior. These run inside the agent process and require monitoring and configuration.

Drift Detection (EWMA)

Monitors RAG quality scores against a rolling baseline using Exponentially Weighted Moving Average. Three severity levels control the response:

Baseline requires a calibration period (~100 queries). Configure thresholds via AppConfig.AI.Governance.Drift.

Circuit Breakers (Polly)

Every external service call (OpenAI, AI Search, graph DB) is wrapped in a Polly circuit breaker. When a provider fails 5 consecutive times, the circuit opens for 30 seconds. During open state, requests fail fast (no timeout wait). The fallback chain:

Escalation Workflows

Multi-approval system for high-risk agent actions. Three approval modes are supported:

• AllOf: all approvers must approve before the action proceeds.
• AnyOf: first approval wins — fastest path for low-risk escalations.
• Quorum: majority of approvers must agree.

Audit trail is stored in JSONL format in Blob Storage for compliance and post-incident review.

CI/CD pipeline

The deployment pipeline runs on GitHub Actions and progresses through five stages. Each stage gates the next — failures halt the pipeline and notify the team.

Operational runbook

Quick-reference for the most common operational scenarios. Diagnosis steps use the OTel trace and metric names from the Observability chapter.

Disaster recovery

The harness itself is stateless — all persistent state lives in external stores. Recovery planning focuses on data durability and geographic availability.

Backup

• Knowledge graph backend: use the backend's native backup — Neo4j on AKS (scheduled dumps to Blob) or Azure Database for PostgreSQL Flexible Server (automated backups, point-in-time restore). The store is plain relational (kg_nodes / kg_edges), so any managed Postgres backup applies.
• Azure SQL: Automated backups with geo-redundant storage.
• Blob Storage: Soft-delete enabled. GRS for production (cross-region replication).

Geo-redundancy

• Deploy to paired Azure regions for failover.
• Use Azure Front Door for geographic routing and automatic failover.
• AI Search geo-replication for read replicas in the secondary region.

RTO/RPO targets