Kindo × Deloitte
Sprint 2 Review · Jul 25, 2026
Sprint 2 · SOC for AI

From Objectives to Working Platform

Sprint 2 focused on translating Deloitte's SOC for AI governance objectives into platform capabilities. This review maps each objective to platform capabilities, research completed, and open questions for the co-design session. The Agent Telemetry demo is a working prototype — not yet in production.

Demo

Agent Telemetry — Working in Kindo

Walkthrough of Agent Telemetry capabilities built into the Kindo platform. Trace export, LLM judges, eval configuration, span detail, and error diagnostics.

00 · Context

From Sprint 1 PoC to Sprint 2

Sprint 1 (Jun 29–Jul 10) delivered a SOC for AI proof of concept. Sprint 2 builds on the co-design session feedback from Jul 10.

Sprint 1 — What We Built

The team built a SOC for AI Governance Monitor as a triggered agent on Kindo. In under 60 seconds, the agent scanned 286 deployed agents, evaluated 45 active integrations, and assessed change management controls — covering all five governance objectives defined by Kishore's team.

Jul 10 — Co-Design Session with Deloitte

The PoC was presented to Kush, Krishna, Nathan, and Zun. Key outcomes from the session:

Kush — Speed acknowledged
"You guys are phenomenal"
"Really appreciate you guys turning this around this fast. I think that's been the cornerstone of our partnership — the velocity and the focus that you guys bring in."
Tony's response: "We definitely don't want to make you look bad. You guys are 550,000 people and you have a structure to accommodate for that."
Kush — The Gateway Philosophy
"If LLM is the choke point, what can we put there?"
"If we had something which acted as a choke point... anything that runs within your four walls, if it has to go to your sanctioned LLM, that's the choke point. So we should be able to catalog, pull what's going through and understand that. Is there an integration connector shim we can insert there — and then we'll be able to block it."
This is the origin of Pillar 2 — Kindo as the LLM gateway. Charlie confirmed: "It's two lines of code to run that through Kindo instead of directly to Bedrock or Anthropic."
Charlie — Three Pillars Defined
"I am thinking of it in three categories"
"There's the platforms which we would reach out onto like Anthropic, Foundry, etc. Then there's the unknown activity that you need to sweep as the long tail with network monitoring. But the middle pillar — we have a mandate to deploy Copilot, etc. — so how do you control those?"
Framework confirmed by Kush and Krishna. These became the three pillars of the Sprint 2 architecture.
Krishna — CrowdStrike Analogy
"We should have a menu of options"
"I'm reflecting on when CrowdStrike EDR came in — they have five different ways to discover endpoints. I think we should have a menu of options. Here are the five different ways, ten different ways. Each option has a pro and con."
This is the origin of the Discovery Menu — Sprint 2 deliverable. The Platform Compatibility Matrix feeds directly into this.
Krishna — Tiered Model
"Basic, Standard, and Elite"
"I'll use the CrowdStrike terminology — Basic, Standard, and Elite. This is AI SOC Basic, this is Standard, and in Elite we can do even blocking of agents before malicious actions take place. The first version should be less intrusive — just doing detection and responding post agent action."
The tier model below uses Krishna's exact framing. Sprint 2 maps platform capabilities to these tiers.
Krishna — Platform Matrix Request
"Is there a list of platforms you were able to discover agents?"
"I'm also assuming some platforms might be better at letting you do controlling actions on agents versus others. Correct."
Charlie: "I think that might be a good follow-up here — go enumerate a more substantial list."
This is the direct origin of the Platform Compatibility Matrix — Krishna asked for it, Charlie agreed, Sprint 2 delivered the research.
Kush — Data Lake Warning
"You guys should not go down the path of building a data lake"
"That is a totally different thing. If we can stay away from it, that would be good."
Noted. The Agent Telemetry proposal (MLflow) uses existing Postgres + S3 — no new data lake. Kush's guidance shaped the architecture constraint.
Kush — Open Telemetry Standard
"There's an open standard which allows agents to be more chatty from a telemetry standpoint"
Kush referenced an open standard for agent telemetry (OpenTelemetry). "It's a simple configuration — you've got to configure your agents to adopt that open standard."
This is the origin of Standard 3 (OTel) — Charlie's telemetry proposal builds on exactly this. The working prototype uses OTel-compatible traces.

Jul 7 — Kishore Defined the Scope

Kishore's email corrected a key misunderstanding: SOC for AI is about monitoring known AI agents/platforms — not shadow AI discovery, which is handled by Deloitte's detection engineering team. He defined the five governance objectives and specified two requirements for implementation: (1) audit logs/telemetry from the platform, and (2) process/data collection architecture to monitor for these risks.

Sprint 2 — What We Set Out to Do

Take the feedback from Jul 10 and Kishore's objectives, and build: the platform matrix Krishna asked for, the three-pillar architecture Charlie and Kush defined, and the telemetry capability that makes all of it possible.

01 · Foundation

Deloitte's SOC for AI Objectives

Five governance objectives defined by Kishore's team (Jul 7, 2026). Focus: monitoring risks from known AI agents/platforms — not shadow AI discovery, which is handled by Deloitte's detection engineering team.

Implementation requires two things: (1) audit logs/telemetry data from the platform and (2) process/data collection architecture to monitor for these risks.

Objective 1
Detect Unauthorized Agent Deployment
Detect when unauthorized users are deploying new agents or updating existing agents.
Platform capability: Agent Telemetry (prototype) — every agent is visible in the catalog with creator, last updated, and type. Telemetry export captures every run. We'd welcome your input on the level of visibility needed for your workflows.
Objective 2
Detect Unauthorized Tool/Data Connections
Detect when new integrations to unauthorized tools/sensitive data sources are added.
Platform capability: Trace detail view shows every tool call — which tools, what arguments, what data returned. An LLM judge ("Grounded tool use") can flag unexpected tool access. We'd welcome your guidance on defining "unauthorized" within your governance framework.
Objective 3 · Biggest Engineering Need
Detect Agent Behavioral Drift
Detect when actions performed are NOT in-line with the SOPs defined for the agents, like deleting JIRA tickets or sending data to unauthorized external data stores.
Platform capability: LLM Judges — plain-language evals that score every agent run. Define criteria, the judge flags deviations run-by-run. 4/4 satisfied = compliant. 0/4 = deviation detected. To maximize this capability, we'd like to co-define which SOPs and behavioral criteria the judges should evaluate against.
Objective 4
Detect Guardrail/Policy Changes
Detect changes in MCP/tool policies to provision edit/delete privileges to agents when not intended.
Platform capability: Configuration panel — telemetry export, sampling rate, retention, and eval configs are visible per agent. Full change tracking and audit trail for configuration modifications is on the roadmap as a next step.
Objective 5
Detect Cross-Tenant Data Contamination
In the multi-tenant environment, detect data breach/contamination when agent for one client cross references data of another client.
Platform capability: Organization-isolated workspaces — traces stored per org. Cross-tenant isolation is enforced at the platform level. Per-org workspace isolation is demonstrated in the prototype. Production-grade cross-tenant monitoring is a planned enhancement.
02 · Capabilities

What the Demo Shows

Breakdown of each capability demonstrated in the video above.

Agent Catalog
Full inventory of deployed agents
Every agent visible — name, type, creator, last updated. One-click access to telemetry view.
Traces
Every run captured and inspectable
Model calls, tool activity, token usage, latency, cost, and LLM judge verdicts — per run. Stored in org-isolated workspace.
Evals & Judges
Plain-language behavioral monitoring
Define judges in natural language. Multiple judge models available (Claude, GPT). Automatic scoring on every exported run.
Trace Detail
Full drill-down into any run
Span waterfall, conversation replay, tool calls with arguments and results, token metrics, cache hit rates, error diagnostics.
Configuration
Governance controls per agent
Telemetry export toggle, sampling rate, trace retention, eval judge model selection — all configurable per agent, per organization.
03 · Architecture

Three-Pillar Discovery Framework

Framework developed collaboratively and confirmed at the Jul 10 co-design session. Three approaches to AI governance discovery — each platform has different strengths.

Pillar 1

Platform Integrations

Reach into Anthropic, Azure Foundry, Copilot, ServiceNow, Bedrock via their admin APIs. Enumerate agents, audit usage, inspect configurations.
Pillar 2

LLM Gateway / Choke Point

Kindo as inference proxy — 2 lines of config to redirect. The LLM is the firewall. Real-time governance at the choke point. Kush: "The choke point IS the governance layer."
Pillar 3

Network/Endpoint Monitoring

Sweep for shadow AI — the long tail. Network and endpoint-level discovery of unauthorized AI usage beyond managed platforms.
04 · Commercial Model

Discovery Tier Model

Inspired by Krishna's CrowdStrike analogy at the Jul 10 co-design — a menu of discovery options, not one-size-fits-all. The tier names and mapping below are our proposed framework, not yet validated with Deloitte. First version: "less intrusive" detection and response post-action.

Basic

Detect & Respond Post-Action

Telemetry export + audit log analysis. After-the-fact detection. Low adoption cost. Entry point for governance.
Standard

Monitor & Alert Live

Native telemetry + LLM judges scoring every run. Real-time drift detection. Configurable evals and alerting thresholds.
Elite

Preventative Blocking

Gateway position (inference + MCP) = real-time preventative control. Block unauthorized actions before they execute. Requires Pillar 2 gateway integration.
05 · Research

Platform Compatibility Matrix

Per-platform capabilities for discovery and governance actions. Determines which tier model and which pillar applies to each platform in a customer's environment.

Capability Anthropic Azure AI Foundry Microsoft Copilot ServiceNow AWS Bedrock
List models/agents ✓ API ✓ ARM API ~ Graph + Studio ~ Table API ✓ API
Usage/audit logs ~ Limited ✓ Azure Monitor ✓ Purview ✓ System logs ✓ CloudTrail
Content/safety filters ✗ Not via API ✓ Full API ~ DLP policies ~ Admin config ✓ Guardrails API
Disable/block access ✓ Key revocation ✓ RBAC + Policy ✓ Entra ID ✓ Role-based ✓ IAM + SCPs
Enforce policies ✗ No policy API ✓ Azure Policy ✓ DLP + labels ~ Workflows ✓ Guardrails + SCPs
Alerting/notifications ✗ None ✓ Monitor Alerts ✓ Sentinel ✓ Event Mgmt ✓ EventBridge
Best Kindo method Gateway (P2) API + Gateway (P1+2) API + Network (P1+3) API + Network (P1+3) API + Gateway (P1+2)

✓ Full API  ·  ~ Partial  ·  ✗ Not available

Key Insight

No single discovery method works across all platforms. This is why Krishna's CrowdStrike analogy is right — customers need a menu of discovery methods, deployed based on: which platforms are in their environment, network disposition (gateway possible?), and governance maturity (Basic/Standard/Elite).

Kindo's advantage: it can operate at all three pillars simultaneously. Rich admin APIs (Azure, Bedrock) → Pillar 1. Limited admin APIs (Anthropic) → Pillar 2 gateway fills the gap. Shadow AI → Pillar 3 catches the rest.

06 · Standards

Kindo's Three Open Standards

Kindo reaches governance outcomes on three existing open standards — no proprietary protocol adoption required.

Standard 1 · Live Today

Inference APIs

Chat Completions, Responses, Messages, Models. 2 lines of config to route Claude Code, Copilot, or any agent through Kindo.
Standard 2 · Live Today

Federated MCP Gateway

Single governed endpoint for tool/integration access. Real-time control over agent tool use across the organization.
Standard 3 · Landing Now

OpenTelemetry

OTLP-based. Kindo ingests traces from any AI workload — completing the Inference + Tools + Telemetry trifecta. Foundation for evals, self-improvement, and governance.

Inference + Tools + Telemetry = comprehensive AI governance. Open standards = low adoption cost. Gateway position is cheap and reversible — addresses "customers worry about choke point."

07 · Sprint Status

Sprint 2 Backlog — Jul 13–25

Prototype

Agent Telemetry — Working Prototype

Working prototype built: trace export, LLM judges, eval configuration, span detail, error diagnostics. Currently in development environment — production deployment planned.

Research

Platform Compatibility Matrix

5-platform research (Anthropic, Azure, Copilot, ServiceNow, Bedrock). Based on public API documentation — not yet validated with hands-on testing or Deloitte input.

Research

Pillar 2 Gateway/Shim Architecture

Kush endorsed gateway approach (Jul 16 email). Charlie documented the three-standard strategy. Architecture documented, not yet implemented as a Deloitte-facing product.

Proposal

OTel/MLflow Architecture Design

Two-tier model proposed: export (current alpha) vs native pillar. Working prototype validated in development. Engineering roadmap being finalized.

In Progress

Discovery Menu of Options (CrowdStrike Model)

Krishna-facing deliverable. 5-7 discovery methods with tier mapping. Dependent on Platform Matrix (complete) — synthesis in progress.

Blocked

Digital Twin L2/L3 Analyst Pilot

Teams integration blocked on Deloitte IT. Krishna circling back internally. Sprint 2 goal was discovery — how to do this. Moved to Sprint 3.

Blocked

Triage Agent Optimization

Requires work session with Zun. Kush OOO Jul 21-25. Moved to Sprint 3.

Blocked

A7 Quality Audit Agent Design

Depends on digital twin approach + design session with Krishna's team. Sprint 3.

08 · Next

Co-Design Input Requested

For the next co-design session (~August), we'd like to walk through these capabilities with your team and get input on:

  • Which eval criteria matter most for your SOC workflows — what should the LLM judges check for?
  • Sampling and retention requirements — what's the right balance between coverage and storage cost?
  • Platform priority — which external platforms (Azure, Copilot, ServiceNow) should Kindo integrate with first?
  • Tier model validation — does Basic/Standard/Elite mapping align with how your customers think about governance?
  • Teams access — needed to unblock digital twin and L2/L3 workflow discovery (Sprint 3)

Kindo × Deloitte · Sprint 2 Review · T&C Collaboration · Jul 2026