The Keep The LLM On Framework
Eight phases that take an AI estate from unknown dependency to tested, governed, continuously improving infrastructure. Every engagement, deliverable, and report we produce maps to this framework — so your team always knows where you are and what comes next.
Methodology
Eight phases, one operating model
Phases 1–4 build resilience. Phases 5–8 keep it — because resilience is a practice, not a project.
Discover
You cannot protect what you cannot see. We surface every AI service your organization depends on — sanctioned and shadow.
Key activities
- AI service discovery workshops with business and IT stakeholders
- Shadow AI audit across departments and SaaS tooling
- Vendor, model, and API inventory
- Business process mapping to AI touchpoints
Deliverable
AI Service Inventory (ASI)
Assess
Every AI service gets a criticality score, a dependency map, and defined recovery objectives — the language of business continuity, applied to AI.
Key activities
- AI Criticality Index (ACI) scoring for each service
- Dependency mapping across models, RAG pipelines, identity, and integrations
- Risk register: outage, cost shock, model deprecation, compliance exposure
- AI-RTO and AI-RPO definition with business owners
Deliverable
AIR Gap Report
Design
We architect for failure: multi-model routing, hybrid deployment, and failover paths that match each service's criticality.
Key activities
- Target architecture design: hybrid, multi-cloud, model routing
- Failover topology selection: cloud-to-cloud, cloud-to-local, active-active
- Governance model for prompts, data, and identity
- Cost resilience modeling against pricing and quota shocks
Deliverable
AIR Architecture Blueprint
Deploy
Blueprints become running infrastructure — routing layers, local inference where warranted, and knowledge kept in sync across failover targets.
Key activities
- Implementation runbooks and change plans
- Model routing layer deployment
- Local and edge inference where criticality requires it
- Knowledge synchronization and identity integration
Deliverable
Deployment Playbook
Protect
Resilient AI is governed AI. We put controls around access, prompts, knowledge, and vendor concentration.
Key activities
- Access controls and identity integration for AI services
- Prompt and knowledge governance policies
- Immutable audit logging for AI interactions
- Vendor diversification controls
Deliverable
AIR Protection Policy
Test
A failover plan that has never been tested is a hypothesis. We run quarterly drills that simulate real failure modes.
Key activities
- Quarterly AI recovery drills with pass/fail criteria
- Failover simulation: vendor outage, API throttling, network loss
- Degradation mode validation for reduced-capability operation
Deliverable
Test Results and Remediation Plan
Operate
AI availability becomes an operational discipline: monitored, measured, and managed like any tier-one system.
Key activities
- 24/7 monitoring of availability, latency, cost, and quota
- Incident response procedures for AI outages
- Change management for model updates and deprecations
Deliverable
AIR Runbook and SLA dashboard
Optimize
Resilience compounds. Each quarter we tune cost, performance, and maturity — and report it in language your board understands.
Key activities
- Cost optimization through model tiering and caching
- Performance tuning across routing and inference
- Maturity advancement roadmap
Deliverable
Quarterly Executive AIR Report
AI Maturity Model
Six levels of enterprise AI maturity
The Assessment maps every AI service in your estate to this model. Risk concentrates where actual maturity outruns the maturity your continuity planning assumes.
| Level | Name | Characteristics | AIR requirement |
|---|---|---|---|
| 0 | No AI | No sanctioned AI use in the organization. | Education and policy groundwork. |
| 1 | Shadow AI | Employees use consumer AI tools ad hoc, without oversight. | Usage policy and inventory discovery. |
| 2 | Department AI | Team-level copilots and siloed tools with local owners. | Dependency mapping per department. |
| 3 | Operational AI | AI embedded in daily workflows: support, development, sales. | Failover design and monitoring. |
| 4 | Mission-Critical AI | Revenue and operations depend on AI availability. | AI-RTO/RPO, tested recovery, governance. |
| 5 | Enterprise-Grade AI | Multi-model, hybrid, governed, and audited AI estate. | Continuous AIR testing and executive reporting. |
From Level 3 upward, AI failure is business failure — and resilience engineering stops being optional.
AIR Terminology
The language of AI Operational Resilience
Shared vocabulary is how a discipline forms. These are the terms we use in every assessment, architecture, and executive report.
- AI Recovery Time ObjectiveAI-RTO
- The maximum acceptable time to restore an AI capability after a failure — the AI equivalent of the RTO your BC plan already defines for email and ERP.
- AI Recovery Point ObjectiveAI-RPO
- The maximum acceptable loss of knowledge, context, or conversation state when failing over to an alternate model or provider.
- AI Service InventoryASI
- A complete catalog of every AI service in the organization — models, agents, copilots, RAG pipelines — with owners, vendors, and dependencies.
- AI Criticality IndexACI
- A 1–5 score of the business impact if a given AI service fails, used to prioritize resilience investment.
- AI Dependency Map
- A visual graph of the chains connecting business workflows to models, RAG stores, APIs, identity providers, and integrations.
- AI Operational Readiness Score
- A composite 0–100 score produced by the AIR Assessment, measuring how prepared the organization is for AI failure scenarios.
- AI Availability Score
- An ongoing metric combining uptime and degraded-mode performance across the AI estate.
- AI Continuity Plan
- A business continuity document specific to AI services: failure scenarios, failover procedures, owners, and communication plans.
- AI Recovery Test
- A simulated failover exercise — vendor outage, API throttling, network loss — executed with pass/fail criteria and documented results.
See the framework applied to your AI estate
The AIR Assessment executes the Discover and Assess phases — and hands you the roadmap for the rest.