AegisLayer Public Threat Model¶

Purpose¶

This document defines a public, high-level threat model for AI systems that can reason, call tools, access data, trigger workflows, and influence real-world systems.

It is intended as an architectural reference. It does not disclose confidential controls, customer environments, proprietary implementations, or unapproved patent-sensitive details.

Security Objective¶

The principal security objective is to prevent an AI system's capability from being treated as execution authority.

A proposed action should be independently evaluated for identity, authority, policy, risk, approval, runtime constraints, and evidence requirements before execution.

System Scope¶

The conceptual system includes:

Human users
AI models and agents
Prompt, retrieval, and memory layers
Policy and approval services
Execution runtimes
Tools, plugins, connectors, and APIs
Data stores and knowledge sources
Evidence, audit, and telemetry systems
Administrative and deployment control planes

Protected Assets¶

Assets that may require protection include:

Identities and credentials
Authorization and policy state
Models, prompts, and system instructions
Training, retrieval, and operational data
Tool and connector permissions
Business workflows
Evidence and audit records
Administrative interfaces
Source code and deployment artifacts
Customer, personal, regulated, and confidential information
Service availability and operational continuity

Trust Boundaries¶

The threat model treats the following as separate trust domains:

End users and external requesters
AI models and autonomous agents
Retrieval, memory, and context systems
Policy and approval services
Execution runtimes
Tools, connectors, and third-party services
Data stores
Evidence and audit infrastructure
Administrative control planes

Crossing a trust boundary should require explicit validation, authorization, and observability.

Threat Actors¶

Potential threat actors include:

External attackers
Malicious or compromised users
Insiders with excessive access
Compromised administrators
Malicious third-party providers
Supply-chain attackers
Automated abuse systems
Compromised AI agents or workloads
Benign users whose requests produce unsafe outcomes

Core Threat Categories¶

1. Prompt Injection and Instruction Manipulation¶

An attacker may attempt to override system instructions, redirect agent behavior, or introduce malicious instructions through user input, retrieved content, files, webpages, messages, or tool outputs.

Potential consequences:

Policy bypass
Data disclosure
Unauthorized tool use
Manipulated decisions
Hidden persistence in memory or context

Conceptual controls:

Separation of instructions from untrusted content
Input provenance and trust labeling
Policy evaluation outside the model
Tool-call authorization
Context minimization
Output and action validation

2. Excessive Agency and Unauthorized Actions¶

An AI agent may attempt actions beyond the user's authority or beyond the scope required for the task.

Potential consequences:

Unauthorized changes
Financial or operational loss
Privilege escalation
Irreversible actions

Conceptual controls:

Least privilege
Capability-scoped execution
Explicit action boundaries
Human approval for consequential actions
Time-limited authorization
Fail-closed runtime behavior

3. Credential Theft and Secret Exposure¶

Credentials may be exposed through prompts, logs, model context, tool outputs, repositories, environment variables, or compromised connectors.

Potential consequences:

Account takeover
Lateral movement
Data exfiltration
Unauthorized infrastructure access

Conceptual controls:

Secret isolation
Short-lived credentials
Scoped tokens
Redaction and log filtering
Rotation and revocation
Prohibition on embedding secrets in prompts or code

4. Tool and Connector Abuse¶

An attacker or compromised agent may misuse legitimate tools, plugins, APIs, browsers, terminals, databases, or cloud connectors.

Potential consequences:

Destructive execution
Unauthorized data access
External communication
Persistence or lateral movement

Conceptual controls:

Connector allowlists
Capability discovery and validation
Parameter and schema validation
Sandboxing and isolation
Rate, time, and resource limits
Action-specific approval

5. Privilege Escalation¶

A user, agent, workload, or connector may attempt to acquire broader permissions than originally granted.

Potential consequences:

Administrative control
Policy bypass
Cross-tenant access
Evidence tampering

Conceptual controls:

Separation of duties
Immutable authorization boundaries
Independent policy enforcement
Strong administrative controls
Reauthorization when context changes
Continuous permission review

6. Data Poisoning and Retrieval Manipulation¶

Training data, retrieval sources, memory, embeddings, or operational knowledge may be manipulated to influence model behavior.

Potential consequences:

Incorrect or malicious recommendations
Persistent behavioral manipulation
Hidden trigger conditions
Corrupted decisions

Conceptual controls:

Source provenance
Integrity validation
Trust scoring
Dataset and retrieval review
Segregation of trusted and untrusted sources
Monitoring for anomalous content changes

7. Model and Artifact Tampering¶

Models, adapters, weights, containers, packages, or configuration artifacts may be altered or replaced.

Potential consequences:

Backdoored behavior
Silent policy degradation
Data leakage
Remote code execution

Conceptual controls:

Signed artifacts
Hash and integrity verification
Controlled model registries
Reproducible deployment processes
Dependency scanning
Provenance records

8. Supply-Chain Compromise¶

Third-party models, libraries, datasets, actions, containers, or hosted services may introduce vulnerabilities or malicious behavior.

Potential consequences:

Code execution
Credential compromise
Data exfiltration
Build or deployment compromise

Conceptual controls:

Dependency pinning
Software bills of materials
Artifact verification
Vendor and source review
Isolated build environments
Continuous vulnerability monitoring

9. Sensitive Data Exposure¶

AI systems may reveal confidential, personal, regulated, or proprietary information through outputs, logs, memory, retrieval, or tools.

Potential consequences:

Privacy violations
Regulatory exposure
Intellectual-property loss
Customer harm

Conceptual controls:

Data classification
Access control
Minimization
Redaction
Output filtering
Retention limits
Context-aware policy enforcement

10. Audit and Evidence Tampering¶

Attackers may alter, delete, reorder, or fabricate records to conceal actions or defeat accountability.

Potential consequences:

Loss of forensic integrity
False attribution
Compliance failure
Inability to reconstruct incidents

Conceptual controls:

Append-oriented records
Hash chaining
Immutable finalized evidence
Sequence validation
Correlation and causation identifiers
Restricted administrative access

11. Governance and Approval Bypass¶

An action may be executed without required policy review, approval, or separation of duties.

Potential consequences:

Unauthorized high-impact actions
Policy violations
Fraud or abuse
Unreviewed exceptions

Conceptual controls:

Mandatory pre-execution checks
Independent approval services
Explicit denial on missing approval
Approval expiry and scope validation
Multi-party review where appropriate

12. Denial of Service and Resource Exhaustion¶

Attackers or malfunctioning agents may consume compute, tokens, storage, network capacity, external API quotas, or human-review capacity.

Potential consequences:

Service degradation
Cost escalation
Delayed incident response
Reduced availability

Conceptual controls:

Quotas and budgets
Rate limits
Timeouts
Concurrency controls
Circuit breakers
Cost and resource telemetry

13. Unsafe Automation and Cascading Failure¶

An apparently valid action may trigger downstream workflows whose combined effect is unsafe or disproportionate.

Potential consequences:

Broad operational disruption
Repeated or duplicate actions
Cross-system propagation
Difficult recovery

Conceptual controls:

Idempotency
Bounded retries
Compensation procedures
Dependency-aware workflows
Staged rollout
Kill switches and cancellation

14. Insider Threat¶

Authorized personnel may misuse access intentionally or accidentally.

Potential consequences:

Data theft
Policy changes
Evidence suppression
Unauthorized releases

Conceptual controls:

Least privilege
Separation of duties
Administrative audit trails
Approval for sensitive changes
Anomaly detection
Periodic access review

Abuse Cases¶

Representative abuse cases include:

A retrieved webpage instructs an agent to reveal secrets.
A user asks an agent to invoke an administrative tool outside their role.
A compromised connector returns malicious output that changes execution behavior.
A poisoned document causes repeated unsafe decisions.
A model artifact is replaced with a tampered version.
An approval record is reused outside its intended scope or validity period.
An agent repeatedly retries an irreversible action.
An administrator disables logging before a sensitive operation.

Security Control Mapping¶

Threat	Primary Control Families
Prompt injection	provenance, policy isolation, tool authorization, context controls
Excessive agency	least privilege, approvals, runtime constraints
Credential theft	secret isolation, scoped credentials, rotation, redaction
Connector abuse	capability validation, sandboxing, allowlists, parameter checks
Privilege escalation	separation of duties, independent authorization, revalidation
Data poisoning	provenance, integrity checks, source trust, monitoring
Artifact tampering	signatures, hashes, controlled registries, reproducible deployment
Supply-chain compromise	dependency review, SBOM, isolation, vulnerability monitoring
Data exposure	classification, minimization, access control, output controls
Evidence tampering	append-only design, hash chaining, sequence validation
Approval bypass	mandatory gates, expiry, scope validation, fail-closed behavior
Resource exhaustion	quotas, rate limits, budgets, circuit breakers
Cascading failure	idempotency, bounded retries, compensation, cancellation
Insider threat	least privilege, audit, dual control, access review

Residual Risk¶

Even with layered controls, residual risk remains. Models may behave unpredictably, controls may be misconfigured, third parties may be compromised, and novel attacks may bypass existing defenses.

The architecture therefore emphasizes:

Continuous monitoring
Evidence preservation
Rapid containment
Credential revocation
Recovery planning
Post-incident review
Ongoing threat-model revision

Validation Approach¶

A mature implementation should validate the threat model through:

Architecture review
Abuse-case testing
Red-team exercises
Connector and permission tests
Policy-denial tests
Evidence-integrity verification
Incident simulations
Recovery drills
Independent security assessment where appropriate

Assumptions and Limitations¶

This document assumes that some external systems, providers, or users may be compromised or untrustworthy.

It does not claim that every listed control is currently implemented, independently tested, or sufficient to prevent every attack.

Review Cadence¶

The threat model should be reviewed when:

New tools or connectors are introduced
Authority boundaries change
New data categories are processed
Models or retrieval systems change
Significant incidents occur
New attack techniques emerge
Deployment architecture changes