🚨 Incident Response Environment

An OpenEnv-compliant RL benchmark where AI agents diagnose production microservices failures

LLM agents act as on-call Site Reliability Engineers, investigating simulated production incidents under partial observability, time pressure, and cascading failures.

🎮 Live Dashboard · 📖 Environment Docs · 📊 Benchmark Guide · 🤖 Agent Guide · 🏆 Reward Design

---

🎯 Overview

When production systems fail at 3 AM, engineers must quickly investigate cascading failures across distributed microservices to identify the root cause. This environment simulates that exact challenge — a gym-compatible RL benchmark where agents must:

• Investigate step-by-step under partial observability (only see what you query)
• Navigate deception from red herring services (high CPU but not the cause)
• Handle multi-fault scenarios where multiple independent failures occur simultaneously
• Adapt to cascade mechanics where secondary faults emerge mid-episode
• Make sequential decisions that accumulate evidence before applying fixes
• Declare root cause analysis based on investigation findings

---

✨ Key Features

Production-Ready Implementation

✅ 16 Incident Tasks — CPU spikes, memory leaks, database deadlocks, SSL cert expiry, multi-fault scenarios, long-horizon planning
✅ 7 Action Types — read_logs, check_metrics, check_health, run_db_query, restart_service, rollback_deployment, declare_rca
✅ 6 Microservices — API gateway (victim), auth, orders, notifications, Redis cache, PostgreSQL database
✅ Multi-Fault Support — Dual independent failures (e.g., Redis + Auth both failing)
✅ Cascade Mechanics — Secondary failures triggered at specific episode steps
✅ Red Herrings — Services with suspicious metrics but NOT the root cause
✅ Long-Horizon Planning — 50-step episodes testing sustained reasoning ability

Advanced Reward Shaping

✅ Evidence-Based Rewards — Investigation actions earn +0.10 to +0.18 per unique evidence type
✅ Sequence Bonuses — Fixes with prior evidence get up to 1.0x multiplier (blind fixes: 0.0x)
✅ Redundancy Penalties — Repeating actions escalates penalties (-0.08 → -0.20)
✅ Efficiency Bonuses — Faster solutions score higher (up to +0.30 for time saved)
✅ RCA Rewards — Correct diagnosis: +0.50 base + bonuses, Wrong: -0.40 penalty
✅ Domain-Aware Dispatch — Microservices, CI/CD, and Kafka reward functions

Interactive Gradio Dashboard

✅ Real-Time Playground — Manual testing interface with live action/observation display
✅ Custom Model Benchmarking — Load any HuggingFace model for real LLM inference evaluation
✅ Leaderboard Visualization — Compare model performance across all tasks from benchmark.json
✅ About Page — Full project documentation and architecture overview
✅ Premium Dark Theme — Sleek UI with amber accents and smooth micro-interactions
✅ CPU-Safe Model Loading — Detailed error diagnostics with graceful fallback

Comprehensive Tooling

✅ Expert Agent — Rule-based baseline achieving >0.80 on all tasks (training/expert_agent.py)
✅ SFT Data Generator — Export expert trajectories as trajectories.jsonl for supervised fine-tuning
✅ Benchmark Runner — Automated evaluation harness with persistent leaderboard (benchmark_runner.py)
✅ REST API — OpenEnv-compliant FastAPI server with full endpoint coverage (server/app.py)
✅ LLM Judge — Adversarial evaluation with phase-order enforcement (judge/llm_judge.py)
✅ Multi-Provider LLM Client — OpenAI, Anthropic, Mock fallback with retry logic (judge/llm_client.py)

---

🚀 Quick Start

Option 1: Local Development

# Clone repository
git clone https://github.com/yourusername/incident-response-env
cd incident-response-env

# Install dependencies
pip install -e .

# Start the server (includes FastAPI + Gradio dashboard)
uvicorn server.app:app --host 0.0.0.0 --port 7860

Open the dashboard: http://localhost:7860/dashboard

Option 2: Docker

docker build -t incident-response-env .
docker run -p 7860:7860 incident-response-env

Option 3: Run Benchmark Evaluation

Configure your LLM provider and run automated benchmark:

Bash:

export API_BASE_URL="https://api.openai.com/v1"
export API_KEY="sk-YOUR_OPENAI_KEY"
export MODEL_NAME="gpt-4o"

python inference.py

PowerShell:

$env:API_BASE_URL="https://api.openai.com/v1"
$env:API_KEY="sk-YOUR_OPENAI_KEY"
$env:MODEL_NAME="gpt-4o"

python inference.py

Supported providers: OpenAI, Anthropic (via LiteLLM), HuggingFace Inference, Together AI, Ollama (local)

---

📋 Task Suite (16 Scenarios)

All tasks test incident response skills with varying difficulty levels, from shallow (10 steps) to long-horizon planning (50 steps):

🎯 Special Task: task_expert_long_horizon

Tests true long-horizon reasoning over 50 steps:

• Initial fault: Postgres slow query causing gradual degradation
• Red herring trap: Agent might apply quick fix at step 10-15 with restart
• Latent secondary fault: Quick fix introduces query planner bug
• Cascade trigger: At step 35+, order-service cascades due to secondary fault
• Required skill: Maintain context over 50-step trajectory, recognize latent issues, implement correct fix

Why this matters: Reveals whether agents develop genuine SRE reasoning vs. pattern-matching quick fixes.

---

🎮 Action Space

Agents have 7 distinct action types for investigation and remediation:

{"action_type": "read_logs",           "target": "auth-service"}
{"action_type": "check_metrics",       "target": "redis-cache"}
{"action_type": "check_health",        "target": "order-service"}
{"action_type": "run_db_query",        "target": "postgres-db"}
{"action_type": "restart_service",     "target": "notification-service"}
{"action_type": "rollback_deployment", "target": "order-service"}
{"action_type": "declare_rca",         "target": "redis-cache"}

Available Services

• api-gateway — Always a victim, never the root cause (forces upstream tracing)
• auth-service — Authentication and authorization service
• order-service — Core business logic service
• notification-service — Async notification processing
• redis-cache — In-memory caching layer
• postgres-db — Primary database

---

📊 Observation Space

Every action returns a structured observation with:

---

🏆 Reward System

The environment uses a domain-aware reward function (reward.py) with sophisticated evidence tracking:

Evidence-Based Investigation Rewards

Intervention Rewards

Bonus Modifiers

Evidence Bonus (0 to +0.20):

• +0.05 per unique evidence type gathered (logs, metrics, health, queries)
• Maximum +0.20 for 4+ evidence types
• Applied to fix actions and RCA declarations

Efficiency Bonus (0 to +0.30):

• Formula: (max_steps - current_step) / max_steps × 0.30
• Example: Solving in 4/10 steps = (10-4)/10 × 0.30 = +0.18
• Rewards faster diagnosis without sacrificing thoroughness

Sequence Multiplier (0.0x to 1.0x):

• Fix actions with prior evidence: 1.0x multiplier (full reward)
• Blind fixes without evidence: 0.0x multiplier (zero reward)
• Encourages investigation before intervention

Redundancy Penalties

Score Clamping

Final scores are clamped to [0.001, 0.999] to prevent:

• Negative scores from repeated mistakes
• Perfect scores from gaming the system

Example Score Calculations

Fast Expert Solve (0.88):

Base RCA:        +0.50
Evidence bonus:  +0.20 (4 evidence types)
Efficiency bonus: +0.18 (solved in 4/10 steps)
Total:           0.88

Slower Correct Solve (0.72):

Base RCA:        +0.50
Evidence bonus:  +0.10 (2 evidence types)
Efficiency bonus: +0.06 (solved in 8/10 steps)
Investigation:   +0.06 (from evidence gathering)
Total:           0.72

Wrong Diagnosis (-0.40):

Incorrect RCA:   -0.40 (hard penalty for guessing)

---

🎨 Gradio Dashboard

The interactive dashboard provides:

1. Manual Playground

• Select task from dropdown
• Execute actions with target service selection
• Real-time observation display
• Step counter and reward tracking
• Episode termination handling

2. Custom Model Benchmarking

# Dashboard automatically handles:
# 1. HuggingFace model loading (transformers pipeline)
# 2. Real LLM inference (no mock data)
# 3. CPU-safe execution with error diagnostics
# 4. Progress tracking across all 16 tasks
# 5. JSON results export

Supported models:

• Any HuggingFace text-generation model
• Local models via Ollama
• OpenAI/Anthropic via API proxy

Example usage in dashboard:

1. Enter model ID: meta-llama/Llama-2-7b-chat-hf
2. Configure max tokens, temperature
3. Click "Run Benchmark"
4. View real-time progress
5. Download results JSON

3. Leaderboard

• Auto-loads from benchmark.json
• Displays average scores across all tasks
• Shows solve rate (tasks with score ≥ 0.70)
• Rank ordering by performance
• Task-specific score breakdown

4. About Page

• Full project documentation
• Architecture diagrams
• Task descriptions
• Action space reference
• Reward system explanation

Dashboard Startup

# Option 1: Via FastAPI server (recommended)
uvicorn server.app:app --host 0.0.0.0 --port 7860
# Dashboard available at: http://localhost:7860/dashboard

# Option 2: Standalone Gradio
python server/gradio_app.py
# Dashboard available at: http://localhost:7860

---

🔌 REST API Reference

The FastAPI server (server/app.py) provides OpenEnv-compliant endpoints:

Core Endpoints

POST /reset

Start a new episode for a specific task.

Request:

{
  "task_id": "task_expert"
}

Response:

{
  "observation": "Alert: High error rate detected on API Gateway...",
  "step": 0,
  "done": false,
  "alert": "High error rate detected...",
  "current_reward": 0.0,
  "cumulative_reward": 0.0
}

POST /step

Execute one action in the current episode.

Request:

{
  "action_type": "read_logs",
  "target": "auth-service"
}

Response:

{
  "observation": "Auth service logs show JWT validation errors...",
  "step": 1,
  "done": false,
  "current_reward": 0.15,
  "cumulative_reward": 0.15,
  "metrics": {
    "cpu_percent": 85.3,
    "memory_mb": 1024,
    "connections": 150
  }
}

GET /grade

Get final episode score.

Response:

{
  "score": 0.88,
  "task_id": "task_expert",
  "solved": true
}

GET /state

Get ground truth state (debug only — spoils the answer).

Response:

{
  "root_cause_service": "redis-cache",
  "secondary_faults": ["auth-service"],
  "red_herrings": ["notification-service"],
  "fault_type": "memory_eviction",
  "cascade_step": 15
}

GET /tasks

List all available tasks.

Response:

{
  "tasks": [
    {
      "id": "task_cpu_spike",
      "difficulty": "easy",
      "max_steps": 10,
      "description": "Auth service CPU hot loop"
    },
    ...
  ]
}

GET /health

Liveness check for monitoring.

Response:

{
  "status": "healthy",
  "version": "1.0.0"
}

Complete API Example (Bash)

# 1. Start episode
curl -X POST http://localhost:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task_id": "task_expert"}'

# 2. Investigate logs
curl -X POST http://localhost:7860/step \
  -H "Content-Type: application/json" \
  -d '{"action_type": "read_logs", "target": "api-gateway"}'

# 3. Check metrics
curl -X POST http://localhost:7860/step \
  -H "Content-Type: application/json" \
  -d '{"action_type": "check_metrics", "target": "redis-cache"}'

# 4. Declare RCA
curl -X POST http://localhost:7860/step \
  -H "Content-Type: application/json" \
  -d '{"action_type": "declare_rca", "target": "redis-cache"}'

# 5. Get final score
curl http://localhost:7860/grade

Complete API Example (PowerShell)

# 1. Start episode
$body = @{task_id = "task_expert"} | ConvertTo-Json
Invoke-RestMethod -Uri "http://localhost:7860/reset" -Method POST -ContentType "application/json" -Body $body

# 2. Investigate logs
$body = @{action_type = "read_logs"; target = "api-gateway"} | ConvertTo-Json
Invoke-RestMethod -Uri "http://localhost:7860/step" -Method POST -ContentType "application/json" -Body $body

# 3. Check metrics
$body = @{action_type = "check_metrics"; target = "redis-cache"} | ConvertTo-Json
Invoke-RestMethod -Uri "http://localhost:7860/step" -Method POST -ContentType "application/json" -Body $body

# 4. Declare RCA
$body = @{action_type = "declare_rca"; target = "redis-cache"} | ConvertTo-Json
Invoke-RestMethod -Uri "http://localhost:7860/step" -Method POST -ContentType "application/json" -Body $body

# 5. Get final score
Invoke-RestMethod -Uri "http://localhost:7860/grade"

---

🏗️ Architecture

System Architecture

┌─────────────────────────────────────────────────────────┐
│         Gradio Dashboard (Dark Theme + Amber UI)        │
│  Manual Playground | Custom Benchmarking | Leaderboard  │
└────────────────────┬────────────────────────────────────┘
                     ▼
┌─────────────────────────────────────────────────────────┐
│         FastAPI Server (server/app.py)                  │
│  /reset | /step | /grade | /state | /tasks | /health   │
└────────────────────┬────────────────────────────────────┘
                     ▼
┌─────────────────────────────────────────────────────────┐
│      IncidentResponseEnv (environment.py)               │
│  State management | Task dispatch | Trajectory logging  │
└────────────────────┬────────────────────────────────────┘
                     ▼
┌─────────────────────────────────────────────────────────┐
│  Reward System (reward.py - Domain-Aware)              │
│  • EvidenceTracker: Multi-type evidence collection      │
│  • Domain dispatch: Microservices | CI/CD | Kafka       │
│  • Sequence bonuses | Redundancy penalties | RCA scoring│
└────────────────────┬────────────────────────────────────┘
                     ▼
┌─────────────────────────────────────────────────────────┐
│  Supporting Components                                  │
│  • models.py: Pydantic data models                      │
│  • task_config.py: Task registry                        │
│  • simulators/: CI/CD + Kafka state machines            │
└─────────────────────────────────────────────────────────┘

File Structure

incident-response-env/
│
├── Core Environment
│   ├── environment.py          # Main Gym environment
│   ├── models.py               # Pydantic data models
│   ├── reward.py               # Domain-aware reward function
│   └── task_config.py          # Task ID registry
│
├── Server & UI
│   ├── server/
│   │   ├── app.py              # FastAPI REST API
│   │   ├── dashboard_impl.py   # Gradio dashboard implementation
│   │   └── gradio_app.py       # Standalone Gradio launcher
│   └── app.py                  # Gradio entry point wrapper
│
├── Evaluation & Training
│   ├── inference.py            # LLM agent benchmark runner
│   ├── benchmark_runner.py     # Full benchmark orchestration
│   └── training/
│       ├── expert_agent.py     # Rule-based expert (>0.80 on all tasks)
│       └── generate_data.py    # SFT trajectory generator
│
├── LLM Infrastructure
│   └── judge/
│       ├── llm_client.py       # Multi-provider client (OpenAI/Anthropic/Mock)
│       └── llm_judge.py        # Adversarial evaluation with phase enforcement
│
├── Simulators (Future Expansion)
│   └── simulators/
│       ├── cicd_simulator.py   # CI/CD pipeline state machine
│       └── kafka_simulator.py  # Kafka cluster state machine
│
├── Tasks & Data
│   ├── tasks/
│   │   ├── cicd_tasks.json     # CI/CD task definitions
│   │   └── kafka_tasks.json    # Kafka task definitions
│   └── sft_data/
│       ├── trajectories.jsonl  # Generated training data
│       └── generation_stats.json
│
├── Testing
│   ├── test/                   # Integration tests
│   │   ├── test_api_tasks.py
│   │   ├── test_full_episode.py
│   │   ├── test_inference_tasks.py
│   │   └── verify_inference_tasks.py
│   └── tests/                  # Unit tests (pytest)
│       ├── conftest.py
│       ├── test_environment.py
│       ├── test_llm_client.py
│       ├── test_llm_judge.py
│       └── test_reward.py
│
├── Documentation
│   └── docs/
│       ├── AGENT.md            # Agent operating manual
│       ├── BENCHMARK.md        # Benchmarking guide
│       ├── DESIGN.md           # Design philosophy
│       ├── ENVIRONMENT.md      # Environment specification
│       ├── REWARDS.md          # Reward engineering
│       └── SKILLS.md           # Agent capability taxonomy
│
├── Configuration
│   ├── Dockerfile              # Production container
│   ├── pyproject.toml          # Python package config
│   ├── requirements.txt        # Dependencies
│   ├── openenv.yaml            # OpenEnv specification
│   └── start.sh                # Docker startup script
│
└── README.md                   # This file

---

🤖 LLM Judge & Multi-Provider Support

LLM Judge (judge/llm_judge.py)

Adversarial evaluation with phase-order enforcement:

• Investigation Phase: Agent must gather evidence before fixing
• Remediation Phase: Agent can apply fixes only after investigation
• RCA Phase: Final root cause declaration with evidence justification

Judge scoring criteria:

• Evidence quality (relevant logs, metrics, health checks)
• Investigation thoroughness (covered all suspect services)
• Fix timing (evidence before intervention)
• RCA accuracy (correct service identified)

Multi-Provider LLM Client (judge/llm_client.py)

Supports multiple LLM providers with unified interface:

Features:

• Exponential backoff with configurable retries
• Graceful degradation across providers
• Detailed error diagnostics
• CPU-safe operation

---

📊 Baseline Performance & Agent Taxonomy

Agent Capability Tiers

Baseline Results

Key insights:

• Expert agent (rule-based) achieves >0.80 on ALL tasks (strong SFT data source)
• GPT-4o shows strong performance on easy/medium, struggles with multi-fault and long-horizon
• GPT-3.5 prone to red herring traps and premature RCA declarations
• Human-AI gap on expert tasks: ~0.30 points (opportunity for improvement)

---

🎓 Training & SFT Data Generation

Expert Agent (training/expert_agent.py)

Rule-based agent that serves as strong baseline and SFT data source:

Strategy:

1. Always start with gateway: Read logs to identify upstream fault
2. Follow dependency chain: Trace to suspect service
3. Gather all evidence: Logs, metrics, health, queries (if DB-related)
4. Apply correct fix type: Restart for runtime errors, rollback for deployment issues
5. Declare RCA: Only after gathering 3+ evidence types

Performance: >0.80 on all 16 tasks

SFT Data Generator (training/generate_data.py)

Exports expert trajectories in JSONL format for supervised fine-tuning:

python run_generate_data.py --num_episodes 100 --output sft_data/trajectories.jsonl

Output format:

{
  "task_id": "task_expert",
  "trajectory": [
    {"step": 0, "action": {"action_type": "read_logs", "target": "api-gateway"}, "observation": "...", "reward": 0.15},
    {"step": 1, "action": {"action_type": "check_metrics", "target": "redis-cache"}, "observation": "...", "reward": 0.12},
    ...
  ],
  "total_reward": 0.88,
  "final_score": 0.88,
  "rca_correct": true
}

Use cases:

• Supervised fine-tuning of LLMs
• Imitation learning from expert demonstrations
• Reward modeling for RLHF
• Behavioral cloning baselines

---

🏆 Benchmark Runner

Automated Evaluation (benchmark_runner.py)

Run comprehensive benchmarks across all tasks and models:

python benchmark_runner.py \
  --model gpt-4o \
  --episodes_per_task 5 \
  --output benchmark.json

Output format (benchmark.json):

{
  "leaderboard": [
    {
      "agent": "gpt-4o",
      "scores": {
        "task_cpu_spike": 0.92,
        "task_expert": 0.54,
        ...
      },
      "average": 0.70,
      "solved": 10,
      "rank": 1
    }
  ],
  "timestamp": "2026-04-26T14:30:00Z"
}

Leaderboard ranking:

• Average score across all tasks (higher is better)
• Solve rate: tasks with score ≥ 0.70
• Automatic rank assignment

---

🔬 Advanced Features

Multi-Fault Support

Task task_expert includes dual independent failures:

• Primary: Redis cache memory eviction
• Secondary: Auth service JWT validation errors
• Challenge: Agent must identify and fix BOTH root causes

Cascade Mechanics

Secondary faults emerge mid-episode:

• Example: Initial fix triggers configuration reload
• Configuration reload exposes latent bug
• New symptoms appear 10+ steps after initial fix
• Tests agent's ability to adapt to evolving incidents

Red Herrings

Deliberately misleading symptoms:

• Notification service shows high CPU (not the cause)
• API gateway error rate spike (victim, not perpetrator)
• Order service slow response (symptom of DB issue, not root cause)

Purpose: Test agent's ability to distinguish correlation from causation

Partial Observability

Agent sees only what it queries:

• No global state visibility
• Must strategically choose investigation targets
• Limited action budget (max steps per episode)
• Tests information gathering strategy

---

🚢 Deployment & Production

Docker Deployment

Dockerfile includes:

• Python 3.11+ runtime
• All dependencies from requirements.txt
• FastAPI server with Uvicorn
• Gradio dashboard
• Health check endpoint

Build and run:

docker build -t incident-response-env .
docker run -p 7860:7860 incident-response-env

Environment Variables

CI/CD Integration

GitHub Actions example:

name: Benchmark Agent
on: [pull_request]

jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - run: pip install -e .
      - env:
          API_BASE_URL: ${{ secrets.OPENAI_BASE_URL }}
          API_KEY: ${{ secrets.OPENAI_API_KEY }}
          MODEL_NAME: gpt-4o
        run: |
          uvicorn server.app:app --host 0.0.0.0 --port 7860 &
          sleep 5
          python inference.py
      - uses: actions/upload-artifact@v3
        with:
          name: benchmark-results
          path: benchmark.json

---

📚 Documentation

---

🤝 Contributing

We welcome contributions! Areas of interest:

New Fault Scenarios

• Network partition incidents
• Certificate rotation failures
• DNS propagation delays
• CDN cache poisoning

Additional Services

• Message queues (RabbitMQ, SQS)
• Load balancers (nginx, HAProxy)
• Service meshes (Istio, Linkerd)
• Monitoring systems (Prometheus, Grafana)

Improved Agents

• Reinforcement learning baselines (PPO, DQN)
• Multi-agent collaborative diagnosis
• Human-in-the-loop integration
• Active learning strategies

Tooling Enhancements

• Real-time visualization dashboards
• Multi-episode trajectory analysis
• Reward shaping experiments
• Adversarial task generation

See ENVIRONMENT.md for extension guide.

---

📈 Real-World Impact

Who benefits from solving this benchmark?

• Cloud providers (AWS, GCP, Azure) — Automated incident triage reducing MTTR by 60–80%
• DevOps teams — AI co-pilot for on-call engineers reducing alert fatigue
• SRE platforms (PagerDuty, Datadog, New Relic) — Intelligent root cause suggestion as product feature
• AI safety researchers — Reproducible benchmark for measuring agent causal reasoning under partial observability

Market context:

• Global SRE market: $8.7 billion (2024), growing at 15% CAGR
• Average cost of downtime: $5,600/minute for enterprise applications
• 70% of incidents require manual investigation (automation opportunity)

---

🏅 OpenEnv Compliance

This environment implements the OpenEnv Specification v1.0:

✅ Standardized REST API (/reset, /step, /grade, /state, /tasks)
✅ Structured observations with done flag and reward signals
✅ Task registry with difficulty levels and descriptions
✅ Reproducible episodes with fixed random seeds
✅ Documented action/observation spaces
✅ OpenEnv manifest (openenv.yaml)

Manifest highlights:

environment:
  name: "Incident Response Environment"
  version: "1.0.0"
  domain: "microservices_sre"
  
tasks:
  count: 16
  difficulty_levels: ["easy", "medium", "hard", "expert"]
  
action_space:
  type: "discrete"
  cardinality: 42  # 7 action types × 6 services
  
observation_space:
  type: "text"
  partial_observability: true

---

📄 License

Apache 2.0 — Free for research, commercial use, and derivative works.

See LICENSE for full terms.

---

🙏 Acknowledgments

Built for the OpenEnv × Scaler Hackathon 2026 with the goal of making AI agents reliable enough to be your on-call engineer.

Special thanks to:

• OpenEnv specification authors for standardized RL benchmarks
• HuggingFace for model hosting and inference infrastructure
• FastAPI and Gradio teams for excellent developer tools
• The SRE community for incident response best practices

---

--- title: Incident Response Env emoji: 🚨 colorFrom: red colorTo: blue sdk: docker pinned: false license: apache-2.0 short_description: LLM agents act as on-call SREs. ---

---

Production Incident Response Environment

An OpenEnv-compliant RL environment where LLM agents act as on-call SREs, investigating simulated microservices failures to identify root causes under time pressure.

Why this environment exists

Every tech company running software in production has incidents. A database slows down, an API starts throwing 500 errors, users cannot log in. The on-call engineer must investigate noisy, incomplete signals and identify the root cause as fast as possible. This environment simulates that exact task.

---

🌍 Why This Problem Matters

Every major tech company loses an average of $300,000 per hour during production incidents. (Gartner, 2023 — IT Downtime Cost Report)

When production goes down, an on-call SRE receives a pager alert at 3 AM. They have minutes — not hours — to investigate a cascade of noisy, incomplete signals across dozens of microservices, form a hypothesis, apply the right fix, and declare a root cause. Getting it wrong means extended downtime, lost revenue, and user churn.

This is not a solved problem. Current AI systems cannot reliably perform sequential diagnostic reasoning under:

• Partial observability — you only see what you query
• Active deception — red herrings are deliberately placed to mislead
• Time pressure — every wasted step increases downtime
• Cascading failures — victims look like culprits

incident-response-env is the first OpenEnv-compliant RL benchmark built around this exact challenge.

---

🎯 What Makes This Different

Unlike benchmarks where the agent sees everything at once, here the agent only knows what it queries. It must build a mental model of a broken system step by step — exactly as a human engineer would.

---

🆕 Recent Enhancements (Phase 1-5)

Multi-LLM Support with Resilient Fallbacks

• OpenAI & Anthropic Integration — Full support for GPT-4, GPT-4o, Claude-3 families
• Retry Logic — Exponential backoff with configurable retries for API failures
• Fallback Mechanisms — Graceful degradation across multiple LLM providers
• LiteLLM Proxy Compatible — Works with standardized OpenAI-compatible APIs

Trajectory Logging & SFT Data Collection

• JSONL Trajectories — Every episode saved as trajectories.jsonl for supervised fine-tuning
• Structured Trajectories — Each trajectory includes:
  • Step-by-step actions and observations
  • Per-step rewards and judge feedback
  • Total accumulated reward
  • Final score and RCA correctness flag
• Reward Metrics — Full visibility into reward signals for training LLMs with reward modeling

Enhanced Service Simulation

• Kafka Simulator — Event streaming tasks for complex incident scenarios
• Improved Health Monitoring — Status management with failure injection capabilities
• Service Registry — Comprehensive dependency graph and service metadata
• Cascading Failures — Multi-level fault propagation for realistic incident patterns

Phase 4 Domain-Aware Reward System

• reward.py integration — Domain-dispatched reward functions (microservices, CI/CD, Kafka)
• Evidence tracking — Domain-specific evidence counters (logs, metrics, integrity checks, lag analysis)
• Adaptive penalties — Redundancy penalties that scale with episode progress (early: gentle, late: harsh)
• RCA scoring — Efficiency bonus + evidence bonus for time-aware reward computation

Phase 1-2 Experimental Extensions

• CI/CD Simulator — GitHub Actions/GitLab CI incident scenarios (reward dispatch ready, integration pending)
• Kafka Simulator — Message streaming failure patterns (reward dispatch ready, integration pending)
• Expert Agent — Rule-based agent for SFT data generation
• Trajectory logging — Full episode paths saved as JSONL for supervised fine-tuning

See docs/ENVIRONMENT.md for simulator roadmap.

---

🎮 Action Space

The agent has 7 distinct action types, each with a specific diagnostic or remediation purpose:

{"action_type": "read_logs",           "target": "<service>"}
{"action_type": "check_metrics",       "target": "<service>"}
{"action_type": "check_health",        "target": "<service>"}
{"action_type": "run_db_query",        "target": "postgres-db"}
{"action_type": "restart_service",     "target": "<service>"}
{"action_type": "rollback_deployment", "target": "<service>"}
{"action_type": "declare_rca",         "target": "<service>"}

Available services: api-gateway · auth-service · order-service · notification-service · redis-cache · postgres-db

Observation space

Every action returns a rich observation:

📋 Task Suite

16 carefully crafted incident scenarios of increasing difficulty, from shallow (10 steps) to long-horizon planning (50 steps):

🚀 Long-Horizon Planning Test: task_expert_long_horizon

Addresses Hackathon Theme #2: Long-Horizon Planning

This 50-step task forces agents beyond shallow next-token reasoning:

• Initial fault: Postgres slow query causing gradual degradation
• Red herring: Agent might fix it at step 10–15 with a quick restart
• Latent secondary fault: The quick fix introduces a query planner bug
• Cascade trigger: At step 35+, order-service cascades due to secondary fault
• Required skill: Agent must track state over 50-step trajectory, recognize the latent bug, and implement the correct fix rather than jumping to quick conclusions

Why this matters: Tests whether agent can maintain context, plan ahead, and avoid optimization traps. A 50-step episode reveals whether the agent develops genuine SRE reasoning vs. pattern-matching lucky guesses.

🟡 Sample Scenario: Bad Deployment Cascade (Red Herring)

"Order service started failing after the 14:30 deployment. Auth service appears degraded."

A bad deployment on order-service cascades to make auth-service appear broken — a deliberate red herring. Logs show missing environment variables. The correct fix is rollback. Tests: Red herring resistance, multi-service correlation.

---

🔬 The Simulated Microservices System

The agent investigates a 6-service production stack comprising an api-gateway (always a victim), auth-service, order-service, notification-service, redis-cache, and postgres-db.

Key design principle: The gateway is always a victim, never the root cause. This forces agents to trace causality upstream.

---

Reward function

• +0.05 to +0.12 — relevant evidence found
• −0.08 — redundant action (early, before 50% steps)
• −0.20 — redundant action (late, after 50% steps)
• +0.30 — correct intervention (restart/rollback)
• −0.30 — wrong service restarted/rolled back
• +0.50 + evidence bonus + efficiency bonus — correct RCA declared
• −0.40 — wrong RCA declared
• Cumulative strictly clamped to [0.001, 0.999]

📊 Baseline Performance

The human-AI gap on task_hard is 0.40 points. Closing it requires genuine sequential reasoning, not pattern matching.

🤖 Agent Skill Taxonomy

The benchmark cleanly separates agents into four capability tiers:

The gap between Level 2 and Level 3 is red herring resistance — the most discriminative signal in this benchmark.

---

📊 Getting Started with Baselines

The environment ships with inference.py, a reference baseline agent that uses chain-of-thought prompting.

Supported LLM Endpoints

The baseline works with any OpenAI-compatible endpoint. OpenAI (GPT-4o) is the recommended baseline.

🌟 Quick Start with OpenAI (Recommended)

Bash:

export API_BASE_URL="https://api.openai.com/v1"
export API_KEY="sk_YOUR_OPENAI_KEY"
export MODEL_NAME="gpt-4o"

python inference.py

PowerShell:

$env:API_BASE_URL="https://api.openai.com/v1"
$env:API_KEY="sk_YOUR_OPENAI_KEY"
$env:MODEL_NAME="gpt-4o"

python inference.py

---

Other Supported Providers

To use other providers, change the API_BASE_URL, API_KEY, and MODEL_NAME accordingly.

---

📈 Interpreting Results

Score Breakdown

Each task returns a score in [0.001, 0.999]. Higher is better. For correct RCA declarations, the final score is computed as:

$$\text{score} = \text{clamp}(0.001, \text{base} + \text{evidence\_bonus} + \text{efficiency\_bonus}, 0.999)$$

Breaking it down:

Examples:

• Fast, thorough solve (0.88): Gather 4 evidence types, declare correct RCA at step 4 of 10

  • Base RCA: +0.50
  • Evidence bonus: min(4 × 0.05, 0.20) = +0.20
  • Efficiency bonus: (10-4)/10 × 0.30 = +0.18
  • Total: 0.50 + 0.20 + 0.18 = 0.88 (clamped to [0.001, 0.999])

• Slower correct solve (0.72): Gather 2 evidence types, declare correct RCA at step 8 of 10

  • Base RCA: +0.50
  • Evidence bonus: min(2 × 0.05, 0.20) = +0.10
  • Efficiency bonus: (10-8)/10 × 0.30 = +0.06
  • Total: 0.50 + 0.10 + 0.06 = 0.66 → +0.06 from careful investigation = 0.72

• Wrong diagnosis (−0.40): Declare incorrect RCA service

  • Result: −0.40 (hard penalty for overconfident guessing)

• Wrong intervention then correct RCA (0.35): Restart wrong service (−0.30), then declare correct RCA

  • Wrong service restart: −0.30
  • Correct RCA (base): +0.50 + bonuses (~0.15)
  • Total: −0.30 + 0.65 ≈ 0.35

Interpreting the Leaderboard

The leaderboard ranks agents by average score across all three tasks:

{
  "agent": "Qwen2.5-72B-Instruct",
  "scores": {
    "task_easy": 0.92,
    "task_medium": 0.68,
    "task_hard": 0.41
  },
  "average": 0.67,
  "solved": 2,  // tasks with score >= 0.70
  "rank": 5
}

Performance tiers:

• 0.85+: Expert SRE — rare; requires strong long-horizon planning
• 0.70–0.85: Competent investigator — solves most easy/medium tasks
• 0.40–0.70: Symptom chaser — finds right service but wrong fix or timing
• 0.15–0.40: Red herring victim — exhausts all services without focus
• 0.00–0.15: Random walker — no coherent strategy

Common Failure Patterns

• Exhaustive search without focus: Checks all 6 services before declaring RCA (high time penalty)
• Red herring trap: Fixes auth-service instead of order-service on task_medium (wrong fix bonus, severe penalty)
• Premature RCA: Declares at step 4 with only 1 evidence type (low RCA bonus, but no time penalty — can still score 0.60–0.70)
• Missing evidence: Finds the right service but never gathers supporting logs/metrics (evidence bonus capped, RCA bonus only; total ~0.70)

---

🔗 Integration and CI/CD

Embed the benchmark in GitHub Actions to test your agents automatically on every PR.

# .github/workflows/benchmark.yml
name: Benchmark Agent
on: [pull_request]
jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with: { python-version: '3.11' }
      - run: pip install -e .
      - env:
          API_BASE_URL: ${{ secrets.OPENAI_BASE_URL }}
          API_KEY: ${{ secrets.OPENAI_API_KEY }}
          MODEL_NAME: gpt-4o
        run: |
          uvicorn server.app:app --host 0.0.0.0 --port 7860 &
          sleep 3
          python inference.py > benchmark.json

---

🚀 Quick Start & Benchmarking

1. Local Setup

Clone the repository and start the FastAPI + Gradio server:

Bash/PowerShell:

git clone https://github.com/Praneeth0910/incident-response-env
cd incident-response-env
pip install -e .
uvicorn server.app:app --host 0.0.0.0 --port 7860

Open http://localhost:7860/dashboard to play manually.

2. Docker (Production)

docker-compose up --build

See docker-compose.yml for port mappings.

3. Run the LLM Agent Baseline

Configure your environment variables and run inference.py.

Unix / Bash:

export API_BASE_URL="https://api.openai.com/v1"
export API_KEY="sk_YOUR_OPENAI_KEY"
export MODEL_NAME="gpt-4o"
python inference.py

Windows / PowerShell:

$env:API_BASE_URL="https://api.openai.com/v1"
$env:API_KEY="sk_YOUR_OPENAI_KEY"
$env:MODEL_NAME="gpt-4o"
python inference.py

4. Direct API Usage (Programmatic)

Send REST commands directly to evaluate custom agents without inference.py:

# Start an episode
curl -X POST http://localhost:7860/reset -H "Content-Type: application/json" -d '{"task_id": "task_easy"}'

# Take step
curl -X POST http://localhost:7860/step -H "Content-Type: application/json" -d '{"action_type": "read_logs", "target": "api-gateway"}'

# Get score
curl http://localhost:7860/grade

---

🏗️ Architecture

The environment is built with a modular domain-aware architecture supporting multiple incident types:

┌─────────────────────────────────────────────────────────┐
│         User Interface (Gradio Dashboard)               │
└────────────────────┬────────────────────────────────────┘
                     ▼
┌─────────────────────────────────────────────────────────┐
│         FastAPI Server (port 7860)                      │
│  /reset, /step, /grade, /tasks, /state, /health        │
└────────────────────┬────────────────────────────────────┘
                     ▼
┌─────────────────────────────────────────────────────────┐
│      IncidentResponseEnv (State Machine)                │
│  Management, task dispatch, trajectory logging          │
└────────────────────┬────────────────────────────────────┘
                     ▼
┌─────────────────────────────────────────────────────────┐
│  Reward System (Phase 4 — Domain-Aware)                │
│  • Reward.py: Domain-dispatched reward functions        │
│  • EvidenceTracker: Multi-type evidence collection      │
│  • Support for: CI/CD, Kafka, Microservices             │
└────────────────────┬────────────────────────────────────┘
                     ▼
┌─────────────────────────────────────────────────────────┐
│  Simulators (Phase 1-2 Extensions — Experimental)      │
│  • cicd_simulator.py: GitHub Actions/GitLab CI          │
│  • kafka_simulator.py: Apache Kafka state machine       │
│  • Status: Integrated into reward.py, optional in env   │
└─────────────────────────────────────────────────────────┘

Current (Phase 1-5): Microservices incidents only.
Phase 1-2 simulators available via reward.py routing (see docs/ for roadmap).

---

📁 Repository Structure

incident-response-env/
├── environment.py          # Core RL environment (primary)
├── models.py               # Pydantic data models
├── reward.py               # Domain-aware reward functions
├── task_config.py          # Task ID registry
├── inference.py            # Benchmark inference script
├── benchmark_runner.py     # Full benchmark orchestration
├── app.py                  # Gradio entry point
├── start.sh                # Docker startup script
│
├── server/
│   ├── app.py              # FastAPI application + REST endpoints
│   ├── dashboard_impl.py   # Gradio terminal dashboard
│   └── gradio_app.py       # Standalone Gradio launcher
│
├── judge/
│   ├── llm_client.py       # LLM client (OpenAI / Anthropic / mock)
│   └── llm_judge.py        # Adversarial phase-aware judge
│
├── simulators/
│   ├── cicd_simulator.py   # CI/CD pipeline state machine
│   └── kafka_simulator.py  # Kafka cluster state machine
│
├── training/
│   ├── expert_agent.py     # Rule-based expert for SFT data generation
│   └── generate_data.py    # SFT dataset generator
│
├── tasks/
│   ├── cicd_tasks.json     # CI/CD task definitions
│   └── kafka_tasks.json    # Kafka task definitions
│
├── docs/
│   ├── AGENT.md            # Full agent operating manual
│   ├── ENVIRONMENT.md      # Complete environment specification
│   ├── BENCHMARK.md        # Multi-model benchmarking guide
│   ├── REWARDS.md          # Reward engineering deep dive
│   ├── SKILLS.md           # Agent capability taxonomy + prompt engineering
│   └── blog.md             # Technical blog — SRE AI innovation story
│
├── sft_data/               # Generated training trajectories
├── test/                   # Integration tests
├── tests/                  # Unit tests (pytest)
├── Dockerfile              # Production container
├── openenv.yaml            # OpenEnv specification manifest
└── README.md               # This file

---

🔌 REST API

Example session (Bash):

# Start episode
curl -X POST http://localhost:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task_id": "task_hard"}'

# Take action
curl -X POST http://localhost:7860/step \
  -H "Content-Type: application/json" \
  -d '{"action_type": "read_logs", "target": "redis-cache"}'

# Get score
curl http://localhost:7860/grade

Example session (PowerShell):

# Start episode
$body = @{task_id = "task_hard"} | ConvertTo-Json
Invoke-WebRequest -Uri "http://localhost:7860/reset" -Method POST -ContentType "application/json" -Body $body

# Take action
$body = @{action_type = "read_logs"; target = "redis-cache"} | ConvertTo-Json
Invoke-WebRequest -Uri "http://localhost:7860/step" -Method POST -ContentType "application/json" -Body $body

# Get score
Invoke-WebRequest http://localhost:7860/grade

---

Real-World Impact

Who benefits from solving this benchmark?

• Cloud providers (AWS, GCP, Azure) — automated incident triage could reduce MTTR by 60–80%
• DevOps teams — AI co-pilot for on-call engineers reduces alert fatigue
• SRE platforms (PagerDuty, OpsGenie, Datadog) — intelligent root cause suggestion as a product feature
• AI safety researchers — a reproducible benchmark for measuring agent causal reasoning under partial observability

The global Site Reliability Engineering market is valued at $8.7 billion (2024) and growing at 15% CAGR. Every percentage point improvement in automated incident resolution translates directly to engineering hours saved and service reliability improved.

---

📚 Documentation

---

🤝 Contributing

Want to add new fault types, tasks, or services? See ENVIRONMENT.md for the extension guide.

Pull requests welcome for:

• New fault scenarios (network partition, disk full, certificate expiry)
• Additional services (message queues, CDN, load balancer)
• Improved baseline agents
• Multi-agent collaborative diagnosis

---

📄 License

Apache 2.0 — free for research, commercial use, and derivative works.

---