Introduction

Colony is a multi-agent orchestration system for Claude Code running on tmux. It enables you to run multiple Claude Code instances in parallel with proper isolation, state management, and coordination.

What is Colony?

Colony provides infrastructure for managing multiple AI agents working together on complex software engineering tasks. Each agent runs in its own isolated environment (tmux pane) with dedicated resources, while a shared state system enables coordination and collaboration.

Key Features

Multi-Agent Orchestration

Run multiple Claude Code instances simultaneously
Each agent operates in an isolated tmux pane
Automatic session management and health monitoring
Interactive TUI for real-time monitoring

Shared State Management

Git-backed state storage (JSONL + SQLite)
Distributed task queues with dependency resolution
Shared memory and context
Automatic state synchronization

Workflow Orchestration

Define complex multi-agent workflows in YAML
DAG-based dependency resolution
Retry policies and error handling
Workflow history and status tracking

Agent Templates

Reusable agent configurations
Built-in template library (code-reviewer, security-auditor, test-engineer, etc.)
Custom template support
Role-based specialization

Plugin System

Extensible architecture
Backend, UI, and Tool plugins
Discovery and lifecycle management
Enable/disable plugin management

Enhanced Logging & Observability

Structured logging with filtering
JSON log output support
Real-time TUI with metrics panel
Health checks and status monitoring

Use Cases

Parallel Development: Multiple agents working on different features simultaneously
Code Review: Specialized review agents analyzing different aspects of code
Testing & QA: Automated testing agents running comprehensive test suites
Documentation: Documentation agents maintaining technical docs
Security Audits: Security-focused agents scanning for vulnerabilities
Data Analysis: Analyst agents processing and visualizing data

Architecture

Colony uses a modular architecture with:

Tmux Integration: Process isolation and session management
Git-Backed State: Reliable, versioned state storage
YAML Configuration: Human-readable configuration files
Rust Implementation: Fast, safe, and reliable execution

Quick Example

# Initialize a new colony
colony init

# Start all agents
colony start

# Monitor with TUI
colony tui

# View agent status
colony status

# Check logs
colony logs --level info

Next Steps

Installation - Install Colony
Quick Start - Get started in 5 minutes
Core Concepts - Understand the fundamentals
CLI Reference - Complete command reference

Installation

This guide will help you install Colony on your system.

Prerequisites

Required

Rust (1.70+): Install from rustup.rs
tmux (2.0+): Terminal multiplexer for agent isolation
Git: For state management
Claude Code: The AI coding assistant that agents run on

Optional

MCP Servers: For extended functionality

Installing Colony

Using Cargo (Recommended)

The easiest way to install Colony is via Cargo from crates.io:

cargo install cc-colony

The colony binary will be installed to ~/.cargo/bin and available in your PATH.

Building from Source

Alternatively, you can clone and build from source:

# Clone the repository
git clone https://github.com/mcfearsome/cc-colony.git
cd cc-colony

# Install to ~/.cargo/bin
cargo install --path .

Installing Claude Code

Colony requires Claude Code to run agents. Install it using npm:

npm install -g @anthropic-ai/claude-code

Or visit the Claude Code documentation for alternative installation methods.

Installing tmux

macOS

brew install tmux

Ubuntu/Debian

sudo apt-get install tmux

Fedora/RHEL

sudo dnf install tmux

Arch Linux

sudo pacman -S tmux

Verifying Installation

Check that everything is installed correctly:

# Check Colony version
colony --version

# Check Claude Code version
claude --version

# Check tmux version
tmux -V

# Check Git version
git --version

# Check Rust version
rustc --version

Configuration

After installation, initialize your first colony:

# Navigate to your project directory
cd /path/to/your/project

# Initialize colony configuration
colony init

This creates a colony.yml configuration file in your project directory.

Updating Colony

To update Colony to the latest version:

cargo install cc-colony --force

Or if you installed from source:

cd cc-colony
git pull
cargo install --path . --force

Troubleshooting

tmux not found

If colony start fails with "tmux not found":

Install tmux using your package manager
Verify tmux is in your PATH: which tmux

Permission denied

If you get permission errors:

# Ensure the binary is executable
chmod +x ~/.cargo/bin/colony

Build failures

If cargo build fails:

Update Rust: rustup update
Clean build artifacts: cargo clean
Try building again: cargo build --release

Next Steps

Quick Start - Create your first colony
Configuration - Configure your colony

Quick Start

Get started with Colony in 5 minutes!

Step 1: Initialize a Colony

Navigate to your project directory and initialize:

cd /path/to/your/project
colony init

This creates a colony.yml configuration file with default settings.

Step 2: Configure Your Agents

Edit colony.yml to define your agents:

agents:
  - id: main-dev
    worktree_branch: feature/main-work
    startup_prompt: |
      You are the main development agent. Focus on implementing
      core features and coordinating with other agents.

  - id: code-reviewer
    worktree_branch: review/automated
    template: code-reviewer
    startup_prompt: |
      Review code changes for quality and best practices.

  - id: test-writer
    worktree_branch: test/automated
    template: test-engineer
    startup_prompt: |
      Write comprehensive tests for new features.

Step 3: Start the Colony

colony start

This command:

Creates a tmux session named colony
Sets up git worktrees for each agent
Launches Claude Code instances in separate panes
Initializes shared state

Step 4: Monitor Your Agents

Option 1: Attach to tmux

colony attach

View agents working in real-time. Press Ctrl+b d to detach.

Option 2: Use the TUI

colony tui

Interactive dashboard with:

Agent status and health
Real-time metrics
Task queue visualization
Log viewing

Option 3: Check Status

colony status

Quick overview of all agents.

Step 5: View Logs

# View all logs
colony logs

# View specific agent
colony logs agent-id

# Filter by level
colony logs --level error

# Follow logs in real-time
colony logs --follow

Step 6: Work with State

Create Tasks

# Add a new task
colony state task add "Implement login feature" \
  --description "Add OAuth login support" \
  --blockers task-123

# List tasks
colony state task list

# List ready tasks (no blockers)
colony state task ready

Sync State

# Pull latest state from remote
colony state pull

# Push local changes
colony state push

# Full sync (pull + push)
colony state sync

Step 7: Stop When Done

# Stop all agents
colony stop

# Stop specific agent
colony stop agent-id

Example: Code Review Workflow

Here's a common workflow for automated code review:

# 1. Initialize colony
colony init

# 2. Install code-reviewer template
colony template install code-reviewer

# 3. Configure agents
cat > colony.yml <<EOF
agents:
  - id: main-dev
    worktree_branch: feature/new-api
  - id: reviewer
    worktree_branch: review/api
    template: code-reviewer
EOF

# 4. Start colony
colony start

# 5. Create a task for the main dev
colony state task add "Implement new API endpoint"

# 6. Monitor progress
colony tui

# 7. When done, sync state and stop
colony state sync
colony stop

Next Steps

Configuration - Learn about advanced configuration
Core Concepts - Understand agents and colonies
Templates - Use and create templates
Workflows - Define complex workflows

Authentication

Colony supports multiple authentication methods to work with Claude AI models, allowing you to choose the approach that best fits your workflow and subscription type.

Supported Methods

Method	Best For	Setup Time
OAuth	Claude Pro/Max users	1 minute
API Key	Developers with API access	30 seconds
Bedrock	AWS/Enterprise users	2 minutes
Vertex AI	Google Cloud users	Coming soon

Quick Start

OAuth (Recommended for Pro/Max Users)

Perfect for individual developers with Claude Pro or Max subscriptions.

# One command - opens browser
colony auth login

# Follow browser prompts
# ✅ Done!

What happens:

Opens your browser to claude.ai
Login with your existing account
Approve access
Automatically saves credentials
Ready to use!

Advantages:

✅ No API key needed
✅ Uses your existing subscription
✅ Secure OAuth 2.0 flow
✅ Automatic token refresh

API Key (For Direct API Access)

For developers with Anthropic API keys.

# Interactive setup
colony auth login --method api-key

# Or set environment variable
export ANTHROPIC_API_KEY=sk-ant-api03-xxxxx

# Verify
colony auth status

Advantages:

✅ Simple setup
✅ Programmatic access
✅ Works in CI/CD pipelines
✅ No browser required

Bedrock (For AWS Users)

For organizations using AWS infrastructure.

# Configure AWS credentials first
aws configure --profile my-profile

# Then authenticate Colony
colony auth login --method bedrock \
  --region us-east-1 \
  --profile my-profile

# Verify
colony auth status

Advantages:

✅ Enterprise-grade
✅ AWS billing integration
✅ IAM role support
✅ Regional deployment

Configuration

Add to your colony.yml:

OAuth Configuration

auth:
  provider: anthropic-oauth
  token_path: ~/.colony/auth/tokens.json

API Key Configuration

# Option 1: Environment variable (recommended)
auth:
  provider: api-key
  # Uses ANTHROPIC_API_KEY from environment

# Option 2: Explicit key (not recommended)
auth:
  provider: api-key
  api_key: sk-ant-api03-xxxxx  # Don't commit this!

Bedrock Configuration

auth:
  provider: bedrock
  region: us-east-1
  profile: default  # AWS profile name

OAuth Deep Dive

How It Works

Authentication Request
- Colony starts local server on port 8888
- Opens browser to claude.ai/oauth/authorize
- Generates secure PKCE challenge
User Authorization
- You log in to claude.ai
- Review and approve access
- Browser redirects to localhost:8888
Token Exchange
- Colony receives authorization code
- Exchanges code for access token
- Stores token securely with 0600 permissions
Automatic Refresh
- Tokens expire after 30 days
- Colony auto-refreshes when needed
- Or manually: colony auth refresh

Security Features

PKCE (Proof Key for Code Exchange)

Prevents authorization code interception
No client secret needed
Secure for public clients (CLIs)

State Parameter

CSRF protection
Validates redirect authenticity

Secure Storage

Tokens stored in ~/.colony/auth/tokens.json
File permissions: 0600 (owner read/write only)
Not in project directory (safe from git)

Token Management

Check status:

colony auth status

Output:

🔐 Authentication Status

Provider: Claude.ai OAuth (Pro/Max)
Status: ✅ Authenticated
Expires: in 29 days, 14 hours

Refresh token:

colony auth refresh

Logout:

colony auth logout

API Key Best Practices

Environment Variables

~/.bashrc or ~/.zshrc:

export ANTHROPIC_API_KEY=sk-ant-api03-xxxxx

Per-project (.env file):

# .env
ANTHROPIC_API_KEY=sk-ant-api03-xxxxx

Then:

# Load environment
source .env

# Or use direnv
echo 'export ANTHROPIC_API_KEY=sk-ant-...' > .envrc
direnv allow

Key Rotation

# 1. Generate new key in Anthropic Console
# 2. Update environment variable
export ANTHROPIC_API_KEY=sk-ant-new-key

# 3. Verify
colony auth status

# 4. Revoke old key in Anthropic Console

Security Checklist

Never commit keys to version control
Add .env to .gitignore
Use separate keys for dev/prod
Rotate keys every 90 days
Monitor usage in Anthropic Console
Revoke compromised keys immediately

Bedrock Setup

Prerequisites

AWS Account with Bedrock access
AWS CLI installed and configured
Bedrock models enabled in AWS Console

Step-by-Step

1. Enable Bedrock Models

# In AWS Console:
# - Go to Amazon Bedrock
# - Click "Model access"
# - Request access to Claude models
# - Wait for approval (usually instant)

2. Configure AWS Credentials

# Method 1: AWS CLI
aws configure --profile my-colony
# Enter: Access Key, Secret Key, Region

# Method 2: IAM Role (EC2/ECS)
# Automatically uses instance role

3. Test Access

# Verify credentials
aws sts get-caller-identity --profile my-colony

# Test Bedrock access
aws bedrock list-foundation-models \
  --region us-east-1 \
  --profile my-colony

4. Authenticate Colony

colony auth login --method bedrock \
  --region us-east-1 \
  --profile my-colony

IAM Permissions

Required IAM policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "arn:aws:bedrock:*::foundation-model/anthropic.claude-*"
    }
  ]
}

Cost Management

Monitor costs:

# AWS Cost Explorer
aws ce get-cost-and-usage \
  --time-period Start=2024-01-01,End=2024-01-31 \
  --granularity MONTHLY \
  --metrics BlendedCost \
  --filter file://bedrock-filter.json

Set budget alerts:

AWS Budgets
Alert when costs exceed threshold
Email/SNS notifications

Troubleshooting

OAuth Issues

Problem: Browser doesn't open

# Manually open the URL shown in terminal
# URL format: https://claude.ai/oauth/authorize?...

Problem: "Port 8888 already in use"

# Find what's using port 8888
lsof -i :8888

# Kill the process
kill -9 <PID>

# Or use different port (future feature)

Problem: Token expired

# Refresh token
colony auth refresh

# Or re-authenticate
colony auth logout
colony auth login

API Key Issues

Problem: "Invalid API key"

# Check key format (must start with sk-ant-)
echo $ANTHROPIC_API_KEY

# Verify in Anthropic Console
# Keys > Your API keys

# Test key
colony auth login --method api-key --api-key sk-ant-...

Problem: Key not found

# Check environment variable is set
env | grep ANTHROPIC

# Add to shell config
echo 'export ANTHROPIC_API_KEY=sk-ant-...' >> ~/.bashrc
source ~/.bashrc

Bedrock Issues

Problem: "Access denied"

# Check AWS credentials
aws sts get-caller-identity

# Verify Bedrock access enabled
aws bedrock list-foundation-models --region us-east-1

# Check IAM permissions
# Need: bedrock:InvokeModel

Problem: "Model not available in region"

# Claude models available in:
# - us-east-1 (N. Virginia)
# - us-west-2 (Oregon)
# - eu-west-1 (Ireland)
# - ap-southeast-1 (Singapore)

# Change region
colony auth login --method bedrock --region us-east-1

Authentication Status

Check Current Auth

colony auth status

Example output:

OAuth:

🔐 Authentication Status

Provider: Claude.ai OAuth (Pro/Max)
Status: ✅ Authenticated
Expires: in 29 days, 14 hours

API Key:

🔐 Authentication Status

Provider: Anthropic API Key
Status: ✅ Configured via environment variable
Validation: ✅ API key is valid

Bedrock:

🔐 Authentication Status

Provider: AWS Bedrock
Region: us-east-1
Profile: default
Status: ✅ Connected

Switching Authentication Methods

From OAuth to API Key

# 1. Logout from OAuth
colony auth logout

# 2. Update colony.yml
# Change: provider: anthropic-oauth
# To:     provider: api-key

# 3. Set API key
export ANTHROPIC_API_KEY=sk-ant-...

# 4. Verify
colony auth status

From API Key to Bedrock

# 1. Configure AWS
aws configure --profile bedrock-profile

# 2. Update colony.yml
# Change: provider: api-key
# To:     provider: bedrock
#         region: us-east-1
#         profile: bedrock-profile

# 3. Authenticate
colony auth login --method bedrock

# 4. Verify
colony auth status

Multiple Projects

Different Auth Per Project

# Project 1: OAuth
cd ~/project1
cat colony.yml  # auth: provider: anthropic-oauth
colony start

# Project 2: API Key
cd ~/project2
cat colony.yml  # auth: provider: api-key
export ANTHROPIC_API_KEY=sk-ant-proj2-...
colony start

# Project 3: Bedrock
cd ~/project3
cat colony.yml  # auth: provider: bedrock
colony start

Shared OAuth Token

OAuth tokens are stored globally:

~/.colony/auth/tokens.json

Multiple projects can use the same OAuth authentication:

# All projects can use this
auth:
  provider: anthropic-oauth
  token_path: ~/.colony/auth/tokens.json

CI/CD Integration

GitHub Actions

name: Colony Tests

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Colony
        run: |
          # Install colony
          curl -sSL https://install.colony.sh | sh

      - name: Authenticate
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: |
          # colony.yml uses environment variable
          colony auth status

      - name: Run tests
        run: |
          colony start --no-attach
          # Run tests...
          colony stop

GitLab CI

colony-test:
  image: ubuntu:latest
  variables:
    ANTHROPIC_API_KEY: $ANTHROPIC_API_KEY
  script:
    - curl -sSL https://install.colony.sh | sh
    - colony auth status
    - colony start --no-attach
    - # Run tests
    - colony stop

Security Considerations

Credential Storage

Method	Storage Location	Permissions	Encryption
OAuth	`~/.colony/auth/tokens.json`	0600	At rest (future)
API Key	Environment variable	N/A	In memory only
Bedrock	`~/.aws/credentials`	0600	AWS managed

Best Practices

For Individuals:

✅ Use OAuth for convenience
✅ Enable 2FA on claude.ai account
✅ Logout from shared machines

For Teams:

✅ Use API keys with rotation
✅ Separate dev/staging/prod keys
✅ Monitor usage per key

For Enterprise:

✅ Use Bedrock with IAM roles
✅ Enable CloudTrail logging
✅ Set up cost alerts
✅ Use SCPs for guardrails

Next Steps

Configuration

Colony uses a colony.yml file to configure agents, state, and behavior.

Basic Structure

agents:
  - id: agent-1
    worktree_branch: feature/agent-1
    startup_prompt: "Your role and instructions"

state:
  backend: git
  location: .colony/state
  schema: beads

tmux:
  session_name: colony
  shell: /bin/bash

Agent Configuration

Required Fields

agents:
  - id: unique-agent-id          # Unique identifier
    worktree_branch: branch-name # Git branch for this agent

Optional Fields

agents:
  - id: my-agent
    worktree_branch: feature/work

    # Template to use (optional)
    template: code-reviewer

    # Startup prompt (overrides template)
    startup_prompt: |
      Your custom instructions here.
      Can be multi-line.

    # MCP servers (optional)
    mcp_servers:
      filesystem:
        command: npx
        args: [-y, "@modelcontextprotocol/server-filesystem", "/path"]

      github:
        command: npx
        args: [-y, "@modelcontextprotocol/server-github"]
        env:
          GITHUB_TOKEN: ${GITHUB_TOKEN}

State Configuration

Git Backend (Recommended)

state:
  backend: git
  location: .colony/state
  schema: beads

  git:
    auto_commit: true
    auto_push: false
    commit_message_prefix: "[colony]"

In-Memory Backend (Testing)

state:
  backend: memory
  schema: beads

Tmux Configuration

tmux:
  # Session name (default: "colony")
  session_name: colony

  # Shell to use (default: /bin/bash)
  shell: /bin/bash

  # Window layout (default: "tiled")
  layout: tiled

  # Custom tmux configuration
  config:
    status: "on"
    mouse: "on"

Template Usage

Using built-in templates:

agents:
  - id: reviewer
    worktree_branch: review/auto
    template: code-reviewer  # Built-in template

  - id: security
    worktree_branch: security/scan
    template: security-auditor

    # Override template's startup prompt
    startup_prompt: |
      Focus specifically on OWASP Top 10 vulnerabilities.

List available templates:

colony template builtin

Environment Variables

Colony supports environment variable substitution:

agents:
  - id: api-dev
    worktree_branch: feature/api
    mcp_servers:
      database:
        command: npx
        args: [-y, "@modelcontextprotocol/server-postgres"]
        env:
          DATABASE_URL: ${DATABASE_URL}
          API_KEY: ${API_KEY}

Set environment variables before starting:

export DATABASE_URL="postgresql://..."
export API_KEY="secret"
colony start

Multiple Agent Examples

Parallel Development

agents:
  - id: frontend
    worktree_branch: feature/ui
    startup_prompt: "Focus on React components and UI"

  - id: backend
    worktree_branch: feature/api
    startup_prompt: "Focus on API endpoints and database"

  - id: reviewer
    worktree_branch: review/all
    template: code-reviewer

Testing Pipeline

agents:
  - id: dev
    worktree_branch: feature/new-feature
    startup_prompt: "Implement the feature"

  - id: unit-tests
    worktree_branch: test/unit
    template: test-engineer
    startup_prompt: "Write unit tests"

  - id: integration-tests
    worktree_branch: test/integration
    template: test-engineer
    startup_prompt: "Write integration tests"

  - id: security
    worktree_branch: security/scan
    template: security-auditor

Documentation Team

agents:
  - id: dev
    worktree_branch: feature/work

  - id: docs
    worktree_branch: docs/auto
    template: documentation-writer
    startup_prompt: |
      Document new features as they are implemented.
      Keep docs in sync with code changes.

Advanced Configuration

Custom Logging

logging:
  level: info  # debug, info, warn, error
  format: json # text, json
  output: .colony/logs

  filters:
    - pattern: "^(INFO|WARN)"
      destination: .colony/logs/filtered.log

Workflow Integration

workflows:
  # Path to workflow definitions
  definitions: .colony/workflows

  # Auto-start workflows on init
  auto_start:
    - code-review-pipeline
    - daily-security-scan

Plugin Configuration

plugins:
  # Plugins directory
  directory: .colony/plugins

  # Auto-enable specific plugins
  enabled:
    - metrics-collector
    - custom-notifier

Configuration Validation

Validate your configuration:

# Colony automatically validates on start
colony start

# Or explicitly check
colony init --validate

Next Steps

Agents - Learn about agent concepts
State Management - Understand shared state
Templates - Use and create templates

Agents

Agents are individual Claude Code instances running in isolated environments.

Overview

Each agent in Colony:

Runs in its own tmux pane
Has its own git worktree
Can use specialized templates
Shares state with other agents
Has a unique ID and configuration

Agent Lifecycle

Configuration - Defined in colony.yml
Initialization - Worktree and environment setup
Start - Claude Code instance launched
Running - Agent performs tasks
Stop - Graceful shutdown

Agent Isolation

Agents are isolated through:

Separate git worktrees (no conflicts)
Individual tmux panes (process isolation)
Unique working directories
Independent MCP server instances

Communication

Agents communicate through:

Shared state system
Task queue
Message passing
Shared memory

See State Management for details.

Colonies

A colony is a collection of agents working together on a project.

What is a Colony?

A colony consists of:

Multiple agent instances
Shared state system
Tmux session for isolation
Configuration file (colony.yml)

Colony Structure

my-project/
├── colony.yml          # Configuration
├── .colony/            # Colony data
│   ├── state/          # Shared state
│   ├── logs/           # Agent logs
│   ├── templates/      # Templates
│   └── plugins/        # Plugins
└── .git-worktrees/     # Agent worktrees

Benefits

Parallel execution
Specialization
Coordination
State sharing

State Management

Colony uses a git-backed state system for reliable agent coordination.

Architecture

Backend: Git (JSONL + SQLite)
Schema: Beads-inspired design
Sync: Push/pull model

State Components

Tasks

Distributed task queue
Dependency tracking
Status management

Workflows

Multi-step processes
State transitions
History tracking

Memory

Shared context
Learned insights
Decision history

See Git-Backed State for implementation details.

Workflows

Workflows define complex multi-agent processes.

Overview

Workflows are DAG-based sequences of steps with:

Dependencies between steps
Retry policies
Error handling
Input/output passing

Workflow Definition

name: code-review-pipeline
description: Automated code review workflow

steps:
  - id: run-tests
    agent: test-engineer
    action: run_tests

  - id: security-scan
    agent: security-auditor
    action: security_scan
    depends_on: [run-tests]

  - id: code-review
    agent: code-reviewer
    action: review
    depends_on: [run-tests, security-scan]

See Workflow CLI for commands.

Templates

Templates provide reusable agent configurations for common roles and tasks.

What are Templates?

Templates are pre-configured agent setups that include:

Role definition: What the agent does
Focus areas: Specific responsibilities
Model selection: Which Claude model to use
Startup prompts: Initial instructions
MCP server configs: Tool integrations (optional)

Built-in Templates

Colony includes 5 built-in templates:

code-reviewer

Reviews code for quality, best practices, and potential issues.

Focus:

Code quality and style
Best practices adherence
Performance considerations
Maintainability

Use case: Automated code review before merging

security-auditor

OWASP Top 10 focused security auditing.

Focus:

Security vulnerabilities
Authentication/authorization issues
Input validation
Dependency vulnerabilities

Use case: Security scanning and vulnerability detection

test-engineer

Automated testing and QA specialist.

Focus:

Unit test coverage
Integration testing
Test quality and maintainability
Edge case identification

Use case: Writing comprehensive test suites

documentation-writer

Technical documentation specialist.

Focus:

API documentation
User guides
Code comments
README files

Use case: Maintaining up-to-date documentation

data-analyst

Data analysis and insights agent.

Focus:

Data processing
Statistical analysis
Visualization
Insights generation

Use case: Analyzing data and generating reports

Using Templates

List Available Templates

# List built-in templates
colony template builtin

# List installed templates
colony template list

Install a Template

# Install a built-in template
colony template install code-reviewer

# View template details
colony template show code-reviewer

Use in Configuration

agents:
  - id: reviewer
    worktree_branch: review/auto
    template: code-reviewer  # Use installed template

Override Template Settings

You can override any template setting:

agents:
  - id: reviewer
    worktree_branch: review/auto
    template: code-reviewer

    # Override the startup prompt
    startup_prompt: |
      Review code with extra focus on:
      - TypeScript type safety
      - React best practices
      - Performance optimization

Template Structure

Templates are stored as YAML files in .colony/templates/:

# .colony/templates/my-template/template.yaml
name: my-custom-template
version: 1.0.0
author: Your Name
description: Custom agent template
license: MIT

agent:
  role: Custom Role
  focus: Specific focus area
  model: claude-sonnet-4-20250514

  startup_prompt: |
    Your custom startup prompt here.

  mcp_servers:
    filesystem:
      command: npx
      args: [-y, "@modelcontextprotocol/server-filesystem", "."]

requirements:
  repo_types:
    - source
    - application

Creating Custom Templates

Step 1: Create Template Directory

mkdir -p .colony/templates/my-template

Step 2: Create template.yaml

# .colony/templates/my-template/template.yaml
name: my-template
version: 1.0.0
description: My custom agent template

agent:
  role: My Role
  focus: My focus area
  model: claude-sonnet-4-20250514

  startup_prompt: |
    Custom instructions for this agent.

Step 3: Use Your Template

agents:
  - id: my-agent
    worktree_branch: feature/work
    template: my-template

Template Best Practices

Clear Role Definition

agent:
  role: Frontend Developer
  focus: React components, TypeScript, and styling

Specific Instructions

agent:
  startup_prompt: |
    Focus on:
    1. Component reusability
    2. Type safety with TypeScript
    3. Accessibility (WCAG 2.1 AA)
    4. Performance optimization

Appropriate Model Selection

agent:
  # Use Sonnet for most tasks (balanced performance/cost)
  model: claude-sonnet-4-20250514

  # Or Opus for complex reasoning tasks
  model: claude-opus-4-20250514

  # Or Haiku for simple, fast tasks
  model: claude-3-5-haiku-20241022

Repository Type Requirements

requirements:
  repo_types:
    - source        # Source code repositories
    - application   # Full applications
    - library       # Libraries and packages
    - documentation # Documentation projects

Template Examples

Full-Stack Developer

name: fullstack-dev
version: 1.0.0
description: Full-stack development specialist

agent:
  role: Full-Stack Developer
  focus: End-to-end feature development
  model: claude-sonnet-4-20250514

  startup_prompt: |
    You are a full-stack developer responsible for:
    - Frontend (React/TypeScript)
    - Backend (Node.js/Python)
    - Database design and queries
    - API development
    - Testing and documentation

requirements:
  repo_types:
    - application

DevOps Engineer

name: devops-engineer
version: 1.0.0
description: DevOps and infrastructure specialist

agent:
  role: DevOps Engineer
  focus: CI/CD, deployment, and infrastructure
  model: claude-sonnet-4-20250514

  startup_prompt: |
    Focus on:
    - CI/CD pipeline configuration
    - Docker and container orchestration
    - Infrastructure as code
    - Monitoring and logging
    - Performance optimization

  mcp_servers:
    docker:
      command: npx
      args: [-y, "@modelcontextprotocol/server-docker"]

requirements:
  repo_types:
    - application
    - infrastructure

Export Template

# Template is just a directory
tar -czf my-template.tar.gz .colony/templates/my-template

Import Template

# Extract to templates directory
cd .colony/templates
tar -xzf /path/to/my-template.tar.gz

# Commit templates directory
git add .colony/templates/
git commit -m "Add custom templates"
git push

Next Steps

Custom Templates Guide - Advanced template creation
CLI Reference - Template commands
Configuration - Using templates in config

Plugins

Plugins extend Colony functionality.

Plugin Types

Backend Plugins

Extend core functionality:

Custom state backends
Integration points
Data processors

UI Plugins

Enhance the TUI:

Custom panels
Visualizations
Dashboards

Tool Plugins

Add capabilities:

MCP server integrations
External tools
Custom commands

Plugin Structure

.colony/plugins/my-plugin/
├── plugin.yaml
└── src/
    └── main.rs

See Plugin Development for creating plugins.

Code Review Workflow

Set up an automated code review workflow with specialized agents.

Overview

This tutorial shows how to create a colony that:

Has a development agent implementing features
Has a reviewer agent checking code quality
Uses task coordination for review workflow
Integrates with your development process

Setup

Step 1: Initialize Colony

cd /path/to/your/project
colony init

Configure 2 agents:

developer - Main development agent
reviewer - Code review specialist

Step 2: Configure Agents

Edit colony.yml:

name: code-review-workflow

agents:
  - id: developer
    role: Software Developer
    focus: Implement features and fix bugs
    model: claude-sonnet-4-20250514
    worktree: agent/dev
    
  - id: reviewer
    role: Code Reviewer
    focus: Review code for quality, best practices, and potential issues
    model: claude-opus-4-20250514
    worktree: agent/review
    startup_prompt: |
      You are a code reviewer focused on:
      - Code quality and maintainability
      - Best practices and design patterns
      - Potential bugs and edge cases
      - Performance considerations
      - Security vulnerabilities
      
      Review changes thoroughly and provide constructive feedback.

Step 3: Start Colony

colony start

Usage Workflow

1. Developer Implements Feature

In the developer pane:

> I need to implement user authentication with JWT tokens

The developer agent will:

Create auth module
Implement login/register endpoints
Add JWT token generation
Write tests

2. Create Review Task

colony state task add "Review authentication implementation" \
  --description "Review JWT auth code for security and best practices"

3. Reviewer Checks Code

In the reviewer pane, assign the task:

> Review the authentication implementation in the auth module

The reviewer will:

Check security vulnerabilities
Verify input validation
Review error handling
Check test coverage
Suggest improvements

4. Address Feedback

Developer implements suggested changes:

> Address the security concerns raised by the reviewer:
> 1. Add rate limiting to login endpoint
> 2. Implement token rotation
> 3. Add comprehensive input validation

5. Final Review

colony state task update <task-id> completed

Advanced: Automated Workflow

Create a workflow file .colony/workflows/code-review.yaml:

name: code-review-pipeline
description: Automated code review workflow

steps:
  - id: implement
    agent: developer
    action: implement_feature
    
  - id: review
    agent: reviewer
    action: review_code
    depends_on: [implement]
    
  - id: address-feedback
    agent: developer
    action: address_feedback
    depends_on: [review]
    
  - id: final-check
    agent: reviewer
    action: final_review
    depends_on: [address-feedback]

Run the workflow:

colony workflow run code-review-pipeline

Monitoring

Watch Progress

colony tui

The TUI shows:

Agent status
Task queue
Active work
Completed tasks

View Logs

# All logs
colony logs

# Developer logs
colony logs developer

# Reviewer logs
colony logs reviewer

# Follow in real-time
colony logs reviewer --follow

Tips

Use Task Dependencies

Structure work with dependencies:

# Create feature task
colony state task add "Implement auth" --id task-auth

# Create review task that depends on implementation
colony state task add "Review auth" \
  --blockers task-auth

Template-Based Reviews

Use the built-in code-reviewer template:

colony template install code-reviewer

Update colony.yml:

agents:
  - id: reviewer
    role: Code Reviewer
    focus: Quality and security review
    template: code-reviewer

Multiple Review Stages

Add specialized reviewers:

agents:
  - id: developer
    role: Developer
    
  - id: security-reviewer
    role: Security Auditor
    template: security-auditor
    
  - id: code-reviewer
    role: Code Reviewer
    template: code-reviewer

Real-World Example

Pull Request Review

# 1. Developer creates feature branch and implements
colony start
# Developer works in their pane

# 2. Create PR review task
colony state task add "Review PR #123" \
  --description "Review authentication feature PR"

# 3. Reviewer checks the changes
# In reviewer pane: git diff main...feature/auth

# 4. Create follow-up tasks
colony state task add "Fix security issue in auth.rs:45"
colony state task add "Add tests for token expiration"

# 5. Developer addresses issues
# In developer pane: work through tasks

# 6. Final approval
colony state task update pr-123 completed

Benefits

Parallel Work: Developer continues on next feature while reviewer checks previous work
Consistent Reviews: Reviewer follows same process every time
Documentation: All feedback and changes tracked in tasks
Isolation: No git conflicts between dev and review worktrees
Persistence: Colony state persists across sessions

Next Steps

Parallel Development - Multiple developers working together
Testing Pipeline - Automated testing workflow
State Management - Learn about task coordination

Parallel Development

Run multiple developers working on different features simultaneously.

Scenario

You need to implement three features in parallel:

User authentication system
Payment processing
Email notification service

Configuration

name: parallel-dev

agents:
  - id: auth-dev
    role: Authentication Developer
    focus: User authentication, JWT tokens, OAuth
    model: claude-sonnet-4-20250514
    worktree: agent/auth
    startup_prompt: |
      Focus on authentication features:
      - Implement secure login/register
      - JWT token management
      - Password hashing with bcrypt
      - Session management

  - id: payments-dev
    role: Payments Developer
    focus: Payment processing, Stripe integration
    model: claude-sonnet-4-20250514
    worktree: agent/payments
    startup_prompt: |
      Focus on payment features:
      - Stripe API integration
      - Webhook handling
      - Payment validation
      - Error handling

  - id: email-dev
    role: Email Developer
    focus: Email notifications, templating
    model: claude-sonnet-4-20250514
    worktree: agent/email
    startup_prompt: |
      Focus on email features:
      - Email template system
      - SMTP configuration
      - Queue management
      - Delivery tracking

Workflow

1. Start All Agents

colony start

2. Create Feature Tasks

# Authentication feature
colony state task add "Implement JWT authentication" \
  --description "User login/register with JWT tokens"

# Payments feature
colony state task add "Integrate Stripe payments" \
  --description "Payment processing with Stripe"

# Email feature
colony state task add "Build email notification system" \
  --description "Send transactional emails"

3. Agents Work Independently

Each agent works in their own worktree without conflicts:

auth-dev works on agent/auth branch
payments-dev works on agent/payments branch
email-dev works on agent/email branch

4. Monitor Progress

colony tui

View all three agents working simultaneously:

Real-time task status
Agent activity
Completion progress

5. Integration Phase

When features are ready, merge them:

# Each agent's work is on their branch
git checkout main
git merge agent/auth
git merge agent/payments
git merge agent/email

Advanced: Cross-Agent Dependencies

Some features depend on others:

# Payments needs auth
colony state task add "Add payment endpoints" \
  --blockers task-auth-complete

# Email needs both
colony state task add "Send payment confirmation emails" \
  --blockers task-auth-complete,task-payments-complete

Coordination

Shared State

Agents can communicate via shared state:

# Auth dev marks API contract ready
colony state memory add context \
  --key "auth_api_contract" \
  --value "POST /api/auth/login, POST /api/auth/register"

# Payments dev can reference it
colony state memory search "auth_api"

Message Passing

# Send message to specific agent
colony messages send payments-dev \
  "Auth API is ready at /api/auth/*"

Benefits

No Git Conflicts: Each agent has isolated worktree
Parallel Execution: All agents work simultaneously
Independent Progress: Features don't block each other
Clear Separation: Each agent focuses on their domain
Easy Integration: Merge branches when ready

Real-World Example

Startup Sprint

name: startup-sprint

agents:
  - id: frontend
    role: Frontend Developer
    focus: React, TypeScript, UI components

  - id: backend
    role: Backend Developer
    focus: Node.js, Express, API endpoints

  - id: database
    role: Database Engineer
    focus: PostgreSQL, migrations, queries

  - id: devops
    role: DevOps Engineer
    focus: Docker, deployment, CI/CD

All four agents work in parallel:

colony start
# Frontend builds UI while backend implements API
# Database designs schema while DevOps sets up infrastructure

Security Auditing

Automated security scanning and vulnerability detection.

Setup

Install Template

colony template install security-auditor

Configure Colony

name: security-audit

agents:
  - id: scanner
    role: Security Auditor
    focus: OWASP Top 10, vulnerability detection
    template: security-auditor
    model: claude-opus-4-20250514
    worktree: agent/security

Usage

1. Full Codebase Scan

colony start

In the scanner pane:

> Perform a comprehensive security audit of the entire codebase.
> Focus on OWASP Top 10 vulnerabilities.

2. Specific Module Scan

> Audit the authentication module (src/auth/) for security issues

3. Review New PR

> Review PR #45 for security vulnerabilities
> git diff main...feature/new-api

What Gets Checked

The security auditor looks for:

Injection Flaws

SQL injection
Command injection
NoSQL injection
LDAP injection

Authentication Issues

Weak password policies
Insecure session management
Missing MFA
Credential exposure

XSS Vulnerabilities

Reflected XSS
Stored XSS
DOM-based XSS

Access Control

Missing authorization checks
Insecure direct object references
Path traversal

Security Misconfiguration

Default credentials
Unnecessary features enabled
Missing security headers
Verbose error messages

Cryptographic Failures

Weak algorithms
Hardcoded secrets
Insecure random number generation
Missing encryption

Dependency Vulnerabilities

Outdated packages
Known CVEs
Vulnerable dependencies

Reports

Generate Report

colony state memory add note \
  --content "Security audit report for v1.2.0"

# View audit findings
colony logs scanner --level warn

Export Findings

colony logs scanner --json > security-report.json

Integration with CI/CD

Pre-Commit Hook

# .git/hooks/pre-commit
#!/bin/bash
colony start --no-attach
colony state task add "Security scan staged files"
# Wait for scan completion
colony logs scanner --level error

GitHub Actions

name: Security Audit
on: [pull_request]

jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run Security Audit
        run: |
          colony init --template security-auditor
          colony start --no-attach
          colony logs scanner --json > audit.json
      - name: Upload Report
        uses: actions/upload-artifact@v2
        with:
          name: security-report
          path: audit.json

Documentation Team

Automatically maintain documentation as code changes.

Concept

Keep documentation synchronized with code by having dedicated documentation agents monitor changes and update docs.

Configuration

name: docs-team

agents:
  - id: developer
    role: Software Developer
    focus: Feature implementation
    model: claude-sonnet-4-20250514
    worktree: agent/dev

  - id: docs-writer
    role: Documentation Writer
    focus: API docs, user guides, README updates
    template: documentation-writer
    model: claude-sonnet-4-20250514
    worktree: agent/docs
    startup_prompt: |
      Monitor code changes and keep documentation updated:
      - API documentation
      - README files
      - User guides
      - Code comments
      - Architecture docs

Workflow

1. Developer Implements Feature

Developer adds new API endpoint:

// src/api/users.ts
export async function createUser(data: CreateUserDTO): Promise<User> {
  // Implementation
}

2. Notify Docs Writer

colony state task add "Document new user creation API" \
  --description "New POST /api/users endpoint added"

3. Docs Writer Updates Documentation

Docs writer agent:

Reads the new code
Updates API documentation
Adds usage examples
Updates CHANGELOG

Creates:

# API Documentation

## Create User

**Endpoint**: `POST /api/users`

**Request Body**:
```json
{
  "email": "user@example.com",
  "name": "John Doe"
}

Response:

{
  "id": "user_123",
  "email": "user@example.com",
  "name": "John Doe",
  "createdAt": "2024-01-15T10:30:00Z"
}

Example:

const user = await createUser({
  email: "user@example.com",
  name: "John Doe"
});


## Automated Documentation

### Watch for Changes

```bash
# Set up a workflow
cat > .colony/workflows/auto-docs.yaml <<EOF
name: auto-documentation
description: Automatically update docs when code changes

trigger:
  type: file_change
  pattern: "src/**/*.ts"

steps:
  - id: detect-changes
    agent: developer
    action: list_changes

  - id: update-docs
    agent: docs-writer
    action: update_documentation
    depends_on: [detect-changes]

Testing Pipeline

Automated test generation and QA workflow.

Overview

Set up a comprehensive testing pipeline with specialized test agents.

Configuration

name: testing-pipeline

agents:
  - id: developer
    role: Software Developer
    focus: Feature implementation
    model: claude-sonnet-4-20250514
    worktree: agent/dev

  - id: unit-tester
    role: Unit Test Engineer
    template: test-engineer
    model: claude-sonnet-4-20250514
    worktree: agent/unit-tests

  - id: integration-tester
    role: Integration Test Engineer
    template: test-engineer
    model: claude-sonnet-4-20250514
    worktree: agent/integration-tests

Workflow

1. TDD Approach

# Create test task first
colony state task add "Write login tests"

# Feature task depends on tests
colony state task add "Implement login" --blockers test-login

2. Run Tests

# Unit tests
npm test -- --coverage

# Integration tests
npm run test:integration

Test Types

Unit Tests

Test individual functions
Mock dependencies
Aim for 90%+ coverage

Integration Tests

Test API endpoints
Database interactions
Third-party integrations

E2E Tests

Complete user journeys
Browser automation
Real environment testing

Metrics

Track test coverage:

colony metrics record test.coverage 87.5
colony metrics show test.coverage

Benefits

High test coverage
Fast feedback
Regression prevention
CI/CD ready

Data Pipeline

Build an automated data processing pipeline with specialized agents for ETL workflows.

Overview

This example demonstrates how to use Colony for data engineering tasks: extracting data from sources, transforming it, loading into destinations, and validating results.

Use Case

Scenario: Daily data pipeline that:

Fetches data from multiple APIs
Cleans and transforms the data
Enriches with additional sources
Validates data quality
Loads into data warehouse
Generates reports

Architecture

┌─────────────┐
│   Fetcher   │ ──┐
└─────────────┘   │
                  ├─→ ┌──────────────┐
┌─────────────┐   │   │              │      ┌──────────┐
│ Transformer │ ──┼─→ │ Orchestrator │ ───→ │  Loader  │
└─────────────┘   │   │              │      └──────────┘
                  ├─→ └──────────────┘           │
┌─────────────┐   │                              │
│  Validator  │ ──┘                              ▼
└─────────────┘                           ┌────────────┐
                                          │ Warehouse  │
                                          └────────────┘

Configuration

colony.yml

name: data-pipeline-colony

# Enable state for pipeline coordination
shared_state:
  backend: git-backed
  location: in-repo

agents:
  - id: orchestrator
    role: Pipeline Orchestrator
    model: claude-sonnet-4-20250514
    worktree_branch: pipeline/orchestrator
    startup_prompt: |
      You are the pipeline orchestrator responsible for coordinating
      the entire ETL workflow.

      RESPONSIBILITIES:
      - Coordinate agent tasks and dependencies
      - Monitor pipeline health
      - Handle failures and retries
      - Generate pipeline reports

      WORKFLOW:
      1. Create tasks for data fetcher
      2. Wait for fetch completion
      3. Assign transform tasks
      4. Coordinate validation
      5. Trigger load when validated
      6. Generate completion report

  - id: data-fetcher
    role: Data Fetcher
    model: claude-sonnet-4-20250514
    worktree_branch: pipeline/fetcher
    env:
      API_KEY: $DATA_API_KEY
      RATE_LIMIT: "100"
    startup_prompt: |
      You are responsible for fetching data from external sources.

      DATA SOURCES:
      - REST APIs (with rate limiting)
      - CSV files from S3
      - Database exports

      REQUIREMENTS:
      - Handle API rate limits gracefully
      - Implement retry logic with exponential backoff
      - Validate data completeness before marking complete
      - Store raw data in staging/raw/
      - Log all fetch operations

      ERROR HANDLING:
      - API errors: Retry up to 3 times
      - Network timeouts: Implement circuit breaker
      - Data corruption: Flag for manual review

  - id: data-transformer
    role: Data Transformer
    model: claude-sonnet-4-20250514
    worktree_branch: pipeline/transformer
    startup_prompt: |
      Transform raw data into clean, structured format.

      TRANSFORMATIONS:
      - Data cleaning (nulls, duplicates, outliers)
      - Schema normalization
      - Type conversions
      - Data enrichment
      - Aggregations

      QUALITY STANDARDS:
      - All dates in ISO 8601 format
      - Standardized field names (snake_case)
      - Required fields must be non-null
      - Numeric fields validated for range
      - Document all transformations applied

      OUTPUT:
      - Store in staging/transformed/
      - Generate transformation report
      - Flag records that couldn't be transformed

  - id: data-validator
    role: Data Quality Validator
    model: claude-sonnet-4-20250514
    worktree_branch: pipeline/validator
    startup_prompt: |
      Validate data quality before loading to warehouse.

      VALIDATION CHECKS:
      - Schema compliance
      - Data completeness (% non-null)
      - Referential integrity
      - Business rule validation
      - Statistical anomaly detection

      THRESHOLDS:
      - Completeness: >= 95%
      - Duplicate rate: < 1%
      - Error rate: < 0.1%

      ACTIONS:
      - PASS: Mark for loading
      - WARN: Load but flag issues
      - FAIL: Block and alert orchestrator

  - id: data-loader
    role: Data Loader
    model: claude-sonnet-4-20250514
    worktree_branch: pipeline/loader
    env:
      WAREHOUSE_CONNECTION: $WAREHOUSE_URL
    startup_prompt: |
      Load validated data into the data warehouse.

      LOADING STRATEGY:
      - Use bulk insert for efficiency
      - Implement idempotent loads (upsert)
      - Handle conflicts (update vs append)
      - Maintain load history

      SAFETY:
      - Always use transactions
      - Verify row counts before/after
      - Create backup before major loads
      - Log all load operations

      POST-LOAD:
      - Update metadata tables
      - Refresh materialized views
      - Run post-load validations
      - Generate load report

  - id: reporter
    role: Report Generator
    model: claude-sonnet-4-20250514
    worktree_branch: pipeline/reporter
    startup_prompt: |
      Generate pipeline execution reports and metrics.

      REPORTS:
      - Daily pipeline summary
      - Data quality metrics
      - Performance statistics
      - Error analysis

      METRICS:
      - Records processed
      - Processing time
      - Error rates
      - Data quality scores

      OUTPUT:
      - Markdown reports in reports/
      - JSON metrics in metrics/
      - Send alerts for anomalies

Workflow Setup

1. Initialize Pipeline

# Initialize colony
colony init

# Copy configuration
cat > colony.yml < [paste config above]

# Create directory structure
mkdir -p staging/{raw,transformed,validated}
mkdir -p reports metrics logs

# Start the colony
colony start

2. Define Workflow

Create .colony/workflows/daily-etl.yaml:

workflow:
  name: daily-etl
  description: "Daily data pipeline execution"

  trigger:
    type: schedule
    cron: "0 2 * * *"  # Run at 2 AM daily

  steps:
    - name: fetch-data
      agent: data-fetcher
      timeout: 10m
      instructions: |
        Fetch data from all configured sources for yesterday's date.
        Store raw data in staging/raw/ with timestamp.
      output: raw_data_path

    - name: transform-data
      agent: data-transformer
      depends_on: [fetch-data]
      timeout: 15m
      instructions: |
        Transform raw data from {{steps.fetch-data.output}}.
        Apply all cleaning and normalization rules.
        Store in staging/transformed/
      output: transformed_data_path
      retry:
        max_attempts: 2
        backoff: exponential

    - name: validate-data
      agent: data-validator
      depends_on: [transform-data]
      timeout: 5m
      instructions: |
        Validate data at {{steps.transform-data.output}}.
        Run all quality checks and generate validation report.
      output: validation_status

    - name: load-data
      agent: data-loader
      depends_on: [validate-data]
      timeout: 20m
      instructions: |
        Load validated data to warehouse.
        Only proceed if validation_status is PASS or WARN.
      output: load_stats

    - name: generate-report
      agent: reporter
      depends_on: [load-data]
      timeout: 5m
      instructions: |
        Generate daily pipeline report with:
        - Records processed: {{steps.load-data.output.record_count}}
        - Quality score: {{steps.validate-data.output.quality_score}}
        - Processing time: total pipeline duration
        - Any warnings or errors

  error_handling:
    - step: notify-on-failure
      agent: orchestrator
      instructions: "Send alert about pipeline failure"

3. Create Task Dependencies

# Orchestrator creates the task chain
colony state task add "fetch-daily-data" \
  --description "Fetch data from all sources"

colony state task add "transform-daily-data" \
  --description "Clean and transform fetched data" \
  --blockers "fetch-daily-data"

colony state task add "validate-daily-data" \
  --description "Run quality validation" \
  --blockers "transform-daily-data"

colony state task add "load-daily-data" \
  --description "Load to warehouse" \
  --blockers "validate-daily-data"

colony state task add "generate-daily-report" \
  --description "Create pipeline report" \
  --blockers "load-daily-data"

Execution

Manual Run

# Start the pipeline manually
colony workflow run daily-etl

# Monitor in TUI
colony tui
# Tab 4: Watch workflow progress
# Tab 2: Monitor task completion

# Check specific agent logs
colony logs data-fetcher
colony logs data-validator

# View workflow status
colony workflow status <run-id>

Monitoring

# Real-time monitoring in separate terminal
watch -n 5 'colony status && echo "" && colony state task list'

# Check for errors
colony logs --level error

# View metrics
colony metrics show pipeline_records_processed
colony metrics show pipeline_duration

Agent Scripts

Data Fetcher Implementation

Create scripts/fetch_data.py:

#!/usr/bin/env python3
import os
import requests
import json
from datetime import datetime, timedelta
from pathlib import Path
import time

class DataFetcher:
    def __init__(self):
        self.api_key = os.environ.get('API_KEY')
        self.rate_limit = int(os.environ.get('RATE_LIMIT', 100))
        self.staging_dir = Path('staging/raw')
        self.staging_dir.mkdir(parents=True, exist_ok=True)

    def fetch_api_data(self, endpoint, date):
        """Fetch data from API with retry logic."""
        max_retries = 3
        for attempt in range(max_retries):
            try:
                response = requests.get(
                    endpoint,
                    params={'date': date, 'api_key': self.api_key},
                    timeout=30
                )
                response.raise_for_status()
                return response.json()
            except requests.exceptions.RequestException as e:
                if attempt == max_retries - 1:
                    raise
                wait_time = 2 ** attempt
                print(f"Retry {attempt + 1}/{max_retries} after {wait_time}s")
                time.sleep(wait_time)

    def save_raw_data(self, data, source_name, date):
        """Save raw data to staging."""
        filename = f"{source_name}_{date}.json"
        filepath = self.staging_dir / filename

        with open(filepath, 'w') as f:
            json.dump({
                'source': source_name,
                'fetch_time': datetime.now().isoformat(),
                'date': date,
                'record_count': len(data),
                'data': data
            }, f, indent=2)

        return str(filepath)

    def run(self):
        """Main fetch process."""
        yesterday = (datetime.now() - timedelta(days=1)).strftime('%Y-%m-%d')

        sources = [
            ('sales_api', 'https://api.example.com/sales'),
            ('inventory_api', 'https://api.example.com/inventory'),
        ]

        results = {}
        for source_name, endpoint in sources:
            print(f"Fetching from {source_name}...")
            data = self.fetch_api_data(endpoint, yesterday)
            filepath = self.save_raw_data(data, source_name, yesterday)
            results[source_name] = {
                'filepath': filepath,
                'records': len(data)
            }
            print(f"✓ Fetched {len(data)} records to {filepath}")

        # Update state
        with open('.colony/state/fetch_results.json', 'w') as f:
            json.dump(results, f, indent=2)

        return results

if __name__ == '__main__':
    fetcher = DataFetcher()
    results = fetcher.run()
    print(f"\nFetch complete: {sum(r['records'] for r in results.values())} total records")

Validator Implementation

Create scripts/validate_data.py:

#!/usr/bin/env python3
import json
import pandas as pd
from pathlib import Path
from datetime import datetime

class DataValidator:
    def __init__(self):
        self.thresholds = {
            'completeness': 0.95,
            'duplicate_rate': 0.01,
            'error_rate': 0.001
        }

    def validate_schema(self, df, required_columns):
        """Check schema compliance."""
        missing = set(required_columns) - set(df.columns)
        if missing:
            return False, f"Missing columns: {missing}"
        return True, "Schema valid"

    def check_completeness(self, df):
        """Check data completeness."""
        completeness = df.notna().sum() / len(df)
        avg_completeness = completeness.mean()

        return (
            avg_completeness >= self.thresholds['completeness'],
            f"Completeness: {avg_completeness:.2%}"
        )

    def check_duplicates(self, df, key_columns):
        """Check for duplicates."""
        duplicates = df.duplicated(subset=key_columns).sum()
        duplicate_rate = duplicates / len(df)

        return (
            duplicate_rate <= self.thresholds['duplicate_rate'],
            f"Duplicates: {duplicate_rate:.2%} ({duplicates} records)"
        )

    def validate(self, filepath):
        """Run all validation checks."""
        df = pd.read_json(filepath)

        results = {
            'file': filepath,
            'timestamp': datetime.now().isoformat(),
            'record_count': len(df),
            'checks': []
        }

        # Run checks
        checks = [
            ('schema', self.validate_schema(df, ['id', 'date', 'value'])),
            ('completeness', self.check_completeness(df)),
            ('duplicates', self.check_duplicates(df, ['id', 'date'])),
        ]

        all_passed = True
        for check_name, (passed, message) in checks:
            results['checks'].append({
                'name': check_name,
                'passed': passed,
                'message': message
            })
            all_passed = all_passed and passed

        results['status'] = 'PASS' if all_passed else 'FAIL'
        return results

if __name__ == '__main__':
    import sys
    validator = DataValidator()
    results = validator.validate(sys.argv[1])

    print(json.dumps(results, indent=2))

    # Save validation report
    report_path = Path('reports') / f"validation_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json"
    report_path.parent.mkdir(exist_ok=True)
    with open(report_path, 'w') as f:
        json.dump(results, f, indent=2)

Best Practices

Error Handling

Implement Circuit Breakers:

class CircuitBreaker:
    def __init__(self, failure_threshold=5, timeout=60):
        self.failure_count = 0
        self.failure_threshold = failure_threshold
        self.timeout = timeout
        self.last_failure_time = None
        self.state = 'CLOSED'  # CLOSED, OPEN, HALF_OPEN

    def call(self, func, *args, **kwargs):
        if self.state == 'OPEN':
            if time.time() - self.last_failure_time > self.timeout:
                self.state = 'HALF_OPEN'
            else:
                raise Exception("Circuit breaker OPEN")

        try:
            result = func(*args, **kwargs)
            if self.state == 'HALF_OPEN':
                self.state = 'CLOSED'
                self.failure_count = 0
            return result
        except Exception as e:
            self.failure_count += 1
            self.last_failure_time = time.time()

            if self.failure_count >= self.failure_threshold:
                self.state = 'OPEN'
            raise

Data Quality Metrics

Track key metrics:

# Record metrics after each pipeline run
colony metrics record pipeline_records_processed 10543
colony metrics record pipeline_duration_seconds 1834
colony metrics record pipeline_error_count 3
colony metrics record data_quality_score 0.98

Alerting

# In reporter agent script
if [ $ERROR_COUNT -gt 10 ]; then
  colony broadcast "🚨 Pipeline errors exceeded threshold: $ERROR_COUNT errors"
fi

if [ $QUALITY_SCORE -lt 0.95 ]; then
  colony broadcast "⚠️ Data quality below threshold: $QUALITY_SCORE"
fi

Monitoring Dashboard

View pipeline health in TUI:

colony tui

# Tab 1: Monitor agent status
# Tab 2: Track task progress
# Tab 4: View workflow state
# Tab 5: Check metrics

# Look for:
# - All agents running (no failures)
# - Tasks completing in sequence
# - No blocked tasks
# - Quality metrics within range

Advanced Features

Incremental Loading

def get_last_load_timestamp():
    """Get timestamp of last successful load."""
    metadata_file = '.colony/state/load_metadata.json'
    if Path(metadata_file).exists():
        with open(metadata_file) as f:
            metadata = json.load(f)
            return metadata.get('last_load_time')
    return None

def fetch_incremental(since_timestamp):
    """Fetch only data since last load."""
    # Only fetch new/updated records
    pass

Parallel Processing

# In workflow definition
steps:
  - name: transform-data
    agent: data-transformer
    parallel: 4  # Spawn 4 parallel workers
    instructions: "Transform batch {{batch_id}} of data"

Data Lineage

Track data provenance:

lineage = {
    'source': 'sales_api',
    'fetch_time': '2024-01-15T02:00:00Z',
    'transformations': [
        {'type': 'clean_nulls', 'rows_affected': 123},
        {'type': 'normalize_dates', 'rows_affected': 10543},
    ],
    'validation_passed': True,
    'load_time': '2024-01-15T02:45:00Z',
    'warehouse_table': 'sales_fact'
}

Troubleshooting

Pipeline Stuck:

# Check which step is blocking
colony workflow status <run-id>
colony state task list

# View agent logs
colony logs data-fetcher --level error

Data Quality Issues:

# Review validation reports
cat reports/validation_*.json | jq '.checks[] | select(.passed == false)'

# Check transformation logic
colony logs data-transformer --pattern "transform"

Performance Issues:

# Check metrics
colony metrics show pipeline_duration_seconds --hours 168  # Last week

# Identify slow steps
colony workflow history daily-etl --limit 10

CLI Reference Overview

Colony provides a comprehensive command-line interface for managing multi-agent systems.

Command Structure

colony [COMMAND] [SUBCOMMAND] [OPTIONS]

Available Commands

Setup & Management

colony init - Initialize a new colony
colony start - Start all agents
colony stop - Stop agents
colony destroy - Destroy colony and cleanup

Monitoring

colony status - Show agent status
colony health - System health check
colony tui - Interactive TUI dashboard
colony logs - View agent logs
colony attach - Attach to tmux session

Agent Templates

colony template list - List templates
colony template show - Show template details
colony template install - Install built-in template
colony template builtin - List built-in templates

Plugins

colony plugin list - List plugins
colony plugin show - Show plugin details
colony plugin enable - Enable a plugin
colony plugin disable - Disable a plugin

Workflows

colony workflow list - List workflows
colony workflow show - Show workflow details
colony workflow run - Run a workflow
colony workflow status - Check run status
colony workflow history - View run history
colony workflow cancel - Cancel a run

State Management

colony state task - Manage tasks
colony state workflow - Manage workflow state
colony state memory - Manage shared memory
colony state pull - Pull from remote
colony state push - Push to remote
colony state sync - Full sync

Communication

colony broadcast - Broadcast message to all agents
colony messages list - List agent messages
colony messages all - List all messages

Global Options

--help, -h     Show help information
--version, -V  Show version information

Common Patterns

Quick Status Check

colony status && colony health

View Recent Errors

colony logs --level error --lines 50

Start and Monitor

colony start && colony tui

Sync and Stop

colony state sync && colony stop

Exit Codes

0 - Success
1 - General error
2 - Command error
130 - Interrupted (Ctrl+C)

Environment Variables

# Colony configuration file (default: colony.yml)
export COLONY_CONFIG=/path/to/config.yml

# Log level (default: info)
export COLONY_LOG_LEVEL=debug

# Tmux session name (default: colony)
export COLONY_TMUX_SESSION=my-session

Shell Completion

Generate shell completions:

# Bash
colony completions bash > ~/.local/share/bash-completion/completions/colony

# Zsh
colony completions zsh > ~/.zsh/completions/_colony

# Fish
colony completions fish > ~/.config/fish/completions/colony.fish

Next Steps

colony init - Initialize a colony
colony start - Start agents
colony template - Manage templates
colony workflow - Manage workflows

colony init

Initialize a new colony configuration with an interactive wizard.

Synopsis

colony init

Description

The init command launches an interactive wizard that guides you through creating a colony.yml configuration file. The wizard asks questions about your desired setup and generates a complete configuration.

Interactive Wizard

The wizard will guide you through:

Agent Count - How many agents to create (default: 2)
Agent Configuration - For each agent:
- Agent ID (unique identifier)
- Template selection (5 built-in templates + manual)
- Role customization
- Focus area customization
- Claude model selection
- Optional startup prompt override
Colony Naming - Optional colony name

Template Selection

Choose from 5 built-in templates:

Code Reviewer - Quality and best practices review
Security Auditor - OWASP Top 10 security scanning
Test Engineer - Automated testing and QA
Documentation Writer - Technical documentation
Data Analyst - Data analysis and insights
None - Manual configuration

Model Selection

Choose from three Claude models:

claude-sonnet-4-20250514 - Balanced performance (recommended)
claude-opus-4-20250514 - Most capable, best for complex tasks
claude-3-5-haiku-20241022 - Fast and efficient

Examples

Basic Initialization

$ colony init

Colony Configuration Wizard
──────────────────────────────────────────────────

This wizard will help you create a colony.yml configuration.
Press Ctrl+C at any time to cancel.

? How many agents do you want to create? (2)

Quick Setup with Defaults

Press Enter to accept all defaults:

$ colony init
# Accept defaults throughout
✓ Created colony.yml
✓ Initialized task queue directories

This creates 2 agents with sensible defaults.

Custom Configuration

Customize each agent:

$ colony init

? How many agents do you want to create? 3

Agent 1 Configuration
? Agent ID: backend
? Choose a template: None - I'll configure manually
? Agent role: Backend Engineer
? Agent focus: API endpoints and database
? Use a different Claude model? Yes
? Select Claude model: claude-opus-4-20250514 (most capable)

Prerequisites

Must be run inside a Git repository
Git must be initialized (git init if needed)

Output

After completion, the wizard:

Creates colony.yml in the current directory
Initializes .colony/ directory structure
Displays the generated configuration
Shows next steps

Generated Configuration Example

name: my-project
agents:
  - id: backend
    role: Backend Engineer
    focus: API endpoints and database
    model: claude-opus-4-20250514
    worktree: agent/backend
  - id: frontend
    role: Frontend Developer
    focus: React components and UI
    model: claude-sonnet-4-20250514
    worktree: agent/frontend

Next Steps

After initialization:

# Start the colony
colony start

# Monitor with TUI
colony tui

# Check status
colony status

Troubleshooting

"colony must be run inside a Git repository"

Initialize Git first:

git init
colony init

colony start

Start all agents in the colony.

Synopsis

colony start [OPTIONS]

Description

The start command launches all configured agents in a tmux session. Each agent runs in its own isolated tmux pane with its own git worktree.

Options

`--no-attach`

Start the colony without automatically attaching to the tmux session.

colony start --no-attach

What Happens

When you run colony start:

Tmux Session - Creates a tmux session (default name: colony-<project>)
Git Worktrees - Sets up isolated git worktrees for each agent
Agent Panes - Creates a tmux pane for each agent
Claude Code - Launches Claude Code in each pane
Startup Prompts - Sends configured startup instructions to each agent
Auto-attach - Attaches to the session (unless --no-attach)

Examples

Start and Attach

colony start

This starts all agents and automatically attaches you to the tmux session.

Start Without Attaching

colony start --no-attach

Agents start in the background. Attach later with:

colony attach

Start and Monitor

colony start --no-attach && colony tui

Start agents in background and launch the TUI monitor.

Tmux Layout

The default layout creates a pane for each agent:

┌─────────────┬─────────────┐
│   agent-1   │   agent-2   │
│             │             │
│             │             │
│             │             │
└─────────────┴─────────────┘

With more agents, tmux automatically tiles the panes.

Tmux Key Bindings

Once attached to the session:

Ctrl+b d - Detach from session (agents keep running)
Ctrl+b arrow - Navigate between panes
Ctrl+b z - Zoom/unzoom current pane
Ctrl+b [ - Enter scroll mode (q to exit)

Agent Isolation

Each agent operates in isolation:

Separate worktree - No git conflicts between agents
Independent directory - Each agent has its own working directory
Isolated processes - Each Claude Code instance runs independently
Unique configuration - Each agent can have different MCP servers

Checking Status

While agents are running:

# Quick status check
colony status

# Real-time TUI
colony tui

# View logs
colony logs agent-id

Stopping Agents

# Stop all agents
colony stop

# Stop specific agent
colony stop agent-id

Troubleshooting

"tmux not found"

Install tmux:

# macOS
brew install tmux

# Ubuntu/Debian
sudo apt-get install tmux

"tmux session already exists"

A colony session is already running. Options:

# Attach to existing session
colony attach

# Stop existing session first
colony stop
colony start

Agent fails to start

Check logs:

colony logs agent-id

Common issues:

Git worktree conflicts
Invalid configuration
MCP server errors

"Cannot create worktree"

Ensure you're in a git repository with at least one commit:

git add .
git commit -m "Initial commit"
colony start

Configuration

Control startup behavior in colony.yml:

name: my-colony

agents:
  - id: agent-1
    role: Backend Engineer
    focus: API development
    model: claude-sonnet-4-20250514
    worktree: agent/backend
    startup_prompt: |
      Custom startup instructions here

# Optional: customize tmux behavior
# (future enhancement)

Behind the Scenes

Colony performs these steps:

Load colony.yml configuration
Validate agent configurations
Create/verify git worktrees
Start tmux session
Create panes for each agent
Set up MCP server configurations
Launch Claude Code processes
Send startup prompts
Attach to session (if not --no-attach)

colony status

Documentation for colony status command.

(Coming soon)

See CLI Overview for all commands.

colony agent

Documentation for colony agent command.

(Coming soon)

See CLI Overview for all commands.

colony template

Manage agent templates for reusable configurations.

Synopsis

colony template <SUBCOMMAND>

Subcommands

list

List all available templates (both installed and built-in).

colony template list

Example Output:

Available Templates

  code-reviewer 1.0.0 (custom)
    Code quality and best practices review agent
    Role: Code Reviewer

  security-auditor 1.0.0 (custom)
    OWASP Top 10 focused security auditing agent
    Role: Security Auditor

show

Show detailed information about a specific template.

colony template show <NAME>

Arguments:

NAME - Template name

Example:

colony template show code-reviewer

Output:

Template: code-reviewer

Version: 1.0.0
Source: custom

Description:
  Code quality and best practices review agent

Author: Colony Community
License: MIT

Agent Configuration:
  Role: Code Reviewer
  Focus: Review code for quality, best practices, and potential issues
  Model: claude-sonnet-4-20250514

Compatible Repository Types:
  - source
  - application

install

Install a built-in template to .colony/templates/.

colony template install <NAME>

Arguments:

NAME - Built-in template name

Example:

colony template install code-reviewer

Output:

✓ Installed template: code-reviewer

Template location: .colony/templates/code-reviewer

You can now use this template when creating agents:
  colony agent create --template code-reviewer --id my-agent

Error if already installed:

Error: Template 'code-reviewer' is already installed

builtin

List all built-in templates available for installation.

colony template builtin

Example Output:

Built-in Templates

  code-reviewer
    Code quality and best practices review agent
    Role: Code Reviewer

  security-auditor
    OWASP Top 10 focused security auditing agent
    Role: Security Auditor

  test-engineer
    Automated testing and QA specialist
    Role: Test Engineer

  documentation-writer
    Technical documentation specialist
    Role: Documentation Writer

  data-analyst
    Data analysis and insights agent
    Role: Data Analyst

Use 'colony template install <name>' to install a template
Use 'colony template show <name>' to see details (after installing)

Template Storage

Templates are stored in:

.colony/templates/
  ├── code-reviewer/
  │   └── template.yaml
  ├── security-auditor/
  │   └── template.yaml
  └── custom-template/
      └── template.yaml

Using Templates

In colony.yml Configuration

agents:
  - id: reviewer
    worktree_branch: review/auto
    template: code-reviewer

  - id: security
    worktree_branch: security/scan
    template: security-auditor

With Custom Startup Prompt

agents:
  - id: reviewer
    worktree_branch: review/auto
    template: code-reviewer
    startup_prompt: |
      Override the template's default prompt.
      Focus on TypeScript and React best practices.

Creating Custom Templates

Create a custom template directory:

mkdir -p .colony/templates/my-template

Create template.yaml:

name: my-template
version: 1.0.0
description: My custom agent template
author: Your Name
license: MIT

agent:
  role: Custom Role
  focus: Specific responsibilities
  model: claude-sonnet-4-20250514

  startup_prompt: |
    Your custom instructions here.

requirements:
  repo_types:
    - source

Use your custom template:

agents:
  - id: custom-agent
    worktree_branch: feature/work
    template: my-template

Built-in Template Details

code-reviewer

Role: Code Reviewer
Focus: Quality, best practices, potential issues
Model: claude-sonnet-4-20250514
Use Case: Automated code review

security-auditor

Role: Security Auditor
Focus: OWASP Top 10, vulnerabilities, security best practices
Model: claude-sonnet-4-20250514
Use Case: Security scanning and auditing

test-engineer

Role: Test Engineer
Focus: Unit tests, integration tests, test coverage
Model: claude-sonnet-4-20250514
Use Case: Writing comprehensive test suites

documentation-writer

Role: Documentation Writer
Focus: API docs, user guides, README files
Model: claude-sonnet-4-20250514
Use Case: Maintaining technical documentation

data-analyst

Role: Data Analyst
Focus: Data processing, analysis, insights
Model: claude-sonnet-4-20250514
Use Case: Data analysis and visualization

Examples

Install and Use Template

# Install the code-reviewer template
colony template install code-reviewer

# Verify installation
colony template show code-reviewer

# Use in configuration
cat > colony.yml <<EOF
agents:
  - id: reviewer
    worktree_branch: review/auto
    template: code-reviewer
EOF

# Start colony
colony start

List All Templates

# List built-in (not installed)
colony template builtin

# Install a few
colony template install code-reviewer
colony template install security-auditor

# List all (installed + custom)
colony template list

Create Custom Template

# Create template directory
mkdir -p .colony/templates/frontend-specialist

# Create template.yaml
cat > .colony/templates/frontend-specialist/template.yaml <<EOF
name: frontend-specialist
version: 1.0.0
description: React and TypeScript frontend specialist

agent:
  role: Frontend Developer
  focus: React components, TypeScript, and styling
  model: claude-sonnet-4-20250514

  startup_prompt: |
    You are a frontend specialist focusing on:
    - React component development
    - TypeScript type safety
    - CSS-in-JS styling
    - Component testing with Jest/React Testing Library
    - Accessibility (WCAG 2.1 AA)
EOF

# List templates (will include custom)
colony template list

# Use in config
cat >> colony.yml <<EOF
  - id: frontend
    worktree_branch: feature/ui
    template: frontend-specialist
EOF

Exit Codes

0 - Success
1 - Template not found
1 - Template already installed
1 - Invalid template format

colony plugin

Documentation for colony plugin command.

(Coming soon)

See CLI Overview for all commands.

colony workflow

Documentation for colony workflow command.

(Coming soon)

See CLI Overview for all commands.

colony state

Documentation for colony state command.

(Coming soon)

See CLI Overview for all commands.

colony logs

Documentation for colony logs command.

(Coming soon)

See CLI Overview for all commands.

colony tui

Interactive Terminal User Interface for monitoring and controlling your Colony.

Overview

The TUI provides a real-time dashboard for managing your multi-agent system, with tabs for agents, tasks, messages, shared state, and help.

colony tui

Features

Real-time monitoring - Auto-refreshes every 2 seconds
Multi-tab interface - Organized views for different aspects
Interactive dialogs - Broadcast messages, create tasks, send messages
Keyboard-driven - Fast navigation with vim-style keys
Status notifications - Instant feedback on actions

Tabs

1: Agents Tab

View all running agents and their current status.

Information Displayed:

Agent ID and role
Status (Running, Idle, Failed, Completed)
Process ID (PID)
Current task being worked on

Status Indicators:

🟢 Running - Agent is actively working
⚪ Idle - Agent is waiting for tasks
🔵 Completed - Agent finished its work
🔴 Failed - Agent encountered an error

2: Tasks Tab

Monitor the task queue and completion progress.

Sections:

Summary - Visual progress bar and task counts
Task List - Tasks grouped by status

Task Statuses:

⏳ Pending - Available to be claimed
👤 Claimed - Assigned to an agent
🔄 In Progress - Actively being worked on
🚫 Blocked - Waiting on dependencies
✅ Completed - Finished

Metrics:

Total tasks
Completion percentage
Progress bar visualization

3: Messages Tab

View inter-agent and operator messages.

Message Format:

[TIME] SENDER → RECIPIENT: Message content
[TIME] SENDER [BROADCAST]: Message to all

Features:

Chronological message history
Broadcast message indicators
Sender and recipient highlighting
Recent messages display (most recent first)

4: State Tab

Monitor git-backed shared state (tasks and workflows).

State Tasks:

Task ID and title
Assigned agent
Status indicators
Blocker counts

State Workflows:

Workflow ID and name
Current step
Running/completed status

Note: This tab only appears when shared state is enabled in colony.yml:

shared_state:
  backend: git-backed
  location: in-repo

5: Help Tab

Complete keyboard shortcuts reference and feature documentation.

Keyboard Shortcuts

Key	Action
`1`, `2`, `3`, `4`, `5`	Switch to specific tab
`Tab`	Next tab
`Shift+Tab`	Previous tab
`↑` / `k`	Scroll up
`↓` / `j`	Scroll down
`PgUp`	Page up
`PgDn`	Page down

Actions

Key	Action
`r`	Refresh data manually
`b`	Broadcast message to all agents
`t`	Create a new task
`m`	Send message to specific agent
`?`	Show help

General

Key	Action
`q`	Quit TUI
`Ctrl+C`	Quit TUI
`Esc`	Cancel current dialog

Interactive Dialogs

Broadcast Message (`b`)

Send a message to all agents simultaneously.

Steps:

Press b to open dialog
Type your message
Press Enter to send
Press Esc to cancel

Example Use Cases:

Announce a priority change
Share important findings
Coordinate major changes
Emergency notifications

Create Task (`t`)

Add a new task to the queue with a multi-step form.

Steps:

Press t to open dialog
Enter each field:
- Task ID: Unique identifier (e.g., feature-123)
- Title: Short description (e.g., "Implement login API")
- Description: Detailed explanation
- Assigned To: Agent ID (optional, leave blank for "auto")
- Priority: low, medium, high, or critical (default: medium)
Press Enter after each field
Task is created after final field
Press Esc to cancel at any step

Features:

Progress indicator shows current step
Previous inputs displayed during later steps
Input validation
Auto-refresh after creation

Send Message (`m`)

Send a directed message to a specific agent.

Steps:

Press m to open dialog
Enter agent ID
Press Enter
Type your message
Press Enter to send
Press Esc to cancel

Example Use Cases:

Request code review
Ask for help with a blocker
Share relevant findings
Coordinate on shared files

Metrics Panel

The metrics panel (always visible at top) shows:

Agent Metrics:

Total agents
Running count
Idle count
Failed count (if any)

Task Metrics:

Total tasks
Pending tasks
In-progress tasks
Completed tasks
Total message count

State Metrics (when shared state enabled):

State tasks (ready, in-progress, blocked, completed)
State workflows (running, completed)

Status Bar

The status bar (always visible at bottom) shows:

Normal Mode:

Status: 3 agents running | Shortcuts: q=Quit r=Refresh b=Broadcast t=Task m=Message ?=Help

After Action:

Status: Broadcast sent: Your message here
Error: Failed to create task: Task ID already exists

Tips & Best Practices

Monitoring Workflows

Watch Agents tab to see which agents are active
Check Tasks tab to monitor queue depth and completion
Review Messages to understand agent communication
Monitor State for cross-session coordination

Creating Effective Tasks

Use descriptive Task IDs: feature-login better than task-1
Write clear titles: Short but informative
Provide context in description: Help agents understand requirements
Set appropriate priority: Reserve critical for urgent issues
Assign strategically: Use auto for flexible claiming

Broadcast Best Practices

Be concise: Agents receive these as notifications
Be specific: "PR #123 ready for review" vs "Something is ready"
Use for coordination: Major changes, blockers, important updates
Don't spam: Use targeted messages (m) for agent-specific communication

Performance

The TUI is designed for efficiency:

Auto-refreshes every 2 seconds
Manual refresh with r for immediate updates
Lightweight polling of filesystem
No impact on agent performance

Troubleshooting

TUI won't start:

# Check if colony is initialized
ls colony.yml

# Verify no permission issues
colony health

No data showing:

# Ensure colony is running
colony status

# Check for data in .colony/
ls -la .colony/

Dialogs not working:

Ensure you're in normal mode (not another dialog)
Press Esc to cancel any active dialog
Check status bar for error messages

Example Session

# Start TUI
colony tui

# In TUI:
# 1. Press '1' to view agents
# 2. Press 'b' to broadcast: "Starting feature work"
# 3. Press 't' to create task:
#    - ID: feature-api-auth
#    - Title: Implement API authentication
#    - Description: Add JWT-based auth to REST API
#    - Assigned: main-dev
#    - Priority: high
# 4. Press '2' to watch task progress
# 5. Press 'm' to message main-dev: "Check security guidelines"
# 6. Press 'r' to refresh manually
# 7. Press '3' to view message delivery
# 8. Press 'q' to quit

Integration with Other Commands

The TUI complements command-line workflows:

# CLI: Check quick status
colony status

# TUI: Deep monitoring session
colony tui

# CLI: Add task via script
colony tasks create task-123 "Build API" "REST endpoints"

# TUI: View task in queue (Tab 2)
# TUI: Send message about task (Press 'm')

# CLI: View logs for debugging
colony logs --level error

# TUI: Monitor agent health (Tab 1)

Advanced Usage

Custom Refresh Interval

The TUI refreshes every 2 seconds by default. For near real-time monitoring on fast systems:

# Monitor in TUI
colony tui  # Default 2s refresh

# For critical monitoring, use status in a loop
watch -n 1 colony status

Multi-Session Workflow

Terminal 1: Run colony tui for monitoring
Terminal 2: Use CLI commands for operations
Terminal 3: Run colony attach to watch agents

Filtering and Focus

While the TUI shows all data by default, you can:

Use colony status for quick checks
Use colony tasks list --status pending for filtered views
Use colony logs agent-id for specific agent details

Then return to TUI for overall coordination.

colony metrics

View and manage metrics for monitoring colony performance.

Synopsis

colony metrics <SUBCOMMAND>

Description

The metrics system collects and displays performance data about your colony, including agent activity, task completion, workflow execution, and system resources.

Subcommands

list

List all registered metrics with current values.

colony metrics list

Output:

Available Metrics

Agent Metrics
  agent.tasks.completed [counter] - 15.00 tasks
    Number of tasks completed by agents
  agent.tasks.failed [counter] - 2.00 tasks
    Number of tasks failed by agents

Task Metrics
  task.queue.depth [gauge] - 5.00 tasks
    Current number of tasks in the queue

Workflow Metrics
  workflow.runs.completed [counter] - 8.00 runs
    Number of workflow runs completed

System Metrics
  system.memory.used [gauge] - 1024.50 MB
    Memory used by the colony system

show

Show detailed statistics for a specific metric.

colony metrics show <NAME> [--hours HOURS]

Arguments:

NAME - Metric name (e.g., agent.tasks.completed)

Options:

--hours - Time period in hours (default: 1)

Example:

colony metrics show agent.tasks.completed --hours 24

Output:

Metric: agent.tasks.completed
Type: Counter
Description: Number of tasks completed by agents
Unit: tasks

Statistics (last 24 hours)

  Current: 45.00 tasks
  Average: 38.50 tasks
  Min: 15.00 tasks
  Max: 45.00 tasks
  Total: 462.00 tasks
  Data Points: 12

Recent Values

  2024-01-15 14:30:00 - 45.00 tasks
  2024-01-15 13:30:00 - 42.00 tasks
  2024-01-15 12:30:00 - 38.00 tasks

export

Export all metrics to JSON format.

colony metrics export [-o FILE]

Options:

-o, --output - Output file path (default: stdout)

Example:

# Export to file
colony metrics export -o metrics.json

# Export to stdout
colony metrics export | jq '.[] | select(.name | contains("agent"))'

JSON Format:

[
  {
    "name": "agent.tasks.completed",
    "metric_type": "Counter",
    "description": "Number of tasks completed by agents",
    "unit": "tasks",
    "points": [
      {
        "timestamp": "2024-01-15T14:30:00Z",
        "value": 45.0,
        "labels": {}
      }
    ]
  }
]

clear

Clear old metrics data or all metrics.

colony metrics clear [--all]

Options:

--all - Clear all metrics, not just old data

Examples:

# Clear data older than retention period (24 hours)
colony metrics clear

# Clear all metrics
colony metrics clear --all

init

Initialize sample metrics for testing.

colony metrics init

Creates standard metrics:

agent.tasks.completed
agent.tasks.failed
task.queue.depth
task.execution_time
workflow.runs.completed
system.memory.used

record

Record a sample metric value (for testing).

colony metrics record <NAME> <VALUE>

Arguments:

NAME - Metric name
VALUE - Numeric value to record

Example:

colony metrics record agent.tasks.completed 10
colony metrics record system.cpu.percent 45.5

Metric Types

Counter

Monotonically increasing values:

Task completions
Workflow runs
Errors

Use case: Counting events that only go up.

Gauge

Point-in-time values that can go up or down:

Queue depth
Memory usage
Active agents

Use case: Current state snapshots.

Histogram

Distribution of values over time:

Execution times
Response latencies
Resource usage patterns

Use case: Understanding value distributions.

Standard Metrics

Agent Metrics

agent.tasks.completed - Tasks completed by agents
agent.tasks.failed - Tasks that failed
agent.active_time - Time agent spent active
agent.idle_time - Time agent spent idle

Task Metrics

task.queue.depth - Current tasks in queue
task.wait_time - Time tasks wait before starting
task.execution_time - Time to complete tasks

Workflow Metrics

workflow.runs.started - Workflows started
workflow.runs.completed - Workflows completed
workflow.runs.failed - Workflows that failed
workflow.step.duration - Time per workflow step

System Metrics

system.memory.used - Memory usage
system.cpu.percent - CPU utilization

Integration

Export to Monitoring Tools

# Prometheus format (future)
colony metrics export --format prometheus

# Grafana (use JSON export)
colony metrics export | curl -X POST grafana-api/metrics

# Custom processing
colony metrics export | jq '...'

Automated Monitoring

# Continuous monitoring
while true; do
  colony metrics export -o metrics-$(date +%s).json
  sleep 60
done

Alerting

# Check thresholds
QUEUE_DEPTH=$(colony metrics show task.queue.depth --json | jq '.current')
if [ $QUEUE_DEPTH -gt 100 ]; then
  echo "Alert: Queue depth is $QUEUE_DEPTH"
fi

Examples

Monitor Task Completion Rate

# Show last 24 hours
colony metrics show agent.tasks.completed --hours 24

# Compare with failures
colony metrics show agent.tasks.failed --hours 24

Track System Resources

# Memory usage
colony metrics show system.memory.used

# Record custom metric
colony metrics record system.cpu.percent $(top -bn1 | grep "Cpu(s)" | awk '{print $2}')

Generate Daily Reports

#!/bin/bash
# daily-metrics.sh
DATE=$(date +%Y-%m-%d)
colony metrics export -o reports/metrics-$DATE.json

# Email report
mail -s "Colony Metrics $DATE" admin@example.com < reports/metrics-$DATE.json

Retention

Default: 24 hours
Configurable: Future feature
Pruning: Automatic via colony metrics clear

Architecture

Colony architecture overview.

Components

CLI: Command-line interface (Rust)
Tmux Integration: Process isolation
State Backend: Git-backed storage
Agent Runtime: Claude Code instances

Design Principles

Isolation through worktrees
Git-backed reliability
YAML configuration
Plugin extensibility

Git-Backed State

Implementation details of the git-backed state system.

Design

Based on Beads principles:

Append-only JSONL logs
SQLite query layer
Git version control
Hash-based IDs

Storage

.colony/state/
├── tasks.jsonl
├── workflows.jsonl
├── memory.jsonl
└── query.db

See BEADS Integration for full details.

Custom Templates

Advanced guide for creating custom agent templates.

Template Schema

name: string          # Template name
version: string       # Semantic version
author: string        # Author name
description: string   # Description
license: string       # License (MIT, Apache-2.0, etc.)

agent:
  role: string
  focus: string
  model: string
  startup_prompt: string
  mcp_servers: object

requirements:
  repo_types: array

See Templates for usage examples.

Plugin Development

Guide for developing Colony plugins.

Plugin Manifest

name: my-plugin
version: 1.0.0
plugin_type: backend  # backend, ui, or tool
entrypoint: src/main.rs

hooks:
  - on_agent_start
  - on_task_complete

Creating a Plugin

Create plugin directory
Write plugin.yaml manifest
Implement plugin logic
Enable in colony.yml

(Detailed guide coming soon)

Best Practices

Guidelines for effective multi-agent orchestration with Colony.

Agent Configuration

Sizing Your Colony

Start Small, Scale Up

# Good: Start with 2-3 agents
agents:
  - id: main-dev
  - id: reviewer
  - id: tester

# Avoid: Too many agents initially
# agents: [10+ agents on day one]

Recommended Colony Sizes:

Small Project (1-2 developers): 2-3 agents
Medium Project (3-5 developers): 4-6 agents
Large Project (6+ developers): 6-10 agents
Maximum Practical: 12-15 agents (beyond this, coordination overhead increases)

Agent Specialization

Do: Clear, Focused Roles

agents:
  - id: backend-api
    role: Backend Engineer
    focus: REST API development and database design

  - id: frontend-ui
    role: Frontend Engineer
    focus: React components and user interface

  - id: security-auditor
    role: Security Specialist
    focus: Security review and vulnerability scanning

Don't: Vague, Overlapping Roles

# Avoid this:
agents:
  - id: dev-1
    role: Developer
    focus: Do stuff

Startup Prompts

Effective Startup Prompts Include:

Primary Responsibility
Specific Focus Areas
Communication Guidelines
Quality Standards

startup_prompt: |
  You are the Backend API Engineer for this project.

  PRIMARY RESPONSIBILITIES:
  - Design and implement REST API endpoints
  - Manage database schema and migrations
  - Write API documentation

  FOCUS AREAS:
  - RESTful design principles
  - Database optimization
  - API security (auth, validation, rate limiting)

  COMMUNICATION:
  - Coordinate with frontend-ui for API contracts
  - Notify security-auditor before merging
  - Report blockers immediately

  QUALITY STANDARDS:
  - All endpoints must have tests
  - Follow project naming conventions
  - Document all public APIs

Task Management

Task Design

Good Task Characteristics:

Atomic: One clear objective
Testable: Success criteria defined
Sized Right: 2-4 hours of work
Well-Scoped: Clear boundaries

# Good Task
colony tasks create api-users-endpoint \
  "Implement /api/users endpoint" \
  "Create GET /api/users with pagination, filtering by role, and proper auth. Include unit tests and update API docs." \
  --priority high

# Too Vague
colony tasks create backend-work \
  "Do backend stuff" \
  "Work on backend"

Task Dependencies

Use Blockers for Sequencing:

# Step 1: Database schema
colony state task add "Create users table schema" \
  --description "Design and implement users table with proper indexes"

# Step 2: Depends on schema
colony state task add "Implement user CRUD" \
  --description "Create, Read, Update, Delete operations for users" \
  --blockers "Create users table schema"

# Step 3: Depends on CRUD
colony state task add "Add user authentication" \
  --description "JWT-based authentication for user endpoints" \
  --blockers "Implement user CRUD"

Task Priority Guidelines

Priority	When to Use	Examples
`critical`	Blocking production, security issues	"Fix auth bypass vulnerability", "Restore down service"
`high`	Core features, important bugs	"Implement login flow", "Fix data loss bug"
`medium`	Standard features, improvements	"Add user profile page", "Optimize query performance"
`low`	Nice-to-haves, refactoring	"Update dependencies", "Improve logging"

Communication

Message Types

Broadcasts (colony broadcast or b in TUI):

# Good uses:
colony broadcast "🚨 Critical: Security patch required in auth module"
colony broadcast "✅ Sprint complete - all tests passing"
colony broadcast "📢 New API contract available in docs/"

# Avoid:
colony broadcast "Working on stuff"  # Too vague
colony broadcast "Hey reviewer-1..."  # Use direct message instead

Direct Messages:

# Coordinate specific work
colony messages send backend-dev "API contract ready for /users endpoint"

# Request help
colony messages send security-audit "Please review auth changes in PR #123"

# Report blockers
colony messages send team-lead "Blocked on database access permissions"

Communication Patterns

1. Pull Request Workflow

# Developer finishes work
colony broadcast "PR #123 ready for review: User authentication"

# Reviewer claims
colony messages send developer-1 "Reviewing PR #123, will have feedback in 30min"

# Review complete
colony messages send developer-1 "PR #123 approved with minor suggestions"

2. Blocker Resolution

# Agent hits blocker
colony broadcast "🚫 Blocked: Need database credentials for integration tests"

# Coordinator responds
colony messages send blocked-agent "DB creds in 1Password vault 'Dev Credentials'"

# Agent unblocks
colony tasks unblock task-123
colony broadcast "✅ Unblocked, resuming work on integration tests"

Shared State

When to Enable Shared State

Enable for:

Multi-session work (resume after breaks)
Cross-session coordination
Long-running projects
Distributed teams

Skip for:

Quick experiments
Single-session work
Prototype/spike projects

State Hygiene

Regular Sync:

# Before starting work
colony state pull

# After significant progress
colony state push

# End of session
colony state sync

Clean Completed Tasks:

# Weekly cleanup
colony state task list --status completed |
  grep "2024-01" | # Old tasks
  xargs -I {} colony state task delete {}

Commit Message Guidelines:

# Good commit messages
git commit -m "state: Add task for user profile feature"
git commit -m "state: Mark authentication tasks as complete"
git commit -m "state: Update blockers for API integration"

# Enable in .git/hooks/prepare-commit-msg

Workflows

Workflow Design Principles

1. Single Responsibility Each workflow should handle one logical process.

# Good: Focused workflow
workflow:
  name: code-review-workflow
  steps:
    - name: lint
    - name: test
    - name: security-scan
    - name: manual-review

# Avoid: Kitchen sink workflow
workflow:
  name: do-everything
  steps: [50+ steps]

2. Idempotent Steps Steps should be safely re-runnable.

# Good: Can retry
steps:
  - name: run-tests
    agent: test-runner
    retry:
      max_attempts: 3
      backoff: exponential

# Avoid: Non-idempotent
steps:
  - name: increment-counter  # Don't do this

3. Clear Dependencies

steps:
  - name: build
    agent: builder

  - name: test
    depends_on: [build]  # Explicit dependency
    agent: tester

  - name: deploy
    depends_on: [test]  # Sequential
    agent: deployer

Error Handling

Graceful Degradation:

workflow:
  steps:
    - name: primary-build
      agent: builder
      on_failure: try-backup-build

    - name: try-backup-build
      agent: backup-builder
      on_failure: notify-team

    - name: notify-team
      agent: coordinator
      instructions: "Send alert about build failure"

Monitoring

What to Monitor

Critical Metrics:

Agent Health: Running vs. failed agents
Task Velocity: Tasks completed per hour
Message Flow: Communication patterns
Error Rate: Failed tasks and retries

Using the TUI Effectively

Daily Workflow:

# Morning: Check overnight progress
colony tui
# 1. Review Agents tab for any failures
# 2. Check Tasks tab for completion rate
# 3. Scan Messages for any issues

# During Work: Monitor in real-time
# Keep TUI open in dedicated terminal
# Watch for agent failures
# Monitor task queue depth

# Evening: Final check
# 1. Verify all tasks claimed/completed
# 2. Check no blocked agents
# 3. Sync state before shutdown

Log Review

Regular Log Patterns:

# Daily error scan
colony logs --level error --last 24h

# Agent-specific debugging
colony logs problematic-agent --pattern "error|warning"

# Performance monitoring
colony logs --pattern "slow|timeout|retry"

Performance Optimization

Resource Management

CPU and Memory:

Limit concurrent agents based on system resources
Use claude-sonnet for routine tasks (faster, cheaper)
Use claude-opus only for complex reasoning

agents:
  - id: code-reviewer
    model: claude-sonnet-4  # Sufficient for reviews

  - id: architect
    model: claude-opus-4  # Complex system design

Git Worktree Strategy

Shared vs. Isolated:

# Isolated: Parallel independent work
agents:
  - id: feature-a-dev
    worktree_branch: feature/payment-system

  - id: feature-b-dev
    worktree_branch: feature/notification-service

# Shared: Collaborative work
agents:
  - id: backend-dev
    worktree: shared-api-work
    worktree_branch: feature/api-refactor

  - id: api-tester
    worktree: shared-api-work  # Same worktree
    worktree_branch: feature/api-refactor

Security

Secrets Management

Do:

# Use environment variables
agents:
  - id: deploy-agent
    env:
      AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID  # From environment
      AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY

Don't:

# Never hardcode secrets
agents:
  - id: deploy-agent
    startup_prompt: "Use API key: sk-1234abcd..."  # NEVER DO THIS

Access Control

Principle of Least Privilege:

agents:
  - id: readonly-reviewer
    # Give minimal permissions
    # Can read code, can't push

  - id: deployer
    # Only this agent can deploy
    env:
      DEPLOY_KEY: $DEPLOY_KEY

Team Collaboration

Multi-User Colonies

Personal Colonies:

# alice/colony.yml
name: alice-dev-colony
agents:
  - id: alice-main-dev
  - id: alice-reviewer

# bob/colony.yml
name: bob-dev-colony
agents:
  - id: bob-main-dev
  - id: bob-tester

Shared State Coordination:

# Alice
colony state pull  # Get latest tasks
colony state task claim task-123 alice-main-dev
colony state push

# Bob
colony state pull  # Sees Alice claimed task-123
colony state task claim task-456 bob-main-dev
colony state push

Code Review Process

Automated Review Colony:

agents:
  - id: developer
    focus: Implement features

  - id: auto-reviewer
    template: code-reviewer
    startup_prompt: |
      Review all PRs for:
      - Code quality
      - Test coverage
      - Security issues
      - Performance concerns

      Comment inline and request changes if needed.

  - id: security-reviewer
    template: security-auditor
    startup_prompt: |
      Security-focused review:
      - Check for OWASP Top 10
      - Validate input sanitization
      - Review authentication/authorization

Troubleshooting

Common Issues

Agents Not Starting:

# Check colony health
colony health

# Verify configuration
cat colony.yml

# Check tmux session
tmux list-sessions
colony attach  # See what's happening

Tasks Not Being Claimed:

# Check task status
colony tasks list --status pending

# Verify dependencies
colony tasks show task-id

# Check agent assignment
colony tasks agent agent-id

State Sync Conflicts:

# Pull latest
colony state pull

# Resolve conflicts manually
# Edit .colony/state/ files

# Push resolved state
colony state push

Debug Mode

Verbose Logging:

# colony.yml
observability:
  logging:
    level: debug  # Detailed logs
    output: both  # File + stdout

Cost Optimization

Efficient API Usage

Model Selection:

# Use cheaper models where appropriate
agents:
  - id: linter
    model: claude-sonnet-4  # Simple tasks

  - id: architect
    model: claude-opus-4  # Complex decisions only

Task Batching:

# Instead of many small tasks
colony tasks create review-file-1 "Review file1.js"
colony tasks create review-file-2 "Review file2.js"
# ... (100 tasks)

# Batch similar work
colony tasks create review-batch-1 \
  "Review all files in src/components/" \
  "Check all .js files for code quality issues"

Resource Monitoring

# Track colony costs
colony metrics show api_calls --hours 24
colony metrics show token_usage --hours 24

# Budget alerts (in your monitoring)
if [ $(colony metrics show token_usage) -gt 1000000 ]; then
  echo "High token usage - review colony efficiency"
fi

Maintenance

Regular Cleanup

Weekly:

# Clean old tasks
colony tasks list --status completed |
  grep "$(date -d '7 days ago' +%Y-%m)"  |
  xargs -I {} colony tasks delete {}

# Clean old logs
colony logs --clean --older-than 7d

Monthly:

# Review agent performance
colony metrics export --output metrics.json
# Analyze which agents are most effective

# Update dependencies
cd ~/.colony/plugins && git pull
colony plugin update --all

# Review and update templates
colony template list
colony template update --all

Backup Strategy

Critical Data:

# Backup colony configuration
cp colony.yml colony.yml.backup

# Backup shared state
git clone .colony/state/.git state-backup

# Backup custom templates
tar -czf templates-backup.tar.gz .colony/templates/

Summary

Golden Rules:

Start small - 2-3 specialized agents
Clear roles - Each agent knows its job
Communicate - Use broadcasts and messages effectively
Monitor - Watch the TUI, review logs
Sync state - Pull before work, push after
Optimize costs - Right model for the task
Document - Keep colony.yml well-commented
Iterate - Adjust based on what works

Success Metrics:

Agents rarely fail or block
Tasks flow smoothly through stages
Communication is clear and purposeful
State stays synchronized
Team velocity increases

When in Doubt:

Check the TUI for current state
Review logs for errors
Consult this guide
Ask in community discussions

Common Issues

Solutions to frequently encountered problems.

Installation Issues

tmux not found

Problem: colony start fails with "tmux: command not found"

Solution:

# macOS
brew install tmux

# Ubuntu/Debian
sudo apt-get install tmux

# Fedora/RHEL
sudo dnf install tmux

Rust/Cargo not installed

Problem: Cannot build colony from source

Solution:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

Initialization Issues

Not a Git repository

Problem: "colony must be run inside a Git repository"

Solution:

git init
git add .
git commit -m "Initial commit"
colony init

colony.yml already exists

Problem: Wizard won't start because config exists

Solution:

# Backup existing config
mv colony.yml colony.yml.backup

# Run init
colony init

# Or use --force (if available)
rm colony.yml
colony init

Start Issues

Tmux session already exists

Problem: "tmux session 'colony-project' already exists"

Solution:

# Stop existing session
colony stop

# Or manually kill tmux session
tmux kill-session -t colony-project

# Then start again
colony start

Worktree creation fails

Problem: "Cannot create worktree"

Solution:

# Ensure you have at least one commit
git status
git add .
git commit -m "Initial commit"

# Clean up old worktrees if any
git worktree prune

# Try again
colony start

Claude Code not found

Problem: Claude Code doesn't launch in tmux panes

Solution:

# Ensure Claude Code CLI is installed
which claude

# Add to PATH if needed
export PATH="$PATH:/path/to/claude"

# Or configure in colony.yml (future feature)

Runtime Issues

Agent not responding

Problem: Agent pane shows no activity

Solution:

# Check logs
colony logs agent-id

# Check agent status
colony status

# Restart specific agent
colony stop agent-id
# Manually restart or use colony restart (if available)

Git conflicts in worktrees

Problem: Agents have merge conflicts

Solution:

# Each agent works on their own branch
# Conflicts only occur when merging to main

# In agent's worktree:
git status
git merge-base HEAD main
git merge main

# Resolve conflicts and continue

Permission denied errors

Problem: Cannot write to .colony directory

Solution:

# Check permissions
ls -la .colony

# Fix permissions
chmod -R u+w .colony

# Or remove and reinitialize
rm -rf .colony
colony init

State Issues

State sync fails

Problem: "Failed to sync state"

Solution:

# Check git status in .colony/state
cd .colony/state
git status

# Resolve any conflicts
git add .
git commit -m "Resolve state conflicts"

# Try sync again
cd ../..
colony state sync

Task not showing up

Problem: Created task doesn't appear in queue

Solution:

# Force refresh
colony state pull

# Check task list
colony state task list

# Verify task was created
ls -la .colony/state/tasks.jsonl

Metrics Issues

No metrics available

Problem: colony metrics list shows "No metrics available"

Solution:

# Metrics are only available after agents run
colony start

# Or initialize sample metrics for testing
colony metrics init

Metrics not persisting

Problem: Metrics disappear after restart

Solution: Metrics are currently in-memory only. Future versions will persist to disk.

Template Issues

Template not found

Problem: "Template 'xyz' not found"

Solution:

# List available templates
colony template builtin

# Install template first
colony template install code-reviewer

# Then use in config

Template won't install

Problem: "Template already installed"

Solution:

# Remove existing template
rm -rf .colony/templates/template-name

# Install again
colony template install template-name

TUI Issues

TUI not updating

Problem: TUI shows stale data

Solution: Press 'r' to refresh, or restart TUI:

# Exit TUI (press 'q')
# Restart
colony tui

TUI crashes

Problem: TUI exits unexpectedly

Solution:

# Check terminal compatibility
echo $TERM

# Try with different terminal
export TERM=xterm-256color
colony tui

# Check for corrupted data
colony health

Dialog not responding

Problem: Pressed b, t, or m but nothing happens

Solution:

Ensure you're not already in a dialog (press Esc first)
Check that colony is properly initialized (ls colony.yml)
Verify agents are configured (colony status)
Restart TUI: press q, then colony tui

Can't type in dialog

Problem: Keyboard input not working in dialogs

Solution:

Ensure dialog is actually open (yellow border should be visible)
Try pressing Esc to cancel and re-open
Check terminal emulator settings (some intercept keys)
Verify terminal size is adequate (minimum 80x24)

Dialog shows garbled text

Problem: Dialog content displays incorrectly

Solution:

# Set proper locale
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

# Restart TUI
colony tui

Status message not clearing

Problem: Status bar stuck showing old message

Solution:

Messages clear automatically after actions
Press r to refresh and clear status
Navigate to different tab and back

Tabs not switching

Problem: Number keys 1-5 don't switch tabs

Solution:

Ensure TUI has focus (click the terminal window)
Check if running inside another tmux session (can interfere with keys)
Use Tab key to cycle through tabs instead
Exit nested tmux if present: tmux detach

Workflow Issues

Workflow not starting

Problem: colony workflow run doesn't execute

Solution:

# Check workflow definition exists
ls .colony/workflows/

# Validate workflow syntax
colony workflow show workflow-name

# Check agents are available
colony status

# View workflow logs
colony logs --pattern "workflow"

Workflow stuck on step

Problem: Workflow halts at specific step

Solution:

# Check workflow status
colony workflow status <run-id>

# View agent logs for that step
colony logs <agent-name>

# Check for blocked tasks
colony state task list --status blocked

# Cancel and retry
colony workflow cancel <run-id>
colony workflow run workflow-name

Steps executing out of order

Problem: Dependencies not respected

Solution:

Verify depends_on is correctly specified in workflow YAML
Check for circular dependencies
Ensure step names match exactly
Review workflow validation: colony workflow show workflow-name

Workflow timeout

Problem: "Workflow exceeded timeout"

Solution:

# Increase timeout in workflow definition
steps:
  - name: long-running-step
    timeout: 30m  # Increase from default
    agent: worker

Communication Issues

Messages not delivered

Problem: Agent doesn't receive messages

Solution:

# Check message queue
colony messages all

# Verify agent is running
colony status

# Check message file permissions
ls -la .colony/messages/

# Manually inspect messages
cat .colony/messages/*.json

Broadcast not visible to all agents

Problem: Some agents miss broadcasts

Solution:

Broadcasts are stored as messages to "all"
Agents must actively check messages
Verify all agents are running: colony status
Check message delivery in TUI: Tab 3

Agent not responding to messages

Problem: Agent ignores directed messages

Solution:

Agents must actively check their message queue
Verify agent ID matches exactly (case-sensitive)
Check agent's startup prompt includes message checking
Review agent logs: colony logs <agent-id>

Performance Issues

Slow agent responses

Problem: Agents take long to respond

Solution:

# Use faster models
agents:
  - id: quick-agent
    model: claude-sonnet-4  # Faster than opus

Additional steps:

Reduce complexity of tasks
Check network connection
Monitor with colony metrics show
Review API rate limits

High memory usage

Problem: Colony uses too much memory

Solution:

# Check current usage
colony health

# Reduce number of concurrent agents
# Stop idle agents:
colony stop agent-id

# Use lighter models in colony.yml

Disk space filling up

Problem: .colony/ directory growing large

Solution:

# Check disk usage
du -sh .colony/*

# Clean old logs
find .colony/logs -type f -mtime +7 -delete

# Remove old metrics
colony metrics clear --older-than 7d

# Clean completed tasks
colony state task list --status completed |
  grep "$(date -d '30 days ago' +%Y-%m)" |
  xargs -I {} colony state task delete {}

# Prune git worktrees
git worktree prune

Debugging Techniques

Enable debug logging

# Set environment variable
export COLONY_LOG_LEVEL=debug

# Start colony
colony start

# Or for specific commands
COLONY_LOG_LEVEL=debug colony tui

Inspect colony state

# View all state files
find .colony/ -type f | head -20

# Check configuration
cat colony.yml

# View tasks
cat .colony/state/tasks.jsonl

# Check messages
ls -la .colony/messages/

# Inspect metrics
ls -la .colony/metrics/

Manual agent testing

# Attach to tmux and watch agent
colony attach

# Navigate to agent pane (Ctrl+b, arrow keys)
# Observe agent behavior in real-time

# Detach without stopping: Ctrl+b, d

Network debugging

# Test Claude API connectivity
curl -I https://api.anthropic.com

# Check for proxy issues
echo $HTTP_PROXY
echo $HTTPS_PROXY

# Test with verbose output
claude --version --verbose

File system debugging

# Check permissions
ls -la .colony/
ls -la .git/

# Verify disk space
df -h .

# Check inode usage (can cause issues)
df -i .

# Test file creation
touch .colony/test.txt
rm .colony/test.txt

Database issues (SQLite)

If using SQLite backend for state:

# Check database integrity
sqlite3 .colony/state/colony.db "PRAGMA integrity_check;"

# View tables
sqlite3 .colony/state/colony.db ".tables"

# Query tasks
sqlite3 .colony/state/colony.db "SELECT * FROM tasks LIMIT 5;"

# Repair corrupted database (backup first!)
cp .colony/state/colony.db .colony/state/colony.db.backup
sqlite3 .colony/state/colony.db ".dump" | sqlite3 .colony/state/colony_new.db
mv .colony/state/colony_new.db .colony/state/colony.db

Git state debugging

# Check git state
cd .colony/state
git status
git log --oneline -5

# Look for conflicts
git diff

# Check remote sync
git remote -v
git fetch origin
git log --oneline origin/main..HEAD

# Return to project root
cd ../..

Resource monitoring

# Monitor CPU and memory
top -p $(pgrep -f colony)

# Monitor file handles
lsof | grep colony

# Monitor network connections
netstat -an | grep ESTABLISHED | grep claude

# Watch disk I/O
iotop -p $(pgrep -f colony)

Getting Help

If none of these solutions work:

Check logs: colony logs --level error
Check GitHub issues: colony/issues
Enable debug logging: COLONY_LOG_LEVEL=debug colony start
Create a minimal reproduction case

Frequently Asked Questions

Parallel Development: Multiple agents work simultaneously
Specialization: Each agent focuses on specific tasks
No Conflicts: Git worktrees prevent merge conflicts
Coordination**: Shared state enables agent communication
Scalable: Add more agents as needed

What are the system requirements?

Rust 1.70+ (for building)
tmux 2.0+
Git
Claude Code CLI
Linux or macOS (Windows via WSL)

Configuration

How many agents should I create?

Start with 2-3 agents. Add more based on:

Complexity of your project
Available system resources
Number of parallel workstreams needed

Yes! Agents work on separate branches but can:

Read from any branch
Merge changes between branches
Share via main/master branch
Communicate through shared state

How do I customize agent behavior?

Use startup_prompt in colony.yml:

agents:
  - id: my-agent
    startup_prompt: |
      Custom instructions here
      Multiple lines supported

Usage

How do agents communicate?

Through shared state:

Tasks: Assign and track work
Messages: Direct communication
Memory: Shared context and knowledge
Workflows: Coordinated multi-step processes

Can I stop and resume a colony?

Yes:

# Stop
colony stop

# Resume later
colony start

State persists in .colony/ directory.

How do I monitor agents?

Three ways:

TUI: colony tui - Real-time dashboard
Status: colony status - Quick check
Logs: colony logs - Detailed output

State Management

Where is state stored?

In .colony/state/ directory:

Git repository (JSONL files)
SQLite database (for queries)
Automatically synced

How do I backup state?

# State is git-backed
cd .colony/state
git push origin main

# Or copy the directory
cp -r .colony/state/ backup/

Can multiple people use the same colony?

Yes! Use git to sync:

# Person A
colony state push

# Person B
colony state pull

Templates

What templates are available?

Five built-in templates:

code-reviewer
security-auditor
test-engineer
documentation-writer
data-analyst

Can I create custom templates?

Yes! Create in .colony/templates/:

name: my-template
version: 1.0.0
description: My custom template

agent:
  role: My Role
  focus: My focus area
  model: claude-sonnet-4-20250514

Define steps
Specify agent assignments
Set dependencies
Configure retry policies

How do I create a workflow?

Create YAML file in .colony/workflows/:

name: my-workflow
steps:
  - id: step1
    agent: agent-1
    action: do_something
  - id: step2
    agent: agent-2
    depends_on: [step1]

Troubleshooting

Agent is stuck or not responding

# Check logs
colony logs agent-id

# Restart agent
colony stop agent-id
colony start  # Restarts all agents

Git worktree conflicts

Each agent has its own branch. Conflicts only occur when merging. Resolve like normal git conflicts.

Performance issues

Use faster Claude models (Haiku)
Reduce number of agents
Check system resources
Monitor with metrics

Advanced

Can I use different Claude models per agent?

Yes:

agents:
  - id: simple-agent
    model: claude-3-5-haiku-20241022  # Fast
  - id: complex-agent
    model: claude-opus-4-20250514     # Powerful

Can I add custom MCP servers?

Yes:

agents:
  - id: my-agent
    mcp_servers:
      custom-tool:
        command: node
        args: [server.js]
        env:
          API_KEY: ${API_KEY}

How do I integrate with CI/CD?

# In CI pipeline
colony init --template test-engineer
colony start --no-attach
colony workflow run test-pipeline
colony logs --json > results.json

Can I run colony in Docker?

Theoretically yes, but requires:

tmux in container
Git repository mounted
Claude Code CLI available

(Docker support is experimental)

Contributing

How can I contribute?

Report bugs on GitHub
Submit pull requests
Write documentation
Share templates
Help others in issues

Where is the source code?

GitHub: yourusername/cc-colony

Getting More Help

Contributing

Guidelines for contributing to Colony.

Development Setup

git clone https://github.com/yourusername/cc-colony.git
cd cc-colony
cargo build

Running Tests

cargo test

Submitting Changes

Fork the repository
Create a feature branch
Make your changes
Run tests
Submit a pull request

Building from Source

Prerequisites

Rust 1.70+
tmux 2.0+
Git

Build

cargo build --release

Install

cargo install --path .

Testing

Unit Tests

cargo test

Integration Tests

cargo test --test '*'

See Testing Plan for comprehensive testing guide.

Colony Documentation