How to Automate GitHub Issue Triage with an AI Agent (Free Workspace Template)

The maintainer's triage problem

The Kubernetes project receives over 100 new issues per day. React sees 30-50. Even a mid-sized open-source project with a few thousand stars gets 5-10 issues daily. Each one needs to be read, classified, labeled, checked for duplicates, and routed to the right person.

Most maintainers do this manually, and it shows. Issues sit unlabeled for days. Duplicates pile up. Feature requests get mislabeled as bugs. Contributors who could help find "good first issues" never find them because nobody tagged them. The maintainer who does triage burns out fastest.

The numbers tell the story:

Project	Issues/day	Avg. time to first label	Maintainer hours/week on triage
kubernetes/kubernetes	100+	2-4 hours	15-20 hours
facebook/react	30-50	4-8 hours	8-12 hours
vercel/next.js	20-30	6-12 hours	6-10 hours
Mid-size OSS (1k-10k stars)	5-15	12-48 hours	3-5 hours

An AI triage agent does not replace the maintainer's judgment. It handles the obvious classifications instantly and surfaces the ambiguous ones for human review. The result: issues get labeled in minutes instead of hours, duplicates get caught before they accumulate, and maintainers reclaim hours every week.

This tutorial walks through building a GitHub issue triage agent as a free, reusable workspace template.

What the triage agent does

Issue Opened on GitHub
        |
        v
[1. Read and Parse]
  Extract title, body, labels requested by author,
  mentioned files or error messages
        |
        v
[2. Duplicate Detection]
  Search open + recently closed issues for similar titles
  and descriptions
        |
        +-- Duplicate found --> Apply "possible-duplicate" label
        |                       Comment with link to original
        |
        +-- No duplicate -----> Continue
        |
        v
[3. Classification]
  Determine issue type (bug, feature, docs, question)
  Assess priority (critical, high, medium, low)
  Identify area/component from file mentions or keywords
        |
        v
[4. Label Application]
  Apply type label + priority label + area label
        |
        v
[5. Auto-Response]
  Post a triage comment:
  - Classification summary
  - If bug: ask for missing reproduction steps
  - If feature: acknowledge and explain the review process
  - If question: answer or redirect to docs/discussions
        |
        v
[6. Cross-Service Notification]
  If critical: alert team in Slack
  If security: private notification to security team

Template directory structure

github-issue-triage-agent/
├── SOUL.md                        # Agent personality + triage rules
├── AGENTS.md                      # Capabilities and boundaries
├── skills/
│   ├── classify-issue.md          # Issue type and priority classification
│   ├── detect-duplicates.md       # Duplicate issue detection
│   ├── auto-respond.md            # Templated responses by issue type
│   ├── label-and-assign.md        # Label application and routing
│   └── critical-alert.md          # Cross-service notifications
├── knowledge/
│   ├── label-taxonomy.md          # Complete label hierarchy
│   ├── component-map.md           # File paths → component/area mapping
│   ├── response-templates.md      # Canned response templates
│   └── triage-examples.md         # Example classifications for calibration
├── .openclaw/
│   └── mcp.json                   # GitHub MCP server config
├── .env.example                   # Required tokens
└── README.md                      # Setup instructions

Step 1: Design the label taxonomy

A good label taxonomy is the foundation of effective triage. It needs to be mutually exclusive within categories and immediately understandable to contributors.

# Label Taxonomy (knowledge/label-taxonomy.md)

## Issue Type (exactly one per issue)
| Label | Color | Description |
|-------|-------|-------------|
| `bug` | #d73a4a | Something is broken |
| `feature` | #a2eeef | Request for new functionality |
| `enhancement` | #7057ff | Improvement to existing functionality |
| `docs` | #0075ca | Documentation issue |
| `question` | #d876e3 | User needs help (not a bug) |
| `security` | #e11d48 | Potential security vulnerability |

## Priority (exactly one per issue)
| Label | Color | Description |
|-------|-------|-------------|
| `priority:critical` | #b60205 | Data loss, security vuln, total breakage |
| `priority:high` | #d93f0b | Major feature broken, no workaround |
| `priority:medium` | #fbca04 | Feature issue, workaround exists |
| `priority:low` | #0e8a16 | Cosmetic, edge case, nice-to-have |

## Status
| Label | Color | Description |
|-------|-------|-------------|
| `needs-info` | #c5def5 | Waiting for more detail from author |
| `needs-human-review` | #f9d0c4 | Agent was not confident, needs maintainer |
| `possible-duplicate` | #cfd3d7 | May duplicate an existing issue |
| `good-first-issue` | #7057ff | Suitable for new contributors |
| `help-wanted` | #008672 | Maintainers welcome outside contributions |

## Area/Component (project-specific)
| Label | Color | Matching patterns |
|-------|-------|-------------------|
| `area:api` | #1d76db | src/api/, routes/, endpoints |
| `area:ui` | #5319e7 | src/components/, src/pages/, CSS |
| `area:auth` | #b4a8d1 | src/auth/, login, session, token |
| `area:database` | #006b75 | prisma/, migration, query, schema |
| `area:ci` | #e4e669 | .github/workflows/, CI, build |

Step 2: Configure the GitHub MCP server

{
  "mcpServers": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_TOKEN}"
      }
    }
  }
}

For cross-service notifications (optional):

{
  "mcpServers": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_TOKEN}"
      }
    },
    "slack": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-slack"],
      "env": {
        "SLACK_BOT_TOKEN": "${SLACK_BOT_TOKEN}"
      }
    }
  }
}

Environment variables:

# .env.example
GITHUB_TOKEN=ghp_...                # Personal access token with repo scope
GITHUB_OWNER=your-org               # Repository owner
GITHUB_REPO=your-repo               # Repository name

# Optional: for cross-service alerts
SLACK_BOT_TOKEN=xoxb-...            # Slack bot token
SLACK_CHANNEL_ALERTS=#triage-alerts # Channel for critical issue alerts

Step 3: Write the SOUL.md

# GitHub Issue Triage Agent

You are an issue triage agent for {{GITHUB_OWNER}}/{{GITHUB_REPO}}.
You classify, label, and respond to new issues to help maintainers
manage the project efficiently.

## Your role
You are the first responder. Your job is to make every issue
actionable within minutes of being opened. You do not solve issues --
you organize them so the right people can solve them faster.

## Classification guidelines

### Bug reports
A bug report describes behavior that differs from documented or
expected behavior. Look for:
- "does not work", "broken", "error", "crash", "regression"
- Stack traces or error messages
- Comparison of expected vs. actual behavior

If the report lacks reproduction steps, classify it as a bug
but add the `needs-info` label and ask for steps.

### Feature requests
A feature request asks for something the project does not do yet.
Look for:
- "would be nice", "please add", "feature request", "proposal"
- Description of a workflow that is currently impossible or difficult

### Enhancement
An enhancement improves something that already exists. The distinction
from a feature request: the underlying capability exists but is
suboptimal. Look for:
- "improve", "better", "faster", "easier"
- References to existing functionality

### Questions
A question asks how to do something. It is not a bug -- the software
is working as designed, the user just needs help. Look for:
- "how do I", "is it possible", "what is the best way"
- Uncertainty about existing behavior

### Security
A security issue reports a vulnerability. These need special handling:
- Apply the `security` label
- Do NOT include vulnerability details in your triage comment
- Alert the security team immediately via Slack

## Priority assessment
- **Critical**: The issue affects core functionality for all users
  with no workaround, or it is a security vulnerability
- **High**: The issue affects a major feature and the workaround
  is painful or requires significant effort
- **Medium**: The issue is real but has a reasonable workaround
- **Low**: Cosmetic issues, edge cases, or nice-to-have improvements

## Hard rules
- NEVER close an issue. Only humans close issues.
- NEVER assign issues to specific people unless the component map
  explicitly names an owner.
- NEVER remove labels that were applied by a human.
- ALWAYS include your confidence score in the triage comment.
- ALWAYS apply the `needs-human-review` label if your confidence
  is below 70%.
- ALWAYS be respectful to issue authors. Many are first-time
  contributors.

Step 4: Build the classification skill

# Classify Issue Skill

## Input
- Issue number, title, body, author, and any existing labels

## Process

### 1. Read the issue carefully
Parse the title and body. Identify:
- Keywords that suggest type (bug, feature, question, etc.)
- Error messages or stack traces (strong signal for bug)
- Version numbers or environment info
- File paths or component mentions

### 2. Check the component map
Cross-reference any file paths or component names mentioned
in the issue against knowledge/component-map.md to determine
the area label.

### 3. Classify type
Using the guidelines in SOUL.md, assign exactly one type label.
If the issue could be multiple types (e.g., a bug report that
is actually a question), use the triage-examples.md for
calibration and pick the primary classification.

### 4. Assess priority
Using the priority guidelines in SOUL.md, assign exactly one
priority label. Consider:
- How many users are likely affected
- Is there a workaround
- Is data loss or security at risk

### 5. Identify special labels
- If the fix is clearly a small, isolated change: add `good-first-issue`
- If maintainers want community help: add `help-wanted`
- If more info is needed from the author: add `needs-info`

### 6. Calculate confidence
Rate your confidence from 0-100%:
- 90-100%: Clear classification, strong signals
- 70-89%: Probable classification, some ambiguity
- Below 70%: Uncertain, add `needs-human-review`

## Output
A structured classification:
- Type: {label}
- Priority: {label}
- Area: {label} (if determinable)
- Special labels: {list}
- Confidence: {percentage}
- Reasoning: {brief explanation}

Step 5: Build the duplicate detection skill

# Duplicate Detection Skill

## Input
- New issue title and body

## Process

### 1. Extract key terms
From the issue title and body, extract:
- Error messages (exact strings)
- Feature names or component names
- Behavioral descriptions

### 2. Search for duplicates
Query GitHub for similar issues:
- Search open issues with similar title keywords
- Search issues closed in the last 90 days
- Focus on issues with the same type label

### 3. Score similarity
For each candidate match:
- Title similarity (keyword overlap)
- Body similarity (shared error messages, same component)
- Same area label

### 4. Apply threshold
- **>85% similarity**: Almost certainly a duplicate
  → Apply `possible-duplicate` label
  → Comment: "This may be a duplicate of #XYZ. If your issue
    is different, please explain how and I will remove the label."
- **60-85% similarity**: Possibly related
  → Comment: "This might be related to #XYZ -- is this the
    same issue or a different one?"
- **<60% similarity**: Not a duplicate. No action.

## Important
- NEVER close the issue as a duplicate. Only humans do that.
- ALWAYS link to the potential duplicate so the author can compare.
- If multiple candidates match, link to the top 2-3 most similar.

Step 6: Build the auto-response skill

The auto-response skill generates contextual replies based on classification results:

# Auto-Response Skill

## Input
- Classification results (type, priority, confidence)
- Duplicate detection results

## For bugs (with reproduction steps)

> Thank you for reporting this bug, @{author}!
>
> **Triage summary:**
> - Type: Bug
> - Priority: {priority}
> - Area: {area}
> - Confidence: {confidence}%
>
> I have labeled this issue accordingly. A maintainer will review
> it based on the current priority queue.

## For bugs (missing reproduction steps)

> Thank you for reporting this, @{author}!
>
> To help us investigate, could you provide:
> 1. **Steps to reproduce** -- what exactly did you do?
> 2. **Expected behavior** -- what should have happened?
> 3. **Actual behavior** -- what happened instead?
> 4. **Environment** -- OS, browser/runtime, version of this project
>
> I have labeled this as `needs-info` for now. Once you add the
> details, a maintainer will pick it up.

## For feature requests

> Thank you for the feature request, @{author}!
>
> **Triage summary:**
> - Type: Feature Request
> - Priority: {priority}
> - Area: {area}
> - Confidence: {confidence}%
>
> Feature requests are reviewed by maintainers during planning cycles.
> If you would like to implement this yourself, please comment and
> we can discuss the approach.

## For questions

> Hi @{author}!
>
> This looks like a usage question rather than a bug.
>
> {if matched_docs: link to relevant documentation}
> {if no match: "A maintainer or community member should be able
> to help -- hang tight!"}
>
> If this turns out to be a bug after all, let us know and we will
> reclassify.

Step 7: Connect to webhooks for real-time triage

For the agent to triage issues in real time, it needs to know when new issues are opened.

Webhook-driven (recommended)

If your agent runs on managed hosting with a public URL, configure a GitHub webhook:

1. Go to your repository Settings → Webhooks → Add webhook
2. Payload URL: your agent's webhook endpoint
3. Content type: application/json
4. Events: select "Issues" (specifically the "opened" event)

The agent receives the webhook payload, extracts the issue number, and runs the triage pipeline.

Polling-based (works locally)

If running locally or without a public URL:

# Polling Configuration

Check for new issues every 3 minutes.

1. Query GitHub API: GET /repos/{owner}/{repo}/issues
   ?state=open&sort=created&direction=desc&per_page=10
2. Filter to issues created since the last poll
3. For each new issue without triage labels, run the full
   triage pipeline
4. Record the timestamp of the most recent processed issue

Rate limit budget:
- GitHub REST API: 5,000 requests/hour (authenticated)
- Polling every 3 min = 20 list requests/hour
- ~5 requests per issue for triage actions
- At 50 issues/day: ~270 requests/day (well within limits)

Tuning the agent with examples

The knowledge/triage-examples.md file is your calibration tool. Include 20-30 examples of correctly triaged issues so the agent learns your project's patterns:

# Triage Examples (knowledge/triage-examples.md)

## Example 1
**Title**: "App crashes when uploading files larger than 10MB"
**Classification**: bug
**Priority**: high
**Area**: area:api
**Reasoning**: Crash on a common operation (file upload) with a
specific trigger. High priority -- no workaround for large files.

## Example 2
**Title**: "Add dark mode support"
**Classification**: feature
**Priority**: medium
**Area**: area:ui
**Reasoning**: New feature request. Medium priority -- nice to have
but does not block core workflows.

## Example 3
**Title**: "How do I configure SSO with Okta?"
**Classification**: question
**Priority**: low
**Area**: area:auth
**Reasoning**: Usage question, not a bug. Low triage priority
(though high value to answer quickly).

## Example 4
**Title**: "Error: Cannot read property 'id' of undefined"
**Body**: "Getting this error after updating to v2.3.0"
**Classification**: bug
**Priority**: high
**Area**: needs investigation
**Reasoning**: Runtime error after version update suggests a
regression. High priority -- likely affects all users who updated.
Missing repro steps -- apply needs-info.

## Example 5
**Title**: "Make the search bar wider"
**Classification**: enhancement
**Priority**: low
**Area**: area:ui
**Reasoning**: Search bar exists (not a new feature). UI polish.
Low priority.

Monitoring triage quality

After deploying the agent, track its accuracy:

# Triage Quality Tracking Skill

## Weekly audit (every Sunday)
1. Query all issues labeled by the agent in the past 7 days
2. Check how many had labels changed by humans afterward
   (indicates a misclassification)
3. Calculate accuracy:
   - Type accuracy: % of type labels unchanged
   - Priority accuracy: % of priority labels unchanged
   - Duplicate accuracy: % of duplicate flags confirmed
4. Post metrics summary:

Triage Quality Report -- Week of March 17, 2026

Issues triaged: 47
Type accuracy: 93.6% (3 reclassified)
Priority accuracy: 87.2% (6 reprioritized)
Duplicate detection: 4 flagged, 3 confirmed (75%)
Needs-human-review: 5 issues (10.6%)
Average confidence: 84.2%

Most common misclassification:
  enhancement → feature (2 cases)

When accuracy drops in a specific area, add more examples to triage-examples.md for that edge case. Over time, the knowledge base becomes your project's institutional knowledge about how to classify issues -- valuable documentation even beyond the agent.

Getting the template

The complete triage agent template is available as a free download:

# Download the template
clawagora template pull github-issue-triage-agent:1.0.0

# Configure
cd github-issue-triage-agent
cp .env.example .env
# Edit .env with your GitHub token and repo details
# Edit knowledge/label-taxonomy.md for your project
# Edit knowledge/component-map.md for your codebase

You can also browse other issue management and GitHub automation templates on ClawAgora, or push your customized version back for other maintainers to discover and adapt.

FAQ

How accurate is AI-powered issue triage compared to manual triage?

Based on reports from teams running AI triage agents, accuracy for straightforward classifications (bug vs. feature vs. question) typically exceeds 90%. Priority assessment and duplicate detection are harder -- expect 75-85% accuracy initially, improving as you tune the SOUL.md rules and add examples. The key insight is that the agent handles the easy 80% perfectly, freeing maintainers to focus on the ambiguous 20% that requires human judgment.

Will the triage agent accidentally close legitimate issues or mislabel them?

The template is designed with a conservative approach: the agent labels issues and adds comments, but never closes issues automatically. For edge cases, it applies a "needs-human-review" label and notifies a maintainer. Mislabeling can happen, but labels are easily corrected by anyone with write access. The agent also includes a confidence score in its triage comment, so maintainers can quickly identify which decisions to double-check.

Can I use this with a private GitHub repository?

Yes. The GitHub MCP server uses a personal access token or GitHub App token with repo scope. As long as the token has access to the private repository, the agent can read issues, apply labels, and post comments. For organizations with strict security requirements, you can use a GitHub App with fine-grained permissions scoped to specific repositories.

How do I handle issues in multiple languages?

Modern LLMs handle multilingual content natively. The SOUL.md can include instructions like "Respond to issues in the same language they were written in" and "Classify issues regardless of language." For label names, keep them in English for consistency across the project. If your project explicitly supports multiple languages, add a language detection step to the triage skill that tags issues with a language label for routing to the right maintainer.

What does this cost to run?

The workspace template itself is free. Running costs depend on your LLM provider and hosting choice. A typical open-source project receiving 20-50 issues per day uses roughly $5-15/month in LLM API calls (at Claude or GPT-4 pricing). Running the agent locally is free beyond API costs. Managed hosting adds a monthly fee but includes uptime monitoring and automatic restarts. For most projects, the maintainer time saved far exceeds the API cost.