Devless — Infrastructure

Automated coding agent that picks up GitHub issues, generates code changes, and opens Pull Requests. PRs auto-merge after passing review and checks.

Orchestrator Implementations

Implementation	Stack	Description
orchestrator/	Python + Aider	Main implementation with state machine pipeline
orchestrator-agent/	Python + Strands + Aider	Agent-based with skills (SKILL.md)

Architecture

Edit docs/architecture.drawio in draw.io — CI auto-renders to SVG on push.

Events handled:

• issues / issue_comment → full pipeline (clone → code → PR)
• pull_request_review_comment / pull_request_review → PR feedback pipeline (checkout → changes → push to same PR)

How it works end-to-end

Orchestrator pipeline (State Machine)

Orchestrator pipeline (Agent-based)

---

Deployment

There are two ways to deploy: automated via GitHub Actions (recommended) or manually from your local machine.

Option A: GitHub Actions (recommended)

Push to main and everything deploys automatically. One-time setup:

1. Create the S3 state bucket:

aws s3 mb s3://devless-tf-state --region eu-central-1

2. Do an initial terraform apply locally to create the OIDC role (chicken-and-egg):

cd terraform
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values (private key is handled separately)

# Save your GitHub App private key PEM file:
# pbpaste > github-app-private-key.pem   (or copy it however you prefer)

# Build Lambda functions
./modules/lambda/build.sh

# Deploy
export TF_VAR_github_app_private_key="$(cat github-app-private-key.pem)"
terraform init
terraform apply

3. Note the github_actions_role_arn output and add these GitHub Actions secrets to this repo (Settings → Secrets and variables → Actions → New repository secret):

Secret	Value
AWS_REGION	eu-central-1
AWS_ROLE_ARN	the github_actions_role_arn terraform output (see step 2)
TF_VAR_github_app_id	your GitHub App ID
TF_VAR_github_app_private_key	your GitHub App private key (full PEM content)
TF_VAR_github_app_installation_id	your GitHub App installation ID
TF_VAR_webhook_secret	your webhook HMAC secret
TF_VAR_github_actions_org	your GitHub org name (e.g. typekpb)

Get the role ARN from the initial local apply:

terraform output github_actions_role_arn

No static AWS keys needed — the workflow uses OIDC federation to assume this role with temporary credentials.

4. Push to main. The deploy.yml workflow handles everything from now on:
   • terraform apply (infra changes)
   • Docker build + push to ECR
   • ECS redeployment

Every subsequent push to main deploys automatically. No manual steps.

Option B: Manual deployment

If you prefer to deploy without CI/CD:

1. Create the S3 state bucket:

aws s3 mb s3://devless-tf-state --region eu-central-1

2. Create terraform/terraform.tfvars from the example:

cd terraform
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values

3. Build Lambda functions:

./modules/lambda/build.sh

4. Deploy infrastructure:

export TF_VAR_github_app_private_key="$(cat github-app-private-key.pem)"
terraform init
terraform apply

5. Build and push the orchestrator image:

ECR_URL=$(terraform output -raw ecr_repository_url)

aws ecr get-login-password --region eu-central-1 | \
  docker login --username AWS --password-stdin "$ECR_URL"

docker build -t "$ECR_URL:latest" ../orchestrator/
docker push "$ECR_URL:latest"

6. Trigger ECS redeployment:

aws ecs update-service \
  --cluster devless-cluster \
  --service devless-orchestrator \
  --force-new-deployment \
  --region eu-central-1

Repeat steps 4–6 for every change.

---

LLM backend configuration

The orchestrator uses Amazon Bedrock (Claude). Authenticates via the ECS task IAM role — no API keys needed.

.devless/config.yml in target repos:

model: bedrock/eu.anthropic.claude-sonnet-4-20250514-v1:0

The IAM module includes bedrock:InvokeModel permission on the task role.

---

Bootstrapping a target repository

Once the infrastructure is deployed, each repo you want Devless to work on needs a one-time webhook registration and a CI workflow.

Required: CI workflow

.github/workflows/ci.yml — must run on pull requests so Devless can detect failures and auto-fix them. The actual steps depend on your language/stack:

name: CI
on:
  pull_request:
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      # Add your language-specific build/test steps here, e.g.:
      # Python:  pip install ".[test]" && pytest
      # Node.js: npm ci && npm test
      # Go:      go test ./...
      # Java:    mvn verify
      # Rust:    cargo test
      - run: <your-test-command>

One-time: webhook registration

In the target repo, go to Settings → Webhooks → Add webhook:

Field	Value
Payload URL	your API Gateway URL (terraform output api_gateway_url)
Content type	application/json
Secret	your webhook HMAC secret (terraform output -raw webhook_secret)
Events	select "Let me select individual events" → check Issues, Issue comments, Pull request review comments, Pull request reviews
Active	checked

Click "Add webhook". This is a one-time step per target repo.

Updating existing webhooks: If you already have a webhook configured, go to Settings → Webhooks → click on your webhook → scroll to "Which events would you like to trigger this webhook?" → ensure all required events are checked: Issues, Issue comments, Pull request review comments, Pull request reviews.

Optional: project config

.devless/config.yml — customize how Devless processes your repo:

# Amazon Bedrock backend (default — uses ECS task role, no API key needed)
model: bedrock/eu.anthropic.claude-sonnet-4-20250514-v1:0
test_command: npm test   # command Aider runs to verify changes (default: pytest)
max_iterations: 10       # max Aider retry loops (default: 10)

# OpenAI backend (requires OPENAI_API_KEY on the orchestrator)
model: o3                # or: gpt-4o, o1, gpt-4-turbo
test_command: npm test
max_iterations: 10

# Azure OpenAI backend (requires AZURE_API_KEY, AZURE_API_BASE, AZURE_API_VERSION)
model: azure/gpt-4o     # must match your Azure deployment name
test_command: npm test
max_iterations: 10

See examples/ for ready-to-copy config files.

The model field uses Aider's model naming: bedrock/<model-id> for Bedrock (default), plain name for OpenAI (e.g. o3), or azure/<deployment-name> for Azure OpenAI. Bedrock authenticates via the ECS task role. OpenAI requires OPENAI_API_KEY. Azure requires AZURE_API_KEY, AZURE_API_BASE, and AZURE_API_VERSION.

The test_command should match whatever your CI runs. Examples:

• Python: pytest
• Node.js: npm test
• Go: go test ./...
• Java: mvn test
• Rust: cargo test
• Generic: make test

Optional: repo-specific skills

.agents/skills/
  my-skill/
    SKILL.md

Auto-merge setup

In the target repo settings:

1. Enable "Allow auto-merge" under General → Pull Requests
2. Add a branch protection rule on the default branch:
   • Require pull request reviews (at least 1 approval)
   • Require status checks to pass (select your CI workflow)

Expected repo structure

Any language, any framework — just needs a CI workflow and optionally a config:

your-project/
├── .devless/config.yml           # Optional: model, test_command, max_iterations
├── .agents/skills/             # Optional: repo-specific Devless skills
├── .github/workflows/
│   └── ci.yml                  # Required: runs your tests on PRs
└── ...                         # Your code (any language/structure)

Then open an issue — Devless picks it up, posts status comments, opens a PR, and auto-merges after approval.

---

Local development

Testing Aider + Bedrock directly

Run Aider against a target repo without the orchestrator to verify the LLM integration works:

# Requires Python 3.11 or 3.12 (aider-chat doesn't support 3.14 yet)
python3.12 -m venv .venv && source .venv/bin/activate
pip install aider-chat

cd /tmp
git clone https://github.com/typekpb/devless-demo.git
cd devless-demo

export AWS_REGION=eu-central-1

aider --model bedrock/eu.anthropic.claude-sonnet-4-20250514-v1:0 \
  --message "Describe the change you want to test"

This uses your local AWS credentials for Bedrock. If it works here but not in ECS, the issue is with the orchestrator or IAM roles, not Aider/Bedrock.

Running the orchestrator locally

The orchestrator can run locally against the real AWS services (SQS, S3, Bedrock). You need the GitHub App private key and AWS credentials configured.

cd orchestrator
# Requires Python 3.11 or 3.12 (aider-chat doesn't support 3.14 yet)
python3.12 -m venv .venv && source .venv/bin/activate
pip install -e .

# Set required environment variables
export SQS_QUEUE_URL=$(cd ../terraform && terraform output -raw sqs_queue_url)
export S3_BUCKET=$(cd ../terraform && terraform output -raw s3_bucket_name)
export AWS_REGION=eu-central-1
export GITHUB_APP_ID=your-app-id
export GITHUB_APP_PRIVATE_KEY="$(cat ../terraform/github-app-private-key.pem)"
export GITHUB_APP_INSTALLATION_ID=your-installation-id

# Run
python -m main

The orchestrator will poll SQS and process messages exactly as it does in ECS. Trigger a webhook by commenting on a GitHub issue in a target repo.

To process a single issue without SQS (useful for debugging):

cd /tmp
git clone https://github.com/typekpb/devless-demo.git
cd devless-demo
# Requires Python 3.11 or 3.12 (aider-chat doesn't support 3.14 yet)
python3.12 -m venv .venv && source .venv/bin/activate
pip install aider-chat

export AWS_REGION=eu-central-1

python -c "
from aider.coders import Coder
from aider.models import Model
from aider.io import InputOutput

io = InputOutput(yes=True)
model = Model('bedrock/eu.anthropic.claude-sonnet-4-20250514-v1:0')
coder = Coder.create(main_model=model, io=io, fnames=[])
coder.run('Create a landing page with hero section and nav bar')
"

---

Prerequisites

• AWS account with CLI configured
• Terraform >= 1.10
• Docker
• GitHub App (see setup below)
• OpenAI API key (only if using OpenAI backend instead of Bedrock)

---

Creating the GitHub App

The orchestrator authenticates as a GitHub App to clone repos, open PRs, and post comments. Apps use short-lived installation tokens — no PATs to rotate.

GitHub Apps can only be created via the web UI (there is no CLI command).

Step 1: Create the app

1. Go to Settings → Developer settings → GitHub Apps → New GitHub App

   • Org-level: https://github.com/organizations/typekpb/settings/apps/new
   • Personal: https://github.com/settings/apps/new

2. Fill in:

   • App name: Devless (or any name you like)
   • Homepage URL: your repo URL (e.g. https://github.com/typekpb/devless)
   • Webhook: uncheck "Active" (the orchestrator doesn't receive webhooks via the app — it uses API Gateway)

3. Set Repository permissions:

   Permission	Access
   Contents	Read & write
   Issues	Read & write
   Pull requests	Read & write
   Checks	Read-only
   Metadata	Read-only (auto-granted)

   No organization or account permissions are needed.

4. Under Where can this GitHub App be installed?, select "Only on this account".

5. Click Create GitHub App.

Step 2: Generate a private key

1. On the app's settings page, scroll to Private keys
2. Click Generate a private key
3. A .pem file downloads — save it as terraform/github-app-private-key.pem

Step 3: Note the App ID

The App ID is shown at the top of the app's settings page (a numeric value like 123456). You'll need this for github_app_id in Terraform.

Step 4: Install the app

1. On the app's settings page, click Install App in the left sidebar
2. Choose your organization or account
3. Select Only select repositories and pick the repos Devless should work on (you can add more later)
4. Click Install

Step 5: Get the Installation ID

After installing, the URL will look like:

https://github.com/organizations/typekpb/settings/installations/12345678

The number at the end (12345678) is your Installation ID. You'll need this for github_app_installation_id in Terraform.

Alternatively, retrieve it via the API using the gh CLI:

APP_ID=123456  # your App ID from Step 3

# List installations (requires the app's JWT — easier to just grab from the URL above)
gh api "/app/installations" \
  --header "Authorization: Bearer $(gh auth token)" \
  | jq '.[].id'

Summary of values

Value	Where to find it	Terraform variable
App ID	App settings page, top	github_app_id
Private key	Downloaded .pem file	TF_VAR_github_app_private_key (env var)
Installation ID	URL after installing the app	github_app_installation_id