AI Code Reviewer (BYOK Edition)

A high-performance, privacy-first code review web application that streams structured, dimension-weighted static code reviews powered by Claude Sonnet, GPT-4o, or local Ollama models.

Designed for safe public demonstration and production deployment using a zero application-level persistence, Bring Your Own Key (BYOK) architecture.

---

Key Features

• ⚡ Real-Time Streaming Reviews: Live object streaming powered by Vercel AI SDK (streamObject and experimental_useObject).
• 🔑 BYOK Zero-Storage Architecture: Visitors supply their own Anthropic or OpenAI API keys directly in the browser. Keys remain in volatile browser memory and are transmitted over HTTPS per review request. The application does not persist or log them.
• 📊 5-Dimension Deterministic Scoring: AI evaluates 5 distinct dimensions (Security, Correctness, Maintainability, Performance, Robustness); the overall Quality Score ($0\text{--}10$) is computed deterministically using weighted formula arithmetic.
• 🛡️ Pre-Flight Secret Scanner: Real-time client and server heuristic scanner detecting sensitive credentials (AWS access keys, OpenAI/Anthropic keys, GitHub tokens, DB URIs, JWTs) with an interactive confirmation guardrail.
• 🔒 HTTP & Streaming Security:
  • Strict Content-Type enforcement (application/json)
  • 50KB chunk-capped streaming reader with defensive cancellation and lock release
  • 10,000-character code limit with live UI counter
  • Strict UTF-8 fatal decoding
  • Security headers (CSP, HSTS, X-Frame-Options DENY, Permissions-Policy)
• 🚦 Dual-Engine Rate Limiter: Sliding-window limiter supporting Upstash Redis with a bounded in-memory LRU fallback (5,000 keys). Features timing-safe demo-token authentication and production fail-closed policies.
• 💻 Local Ollama Diagnostics & Hosted Gating: Full support for local models (codellama, qwen2.5-coder) with actionable diagnostics when Ollama is offline or models are missing. Automatically gated in hosted production.
• 📋 Copy Report & New Review Reset: One-click Markdown export of the entire structured review, and zero-reload review resets.
• 🧪 LLM Evaluation Benchmark Suite: Ground-truth dataset evaluation (evals/benchmark.ts) measuring vulnerability recall, false positive rates, prompt injection resistance, line reference accuracy, latency percentiles (P50/P95), and provider-specific estimated cost using an approximate token heuristic.

---

Tech Stack

Layer	Technology
Framework	Next.js 14 (App Router, Serverless Route Handlers)
AI Streaming	Vercel AI SDK v6 (streamObject, experimental_useObject)
AI Providers	@ai-sdk/anthropic, @ai-sdk/openai, ollama-ai-provider
Validation	Zod
Styling	Tailwind CSS v3
Syntax Highlighting	react-syntax-highlighter (Prism / VS Code Dark Plus)
Rate Limiting	@upstash/ratelimit, @upstash/redis + In-memory bounded LRU
Testing	Jest 30 + React Testing Library (13 suites, 114 tests)
Language	TypeScript (Strict Mode)

---

Getting Started

1. Clone & Install

git clone https://github.com/devtorque/ai-code-reviewer.git
cd ai-code-reviewer
npm install

2. Configure Environment

Copy .env.example to .env.local:

cp .env.example .env.local

# Optional Server-Side Keys (If omitted, visitors supply keys via BYOK)
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...

# Production Rate Limiting (Required for production deployments)
UPSTASH_REDIS_REST_URL=
UPSTASH_REDIS_REST_TOKEN=
DEMO_ACCESS_TOKEN=

# Local development only (Do NOT enable on hosted/public deployments)
# NEXT_PUBLIC_ENABLE_OLLAMA=true

3. Production Deployment (e.g. Vercel)

• BYOK Provider Keys: Optional on hosted platforms. Visitors supply their own API keys in their browser session.
• Production Rate Limiting: In production (NODE_ENV=production), rate limiting fails closed (503 Service Unavailable) by default if Redis is not configured. For public deployments, configure Upstash Redis credentials:

  UPSTASH_REDIS_REST_URL=https://...
  UPSTASH_REDIS_REST_TOKEN=...

  (Alternative for simple demos: set RATE_LIMIT_ALLOW_MEMORY_FALLBACK=true to enable bounded per-instance in-memory rate limiting).
• Ollama Gating: Do not set NEXT_PUBLIC_ENABLE_OLLAMA=true on hosted public deployments, as cloud visitors cannot reach a server-local daemon.

4. Run Development Server

npm run dev

Open http://localhost:3000 in your browser.

---

Running Local Ollama Models

1. Install Ollama
2. Pull a code model:

   ollama pull codellama

3. Start the Ollama daemon:

   ollama serve

4. Select Ollama (local) in the UI model picker.

---

Review Schema & Deterministic Scoring

Every review is validated against a structured Zod schema:

{
  summary: string
  dimensions: {
    security: number          // 0–10 (30% weight)
    correctness: number       // 0–10 (30% weight)
    maintainability: number   // 0–10 (20% weight)
    performance: number       // 0–10 (10% weight)
    robustness: number        // 0–10 (10% weight)
  }
  bugs: Array<{
    line: string
    description: string
    severity: 'low' | 'medium' | 'high'
  }>
  security: Array<{
    description: string
    severity: 'low' | 'medium' | 'high'
  }>
  suggestions: string[]
  positives: string[]
}

Deterministic Score Formula

$$\text{Quality Score} = \text{round}(0.3 \times \text{security} + 0.3 \times \text{correctness} + 0.2 \times \text{maintainability} + 0.1 \times \text{performance} + 0.1 \times \text{robustness}, 1)$$

---

Evaluation Benchmark Suite

Run the automated evaluation benchmark across the ground-truth dataset (evals/dataset.json):

# Run against Claude Sonnet (default)
RUN_LIVE_EVALS=true npm run eval

# Run against GPT-4o
RUN_LIVE_EVALS=true EVAL_MODEL=gpt4o npm run eval

The benchmark outputs persistent metric artifacts to evals/benchmark-report.json evaluating:

• Vulnerability detection recall
• Clean-code false positive rate
• Prompt injection resistance
• Line/symbol reference accuracy
• P50 & P95 response latencies
• Provider-specific estimated cost using an approximate token heuristic

---

Testing & Quality Assurance

# Run all unit, component, and route test suites (13 suites, 114 tests)
npm test

# Run ESLint validation
npm run lint

# Run full production build & TypeScript type check
npm run build && npx tsc --noEmit

---

Privacy & Security

For details on data handling, in-memory processing, and credential safety, visit the live /privacy page or read PRIVACY.md.