Local Whisper - Private AI Chat Assistant: Pure Browser, Zero Backend

Download and run local LLMs within your browser.

Live site: https://encryptioner.github.io/private-chat/

Blog post:

Features

• 🤖 Local AI Chat: Run large language models entirely in your browser
• 🔌 Plug-and-Play Embed: Easily integrate into any website
• 🔒 Privacy First: No data leaves your browser, completely offline-capable
• 📱 Responsive Design: Works across all devices and screen sizes
• ⚡ WebAssembly Powered: Fast inference using Wllama
• 🎯 Easy Integration: Simple embed script for websites
• 🧠 Site-Aware (RAG): When embedded, the widget scrapes the host page and answers from its own content — with clickable links that scroll to the relevant section. 100% in-browser, no backend.

Quick Start

Running Locally

1. Install dependencies

pnpm install

2. Start development server

pnpm run dev

3. Navigate to http://localhost:5173/

Building for Production

GitHub Pages (Default):

# Build for GitHub Pages
pnpm run build

# Preview production build
pnpm run preview

Standalone Domain:

# Build for custom domain
DEPLOYMENT_TYPE=standalone pnpm run build

The application automatically detects the deployment environment and adjusts URLs accordingly.

Embed Integration

Plug-and-Play (Recommended)

Just add one script tag - no setup required! A floating chat button appears automatically:

<script 
  id="aiChatEmbedScript" 
  defer 
  src="https://encryptioner.github.io/private-chat/embed.js">
</script>
<!-- That's it! Floating chat widget appears automatically -->

Custom Div Integration

If you want the chat to load in a specific location:

<script 
  id="aiChatEmbedScript" 
  defer 
  src="https://encryptioner.github.io/private-chat/embed.js">
</script>

<!-- Chat loads automatically here -->
<div id="ai-chat-embed-div"></div>

Site-Aware Mode (RAG)

When embedded, the widget reads the host page and answers from its own content — with clickable "Related sections" links. It works on any site (same-origin, cross-origin, static, or dynamic SPA) with zero config: embed.js scrapes the host page and bridges the content to the chat iframe via postMessage.

Set window.PRIVATE_CHAT_CONFIG before the embed script loads to customize it (all fields optional):

<script>
  window.PRIVATE_CHAT_CONFIG = {
    label: "Acme Labs",              // shown in the widget greeting
    siteIndexUrl: "/site-index.json", // optional pre-built cross-page index
    getSections: null                 // optional custom scraper (see docs)
  };
</script>
<script id="aiChatEmbedScript" defer src="https://encryptioner.github.io/private-chat/embed.js"></script>

• No config → the widget live-scrapes the current page and grounds answers in it.
• getSections → supply your own scraper (CMS, JSON-LD, an API, a content region). Runs in your page's context; return {anchor?, title?, url?, text} per section.
• Dynamic/SPA sites re-scrape automatically on client-side navigation.
• The widget answers only from retrieved content; if a question isn't covered, it says so.

📖 Full integration guide: docs/SITE-INTEGRATION.md — the three running modes, the custom-scraper contract with examples, SPA behavior, link handling, troubleshooting, and privacy/security notes. Writing your own scraper? docs/CUSTOM-SCRAPER-GUIDE.md — step-by-step cookbook with patterns + a local test tool. Want to load a custom/self-trained model instead of just grounding/persona config? docs/CUSTOM-MODEL-TRAINING.md — the planned modelUrl loading support, an optional CPU LoRA training recipe, and why training usually isn't needed.

Advanced Integration

<button onclick="openChat()">Open AI Chat</button>
<div id="my-chat-container"></div>

<script 
  id="aiChatEmbedScript" 
  defer 
  src="https://encryptioner.github.io/private-chat/embed.js">
</script>

<script>
function openChat() {
  if (window.loadChatApp) {
    window.loadChatApp('my-chat-container');
  }
}
</script>

Development & Testing

For detailed development and testing instructions, see Testing Guide.

Quick Testing

1. Local Development

   pnpm run dev  # Main app at http://localhost:5173

2. Test Embed Locally

   pnpm run build:embed  # Creates dist/embed.js

3. Production Build

   pnpm run build  # Full production build

Deployment

Automatic GitHub Pages

Push to main branch - GitHub Actions automatically:

• Builds the application
• Downloads models
• Deploys to GitHub Pages
• Makes embed script available at: https://encryptioner.github.io/private-chat/embed.js

Manual Deployment

pnpm run build
# Deploy dist/ folder to your hosting provider

Configuration

The app supports various GGUF models from Hugging Face:

• Gemma 3 270M (default)
• Gemma 3 1B
• Llama 3.2 1B
• Qwen3 0.6B
• SmolLM2 360M
• Custom GGUF file upload (max 2GB)

Models are automatically cached in browser for offline use. A failed or interrupted download always recovers on reload, downloads aren't resumable, and ~2GB is the practical ceiling (WebAssembly memory). See docs/MODEL-LOADING.md for every failure mode, the recovery flow, and browser storage limits.

Credits

• Wllama
• SmolLm - HuggingFace
• Llama 3.2 - Meta
• Daniel Chifamba

---

Support

If you find my work useful, consider supporting it: