Bug Report: Mutex Lock Failed During Indexing on Intel Mac #68

Activity

1. rothnic commented on Mar 10, 2026

   

   rothnic

   on Mar 10, 2026

   Author

   More actions

   Additional Analysis: Embedding Provider Issue

   We investigated using an external embedding provider (Ollama) as a workaround, but discovered the tool doesn't currently support custom API endpoints via environment variables.

   Code Analysis

   Looking at the embedding provider code in dist/embeddings/:

   1. OpenAI Provider (openai.js) accepts an apiEndpoint parameter:

   constructor(modelName = 'text-embedding-3-small', apiKey, apiEndpoint = 'https://api.openai.com/v1')

   2. Environment Variables (types.js) only support:

      • EMBEDDING_PROVIDER (transformers|openai)
      • EMBEDDING_MODEL
      • OPENAI_API_KEY

   3. Missing: No OPENAI_BASE_URL or similar for custom endpoints

   Workaround Attempted

   We tried using Ollama (OpenAI-compatible API at http://100.79.168.98:11434/v1):

   EMBEDDING_PROVIDER=openai \
   OPENAI_BASE_URL=http://100.79.168.98:11434/v1 \
   OPENAI_API_KEY=ollama \
   npx codebase-context metadata

   But the apiEndpoint parameter isn't read from environment variables, so it still tries to use https://api.openai.com/v1.

   Potential Fix

   The fix would be in dist/embeddings/types.js:

   export const DEFAULT_EMBEDDING_CONFIG = {
       provider: parseEmbeddingProviderName(process.env.EMBEDDING_PROVIDER) ?? 'transformers',
       model: DEFAULT_MODEL,
       batchSize: 32,
       maxRetries: 3,
       apiKey: process.env.OPENAI_API_KEY,
       apiEndpoint: process.env.OPENAI_BASE_URL || 'https://api.openai.com/v1'  // Add this
   };

   Alternative: Native Ollama Support

   We also noticed there's a placeholder for Ollama:

   if (mergedConfig.provider === 'ollama') {
       console.warn('Ollama provider not yet implemented, falling back to Transformers.js');
   }

   Native Ollama support would be ideal since it would allow using local embeddings without the mutex/threading issues on Intel Macs.

   Summary

   The mutex crash appears to be in the @huggingface/transformers library's native bindings on Intel Macs. Workarounds:

   1. Short-term: Add OPENAI_BASE_URL env var support to use external providers
   2. Medium-term: Implement native Ollama provider
   3. Long-term: Fix the underlying threading issue with Transformers.js on Intel Mac

   Happy to test any patches or provide more system info!
2. rothnic commented on Mar 11, 2026

   

   rothnic

   on Mar 11, 2026

   Author

   More actions

   Created separate issue #70 to track the custom OpenAI-compatible API endpoint support (OPENAI_BASE_URL). This will allow using Ollama, LiteLLM, and other OpenAI-compatible providers instead of the official OpenAI API.\n\nThe mutex lock issue (#68) and the base URL support (#70) are separate concerns, so tracking them independently.
3. rothnic

   added a commit that references this issue on Mar 11, 2026

   feat(embeddings): add native Ollama provider for local embeddings

   ...

   75d66d3
4. PatrickSys commented on Mar 12, 2026

   

   PatrickSys

   on Mar 12, 2026

   Owner

   More actions

   Hey @rothnic, thanks for the detailed analysis; you basically solved it.

   The root cause is onnxruntime-node@1.21.0, which has a well-documented C++ mutex
   destruction order bug on macOS tracked in microsoft/onnxruntime#24579, fixed in 1.24.x. @huggingface/transformers@3.8.1 (current stable) still pins 1.21.0 exactly.

   To answer your first post questions directly:

   1. No special system requirements needed for Intel Macs - the issue isn't the setup here.
   2. Yes, exactly this. Intel vs Apple Silicon doesn't matter - both will have the same bug.
   3. Isn't a memory or thread limit. The crash is in C++ static destructor ordering at
      shutdown, not resource exhaustion. The 50% mark is coincidental.
   4. Yes: once the fix for Feature Request: Support custom OpenAI-compatible API endpoints (OPENAI_BASE_URL) #70 lands, EMBEDDING_PROVIDER=openai + OPENAI_BASE_URL=http://your-ollama:11434/v1 bypasses Transformers.js entirely and sidesteps the crash.

   On your three workarounds:

   • Short-term (OPENAI_BASE_URL) is covered by Feature Request: Support custom OpenAI-compatible API endpoints (OPENAI_BASE_URL) #70, which you already opened and have a branch for.
   • Medium-term (native Ollama provider): I've opened feat: implement native Ollama embedding provider #71 to track this properly.
   • Long-term (fix the threading issue): @huggingface/transformers v4 pre-release already brings onnxruntime-node@1.24.3 which fixes this. I've opened chore: upgrade @huggingface/transformers to v4 #72 to track the upgrade once v4 is stable.