-
Notifications
You must be signed in to change notification settings - Fork 0
feat(ai): adaptive embedding batch sizing circuit breaker and timeout fallback #117
Copy link
Copy link
Closed
Labels
priority/p1-highHigh impact work required for upcoming release milestoneHigh impact work required for upcoming release milestonescope/aiAI review pipeline, personas, prompts, LLM client, or RAGAI review pipeline, personas, prompts, LLM client, or RAGstatus/triageNew issue or PR awaiting technical triage and scopingNew issue or PR awaiting technical triage and scopingtype/featureNew user-facing functionality or capability additionNew user-facing functionality or capability addition
Milestone
Description
Activity
Metadata
Metadata
Assignees
Labels
priority/p1-highHigh impact work required for upcoming release milestoneHigh impact work required for upcoming release milestonescope/aiAI review pipeline, personas, prompts, LLM client, or RAGAI review pipeline, personas, prompts, LLM client, or RAGstatus/triageNew issue or PR awaiting technical triage and scopingNew issue or PR awaiting technical triage and scopingtype/featureNew user-facing functionality or capability additionNew user-facing functionality or capability addition
Problem Statement
Distributed OpenTelemetry traces indicate
ai.rag.ollama_embed_batchrepeatedly encounters 15.0sReadTimeout: timed outexceptions (75s+ total latency) during dense repository indexing, stalling RAG operations on overloaded local Ollama nodes.Proposed Changes
devops_cli.ai.rag.embeddingsadapting batch size dynamically (halving from 32 down to 16, 8, 4 on latency degradation or timeout).Acceptance Criteria
ReadTimeoutexceptions during full workspace re-indexing.tests/test_rag_embeddings.py.