Repository navigation
Add ModelScope as a llama.cpp model download source - #186
Open
NV-sschneider wants to merge 4 commits into
Open
NV-sschneider wants to merge 4 commits into
NV-sschneider wants to merge 4 commits into
Conversation
Translate ModelScope metadata and file requests into the Hugging Face API shape expected by the pinned llama.cpp downloader. Keep GGUF selection, caching, progress and cancellation in the router. Review guide (1 of 4; review the series oldest first): - Start with services/nvpair-engine-manager/modelscope.go, then modelscope_test.go. Later commits select and connect this adapter. - Check refs/tree/resolve translation, immutable file revisions (including abbreviated repository commits), snapshot bounds, path/hash validation, and rejection of incomplete 3,000-entry listings. - Follow GET/HEAD streaming through CDN redirects: Range headers and cancellation must survive; Hugging Face Authorization/Cookie headers must stay inside the loopback adapter; interrupted transfers must abort. Key coverage: TestModelScopeMetadata_PreservesSnapshotAfterBranchChanges, TestModelScopeMetadata_UsesRepositoryCommitWhenFileRevisionIsAbsent, and TestModelScopeDownload_PreservesRangesWithoutForwardingCredentials. From services/nvpair-engine-manager: go test ./... -run '^TestModelScope(Metadata|Download)_' -count=1 Validation uses local HTTP fixtures plus live public metadata/HEAD checks. A complete live llama.cpp model-weight download was not exercised. Signed-off-by: Seth Schneider <sschneider@nvidia.com>
Carry the node-owned modelSource choice through the existing settings protocol, persist it as runtime.model_source, and select MODEL_ENDPOINT at engine start. Review guide (2 of 4): - Start with services/shared/enginesettings/settings.go, then engine-manager settings.go and setport.go, followed by broker enginesettings.go. Trace the field through snapshot, preview, apply, journal and worker reload. - Check the Hugging Face default, llama.cpp-only validation, preservation by other settings consumers, and source-only restart decisions. A stopped engine must stay stopped and previews must remain read-only. - Inspect engine-manager lifecycle.go and modelsource.go: create the loopback adapter only during a ModelScope start, close it on shutdown, and prevent a saved MODEL_ENDPOINT with different Windows casing from overriding the choice. Key coverage: TestModelSource_*, the broker's TestSettingsCoordinator_PreservesModelSourceAcrossWorkerAndJournal, and TestEngineManager_RelaysSettingsRebindAndReloadsWorker (real JSON-RPC workers with temporary configuration and an isolated port). From services/nvpair-engine-manager: go test ./... -run '^TestModelSource_|TestEngineManager_RelaysSettings' -count=1 From services/nvpair-ui-broker: go test ./... -run '^TestSettingsCoordinator_PreservesModelSource' -count=1 Signed-off-by: Seth Schneider <sschneider@nvidia.com>
Use the saved source for llama.cpp catalogue requests and reuse the existing public, generative-model and Q4_K_M GGUF normalization. Review guide (3 of 4): - Start with services/nvpair-engine-manager/modelscope_catalog.go, then the source selection in catalog.go and manager.go. - Check the official GET listing parameters, bounded candidate/file requests, existing GGUF/quantization filters, and ModelScope model-page links. Repositories with truncated 3,000-entry file lists must not block valid ones. - Follow provider-prefixed cache keys through a source switch: identical queries must not reuse another provider's results. Errors must omit search text. Reading the selected source must not wait for an engine lifecycle operation. Key coverage: TestModelScopeCatalog_* and TestCatalog_KeepsSourceCachesSeparateWhenSelectionChanges. From services/nvpair-engine-manager: go test ./... -run '^TestModelScopeCatalog_|^TestCatalog_KeepsSourceCachesSeparate' -count=1 No new dependency or desktop download implementation is introduced. Signed-off-by: Seth Schneider <sschneider@nvidia.com>
Expose Hugging Face (default) and ModelScope in the existing llama.cpp Settings editor, using the normal draft, preview, Apply and restart flow. Review guide (4 of 4): - Start with desktop/src/ui/components/BackendRow/EngineSettingsSection.tsx, then shared/types/engine-settings.ts and electron/service-bridge/engine-settings.ts. - Check llama.cpp-only visibility, capability/disabled states, source-only dirty detection, stale-draft protection, enum validation, and preservation of the normalized source in preview/apply payloads. - Read desktop/tests/modular/engine-settings-bridge.test.ts for payload coverage. These tests do not exercise dropdown interaction. - Manually choose each source, Apply, confirm a running-engine restart when prompted, reopen Settings, and check the saved value and model browser. Also check that applying to a stopped engine leaves it stopped. docs/engine-settings.mdx describes the user flow. Validation for the series: Windows x64 desktop and all bundled executables built; all desktop gates passed (311 tests, two Unix-only skips); broker tests passed. The engine-manager suite passed with TestRemoveTreePreserving_ReportsUndeletableEntriesAndRemovesOthers excluded: that permission test fails identically on untouched develop on this PC. Feature tests passed again after the final ModelScope metadata correction. Signed-off-by: Seth Schneider <sschneider@nvidia.com>
Noah-Tervalon-Nvidia
self-requested a review
October 29, 2026 19:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changelog title
Choose ModelScope for llama.cpp model downloads
Changelog body
Bumps
Summary
Add a Model download source dropdown to llama.cpp Settings, with Hugging Face as the default and ModelScope as an alternative. The choice uses the existing preview, Apply, restart, persistence, and remote-settings flow.
The pinned llama.cpp downloader understands Hugging Face metadata. A Go loopback adapter translates ModelScope metadata and streams files while llama.cpp retains GGUF selection, caching, progress, and cancellation. The model browser and search use the selected source with separate caches. No dependencies are added.
Review guide
Review the four signed-off commits oldest first; each commit message includes a detailed guide and focused test commands.
c87a529: start withservices/nvpair-engine-manager/modelscope.goand its tests. Check immutable revisions, file-list limits, range/cancellation behavior, and credential isolation.c277c92: follow shared settings through the engine manager and broker. Check defaults, persistence/reload, source-only restart decisions, stopped-engine behavior, and Windows environment key casing.4c41334: start withmodelscope_catalog.go, then the source selection incatalog.goandmanager.go. Check existing model filters, bounded requests, and provider-specific caches.eef90e5: start withEngineSettingsSection.tsx, then desktop types and the settings bridge. Check llama.cpp-only visibility and the existing draft/preview/Apply flow.Test plan
Local verification on Windows x64:
Desktop gates passed: 26 Node tests and 285 Vitest tests passed, with two Unix-only cases skipped. Lint, typecheck, dead-code, contract, build-script, and SPDX checks passed.
The Electron production build and all 13 bundled Windows x64 executables built successfully. Broker, TUI, and shared-settings suites passed. All ModelScope and ModelSource tests, including the real-worker settings rebind/reload test, passed.
node scripts/spdx-headers.mjsdesktop/:npm run verify:build-scripts,npm run service-contracts:check,npm run typecheck,npm run lint,npm run dead-code:check, andnpm run test:unit -- --cache=false.desktop/:npm run buildandnpm run build:modular-binaries.services/nvpair-engine-manager/:go test ./... -count=1 -skip '^TestRemoveTreePreserving_ReportsUndeletableEntriesAndRemovesOthers$'.services/nvpair-ui-broker/andservices/nvpair-tui/:go test ./... -count=1.services/shared/:go test ./enginesettings -count=1.python scripts/release-intent/validate_pr.py --description-file .build/modelscope-pr/pr-description.md, with the PR base/head supplied to enforce bot-owned paths.Coverage includes HTTP fixtures, real worker settings rebind/reload, persisted source selection, actual child environment handling, broker journal recovery, catalogue cache separation, and desktop bridge payloads. Initial implementation validation also checked live public ModelScope metadata and file HEAD requests.
The named engine-manager permission test is excluded on Windows; it failed identically on the untouched upstream baseline during initial validation. The remaining full engine-manager suite has one failure,
TestStopAll_StopsDetachedCommandDaemonBeforeReadiness(missing stop-marker file). The identical failure was reproduced by the full suite on untoucheddevelopatc0d2b62; the test passes in isolation on this branch and in ten isolated upstream runs. This existing Windows suite-order/timing issue is not fixed in this PR.Unix-only, Windows privilege-dependent, and opt-in live-engine checks remain untested where skipped. A complete live model-weight download and automated dropdown interaction have not been exercised.
Manual review: choose each source, Apply, reopen Settings, and confirm the saved value and model browser. Check the running-engine restart confirmation and that applying to a stopped engine leaves it stopped.
Compatibility and limits
Only public, ungated ModelScope repositories are supported. Hugging Face authorization and cookies are never forwarded to ModelScope or its download hosts. Repositories reaching ModelScope's 3,000-entry recursive listing cap are rejected rather than presenting incomplete models. Repository availability can differ between sources.
The engine-manager and broker bumps cover the additive setting/API behavior; the TUI patch covers the shared settings payload it consumes. Version and changelog files remain owned by release automation.
Checklist