Skip to content

Add ModelScope as a llama.cpp model download source - #186

Open
NV-sschneider wants to merge 4 commits into
NVIDIA:developfrom
NV-sschneider:codex/modelscope-download-source
Open

NV-sschneider wants to merge 4 commits into
NVIDIA:developfrom
NV-sschneider:codex/modelscope-download-source

Conversation

@NV-sschneider

Copy link
Copy Markdown
Collaborator

Changelog title

Choose ModelScope for llama.cpp model downloads

Changelog body

  • Select Hugging Face or ModelScope in llama.cpp Settings. The saved choice controls model browsing, search, and downloads on that device; Hugging Face remains the default.

Bumps

  • services: minor
  • nvpair-cluster-manager: none
  • nvpair-engine-manager: minor
  • nvpair-errors: none
  • nvpair-job-scheduler: none
  • nvpair-manual-nodes: none
  • nvpair-node-info: none
  • nvpair-node-scanner: none
  • nvpair-node-settings: none
  • nvpair-proxy: none
  • nvpair-tui: patch
  • nvpair-ui-broker: minor
  • nvpair-workload-manager: none

Summary

Add a Model download source dropdown to llama.cpp Settings, with Hugging Face as the default and ModelScope as an alternative. The choice uses the existing preview, Apply, restart, persistence, and remote-settings flow.

The pinned llama.cpp downloader understands Hugging Face metadata. A Go loopback adapter translates ModelScope metadata and streams files while llama.cpp retains GGUF selection, caching, progress, and cancellation. The model browser and search use the selected source with separate caches. No dependencies are added.

Review guide

Review the four signed-off commits oldest first; each commit message includes a detailed guide and focused test commands.

  1. c87a529: start with services/nvpair-engine-manager/modelscope.go and its tests. Check immutable revisions, file-list limits, range/cancellation behavior, and credential isolation.
  2. c277c92: follow shared settings through the engine manager and broker. Check defaults, persistence/reload, source-only restart decisions, stopped-engine behavior, and Windows environment key casing.
  3. 4c41334: start with modelscope_catalog.go, then the source selection in catalog.go and manager.go. Check existing model filters, bounded requests, and provider-specific caches.
  4. eef90e5: start with EngineSettingsSection.tsx, then desktop types and the settings bridge. Check llama.cpp-only visibility and the existing draft/preview/Apply flow.

Test plan

Local verification on Windows x64:

Desktop gates passed: 26 Node tests and 285 Vitest tests passed, with two Unix-only cases skipped. Lint, typecheck, dead-code, contract, build-script, and SPDX checks passed.

The Electron production build and all 13 bundled Windows x64 executables built successfully. Broker, TUI, and shared-settings suites passed. All ModelScope and ModelSource tests, including the real-worker settings rebind/reload test, passed.

  • node scripts/spdx-headers.mjs
  • From desktop/: npm run verify:build-scripts, npm run service-contracts:check, npm run typecheck, npm run lint, npm run dead-code:check, and npm run test:unit -- --cache=false.
  • From desktop/: npm run build and npm run build:modular-binaries.
  • From services/nvpair-engine-manager/: go test ./... -count=1 -skip '^TestRemoveTreePreserving_ReportsUndeletableEntriesAndRemovesOthers$'.
  • From services/nvpair-ui-broker/ and services/nvpair-tui/: go test ./... -count=1.
  • From services/shared/: go test ./enginesettings -count=1.
  • python scripts/release-intent/validate_pr.py --description-file .build/modelscope-pr/pr-description.md, with the PR base/head supplied to enforce bot-owned paths.

Coverage includes HTTP fixtures, real worker settings rebind/reload, persisted source selection, actual child environment handling, broker journal recovery, catalogue cache separation, and desktop bridge payloads. Initial implementation validation also checked live public ModelScope metadata and file HEAD requests.

The named engine-manager permission test is excluded on Windows; it failed identically on the untouched upstream baseline during initial validation. The remaining full engine-manager suite has one failure, TestStopAll_StopsDetachedCommandDaemonBeforeReadiness (missing stop-marker file). The identical failure was reproduced by the full suite on untouched develop at c0d2b62; the test passes in isolation on this branch and in ten isolated upstream runs. This existing Windows suite-order/timing issue is not fixed in this PR.

Unix-only, Windows privilege-dependent, and opt-in live-engine checks remain untested where skipped. A complete live model-weight download and automated dropdown interaction have not been exercised.

Manual review: choose each source, Apply, reopen Settings, and confirm the saved value and model browser. Check the running-engine restart confirmation and that applying to a stopped engine leaves it stopped.

Compatibility and limits

Only public, ungated ModelScope repositories are supported. Hugging Face authorization and cookies are never forwarded to ModelScope or its download hosts. Repositories reaching ModelScope's 3,000-entry recursive listing cap are rejected rather than presenting incomplete models. Repository availability can differ between sources.

The engine-manager and broker bumps cover the additive setting/API behavior; the TUI patch covers the shared settings payload it consumes. Version and changelog files remain owned by release automation.

Checklist

  • Read the contribution guidelines and applicable architecture rules.
  • Every commit has a matching DCO sign-off.
  • Tests cover the change and documentation is updated.
  • Reviewed changed filenames, diff, and commit messages for credentials, private data, internal URLs, and generated artifacts.
  • Declared service bumps in the release-intent block.

Translate ModelScope metadata and file requests into the Hugging Face API shape
expected by the pinned llama.cpp downloader. Keep GGUF selection, caching,
progress and cancellation in the router.

Review guide (1 of 4; review the series oldest first):
- Start with services/nvpair-engine-manager/modelscope.go, then
  modelscope_test.go. Later commits select and connect this adapter.
- Check refs/tree/resolve translation, immutable file revisions (including
  abbreviated repository commits), snapshot bounds, path/hash validation,
  and rejection of incomplete 3,000-entry listings.
- Follow GET/HEAD streaming through CDN redirects: Range headers and
  cancellation must survive; Hugging Face Authorization/Cookie headers must
  stay inside the loopback adapter; interrupted transfers must abort.

Key coverage: TestModelScopeMetadata_PreservesSnapshotAfterBranchChanges,
TestModelScopeMetadata_UsesRepositoryCommitWhenFileRevisionIsAbsent, and
TestModelScopeDownload_PreservesRangesWithoutForwardingCredentials.
From services/nvpair-engine-manager:
  go test ./... -run '^TestModelScope(Metadata|Download)_' -count=1

Validation uses local HTTP fixtures plus live public metadata/HEAD checks.
A complete live llama.cpp model-weight download was not exercised.

Signed-off-by: Seth Schneider <sschneider@nvidia.com>
Carry the node-owned modelSource choice through the existing settings protocol,
persist it as runtime.model_source, and select MODEL_ENDPOINT at engine start.

Review guide (2 of 4):
- Start with services/shared/enginesettings/settings.go, then engine-manager
  settings.go and setport.go, followed by broker enginesettings.go. Trace the
  field through snapshot, preview, apply, journal and worker reload.
- Check the Hugging Face default, llama.cpp-only validation, preservation by
  other settings consumers, and source-only restart decisions. A stopped engine
  must stay stopped and previews must remain read-only.
- Inspect engine-manager lifecycle.go and modelsource.go: create the loopback
  adapter only during a ModelScope start, close it on shutdown, and prevent a
  saved MODEL_ENDPOINT with different Windows casing from overriding the choice.

Key coverage: TestModelSource_*, the broker's
TestSettingsCoordinator_PreservesModelSourceAcrossWorkerAndJournal, and
TestEngineManager_RelaysSettingsRebindAndReloadsWorker (real JSON-RPC workers
with temporary configuration and an isolated port).
From services/nvpair-engine-manager:
  go test ./... -run '^TestModelSource_|TestEngineManager_RelaysSettings' -count=1
From services/nvpair-ui-broker:
  go test ./... -run '^TestSettingsCoordinator_PreservesModelSource' -count=1

Signed-off-by: Seth Schneider <sschneider@nvidia.com>
Use the saved source for llama.cpp catalogue requests and reuse the existing
public, generative-model and Q4_K_M GGUF normalization.

Review guide (3 of 4):
- Start with services/nvpair-engine-manager/modelscope_catalog.go, then the
  source selection in catalog.go and manager.go.
- Check the official GET listing parameters, bounded candidate/file requests,
  existing GGUF/quantization filters, and ModelScope model-page links.
  Repositories with truncated 3,000-entry file lists must not block valid ones.
- Follow provider-prefixed cache keys through a source switch: identical queries
  must not reuse another provider's results. Errors must omit search text.
  Reading the selected source must not wait for an engine lifecycle operation.

Key coverage: TestModelScopeCatalog_* and
TestCatalog_KeepsSourceCachesSeparateWhenSelectionChanges.
From services/nvpair-engine-manager:
  go test ./... -run '^TestModelScopeCatalog_|^TestCatalog_KeepsSourceCachesSeparate' -count=1

No new dependency or desktop download implementation is introduced.

Signed-off-by: Seth Schneider <sschneider@nvidia.com>
Expose Hugging Face (default) and ModelScope in the existing llama.cpp Settings
editor, using the normal draft, preview, Apply and restart flow.

Review guide (4 of 4):
- Start with desktop/src/ui/components/BackendRow/EngineSettingsSection.tsx,
  then shared/types/engine-settings.ts and
  electron/service-bridge/engine-settings.ts.
- Check llama.cpp-only visibility, capability/disabled states, source-only dirty
  detection, stale-draft protection, enum validation, and preservation of the
  normalized source in preview/apply payloads.
- Read desktop/tests/modular/engine-settings-bridge.test.ts for payload coverage.
  These tests do not exercise dropdown interaction.
- Manually choose each source, Apply, confirm a running-engine restart when
  prompted, reopen Settings, and check the saved value and model browser.
  Also check that applying to a stopped engine leaves it stopped.
  docs/engine-settings.mdx describes the user flow.

Validation for the series: Windows x64 desktop and all bundled executables
built; all desktop gates passed (311 tests, two Unix-only skips); broker tests
passed. The engine-manager suite passed with
TestRemoveTreePreserving_ReportsUndeletableEntriesAndRemovesOthers excluded:
that permission test fails identically on untouched develop on this PC.
Feature tests passed again after the final ModelScope metadata correction.

Signed-off-by: Seth Schneider <sschneider@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant