Feature/agent test set p1 UI - #478
Merged
Merged
Conversation
Add the agent-test-service.js REST client, its api-endpoints.js URL group, and agentTestTypes.js JSDoc typedefs for suites/cases/runs/results, following the existing agent-service.js conventions. Wires the $agentTestTypes alias in svelte.config.js. No page consumes this yet (Tasks 3-6).
Adds the suite list page (page/agent-test): agent filter with Filter/Reset, table of suites (name/agent/enabled), create modal, Swal delete confirmation, and loading/empty/error states. Registers the new route in svelte.config.js's prerender entries, required for adapter-static's crawl:false + strict build to succeed.
Completes the four agent-test pages on top of the existing suite list, then adds the UI for multi-model runs and AI-assisted case extraction. New routes: the suite detail page (settings, case list with multi-select, run history), the case editor (turns, per-turn and case-level assertions, initial states, tool mocks with state writes; caseId=new creates), and the run detail page (case -> turn -> assertion, observed tool calls, 2s polling while non-terminal, cancel, re-run failures). Shared status/validation helpers live in $lib/helpers/utils/agent-test.js. Failed and Errored are deliberately styled and worded differently throughout. Collapsing them is the one mistake that makes "the harness broke" read as "the agent regressed", and the run page exists to tell those apart. Multi-model runs. The run button opens a modal that picks any number of chat-capable models and shows the resulting execution count up front, since each model multiplies the token cost. The run page grows a comparison grid: one row per case, one column per model, status and duration in each cell, per-model totals in the footer. Durations are summed rather than averaged -- "how long does the whole suite take on this model" is what decides whether an upgrade is affordable, and an average hides one pathological case. AI extraction. The record modal takes an optional extraction model, defaulting to off so the deterministic recorder (which never leaves the system) stays the default. Picking one warns which vendor the user messages and tool names go to, and states that tool arguments and results do not. recordCase became recordCases and returns a list. Judge provider/model became cascading dropdowns fed by /llm-configs and filtered to chat-capable models. Unlike the pattern they copy, a stored value the catalogue no longer offers stays selectable -- suite PUT is a full replace, so silently dropping it would blank the field on any unrelated save. i18n: the pages already called $_(), but no key existed in en.json or zh.json, so svelte-i18n fell back to the key and Chinese rendered as English. All 248 keys now resolve in both locales, including the strings that were previously hardcoded outside the markup (confirm dialogs, toasts, validation text, and the accessible names of icon-only buttons). Also fixes a relative goto in the suite list that resolved against the current URL. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
A run that stops before executing anything produces no case results, so the per-case error alert had nothing to render and the page showed status Error, 0/0/0/0, and "This run produced no case results." -- with the actual reason nowhere on screen. It now renders run.error above the (empty) results section, replaces that misleading empty-state line with a pointer to it, and repeats the reason inline in the suite's run history so it is visible without opening the run at all. The commonest way to reach that state was a trap this UI created: disabled cases were selectable and "Run Selected" happily queued them, which the executor then rejected wholesale. Selection counts now consider only enabled cases, a mixed selection warns how many will be skipped, an all-disabled selection is blocked with the reason, and only enabled ids are sent. Also translates the remaining Chinese in agentTestTypes.js to English. Only comments and JSDoc -- zh.json holds translations, which are data and untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
yuyixg
pushed a commit
to yuyixg/BotSharp-UI
that referenced
this pull request
Aug 21, 2026
main gained "replace agent test styles" (88568bc) after PR SciSharp#478 was merged: the four agent-test pages were rewritten against a new 880-line stylesheet, +2321/-1055. This branch had added its features to the pages as they were before that, so three of them conflicted. Resolved by taking main's version of all three wholesale and re-applying each addition in the new idiom, rather than untangling the conflict markers. A mechanical merge would have left the new features wearing the old Bootstrap classes inside restyled pages -- balanced markup that looks broken. Every addition now uses the ats-* classes: ats-card / ats-card-section for the two new cards, ats-input, ats-textarea, ats-switch, ats-badge with ats-tone-*, ats-btn-icon for the row actions, and the grid/col-span layout the rest of the page uses. Two things the conflict surfaced rather than caused: - One nesting mistake of mine, caught by reading the structure rather than by the build: the cards share a single flex column, and my first attempt closed it after the new Registration card and opened two invented wrappers. Tags balanced, so it compiled -- the remaining cards would just have lost their column spacing. - The restyled markup still carried the pre-judge llmJudge wording ("Always fails in P1"), whose keys this branch had already replaced, and which is no longer true: llmJudge is scored by LlmAgentTestJudge. Pointed both usages at the corrected strings instead of reviving the stale keys, and dropped the warning tone from a hint that now describes normal behaviour. Four help strings picked up their apostrophes again, so they reuse the keys the dictionaries already hold rather than adding near-duplicates that differ by punctuation. Verified: the UI builds, no missing translation keys in either dictionary, and every feature this branch adds is present in the merged files -- case type, entry agent picker, history rows, registration panel, involved-agent picker, agentChain mode picker, copy button, the enable-toggle payload fix, the type and history columns, routing accuracy, the latency/token/cost table, agent chain badges, per-result model time, and the scope panel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.