Skip to content

FEAT: Load Scores Per Message in GUI - #2352

Open
jbolor21 wants to merge 27 commits into
microsoft:mainfrom
jbolor21:jbolor/GUI_load_message_scores
Open

FEAT: Load Scores Per Message in GUI#2352
jbolor21 wants to merge 27 commits into
microsoft:mainfrom
jbolor21:jbolor/GUI_load_message_scores

Conversation

@jbolor21

@jbolor21 jbolor21 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Description

Adds score visibility to the conversation UI and visibility into the objective for loaded attacks.

  • Loads scores with their originating message pieces.
  • Displays text and media scores next to the content they evaluate.
  • Marks and prioritizes the attack’s objective score.
  • Supports multiple scores with compact tabs and a More menu.
  • Shows scorer, score type, objective status, category, rationale, and source piece in the details panel.
  • Adds accessible names, tooltips, and mobile-sized touch targets.
  • Displays the attack objective in a collapsible conversation header.

UI behavior

  • A single score appears as a compact score chip.
  • Multiple scores use a stacked-score trigger.
  • Up to three score-value tabs are visible without horizontal scrolling.
  • Additional scores are available from the More menu.
  • Scorer and objective metadata remain available in the selected details panel, tooltip, and accessible name.

Screenshots:

Single Score & Single Piece Response:
image

Sample Score Details Card (when you press on score value)
image

multiple scores on single piece:
(stacked scores indicate multiple scores)
image
tabs revealing details on multiple scores:
image
Overflow (more scores than what fits in window)
image

multiple pieces:
image

Tests and Documentation

All existing tests pass and new tests added/updated:

  • Backend score serialization and response-contract tests.
  • Backend message mapper tests using persisted score data.
  • Attack-service tests for objective-score propagation and manual-attack objective handling.
  • Frontend message-mapping tests for text and media score provenance.
  • Score rendering, selection, overflow-menu, tooltip, and accessibility tests.
  • Objective-header rendering, overflow, and expansion tests.
  • Mobile E2E touch-target coverage for score tabs, menus, and the objective header.

@romanlutz Roman Lutz (romanlutz) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you attach screenshots or videos?

Other comments AI generated but they apply nontheless :-)

Comment thread frontend/src/components/Chat/ObjectiveHeader.styles.ts
Comment thread frontend/src/App.tsx
Comment thread frontend/src/components/Chat/MessageList.styles.ts Outdated
Comment thread frontend/e2e/touch-targets.spec.ts
Comment thread frontend/src/utils/messageMapper.test.ts Outdated
@romanlutz

Copy link
Copy Markdown
Contributor

What does this look like with multiple scores?

Also the spacing is not right. This doesn't look very good with "Score" and no space between.

Comment thread frontend/src/utils/messageMapper.ts Outdated
Comment thread frontend/src/types/index.ts Outdated
Comment thread pyrit/backend/services/attack_service.py Outdated
Comment thread frontend/src/components/Chat/MessageList.styles.ts
Comment thread frontend/e2e/touch-targets.spec.ts Outdated
Comment thread frontend/src/utils/messageMapper.ts Outdated
Comment thread frontend/src/components/Chat/MessageList.test.tsx Outdated
@romanlutz

Copy link
Copy Markdown
Contributor

I think would be preferable if the score was at the bottom of a piece rather than vertically centered.

I also think the multiple score experience is a bit confusing.

Image

I had to look back and forth a few times to realize the 0.91 was the same as the one presented to the left and not a separate score.

Wild idea: Would it be hard to show them visually stacked (can be static with 3 maybe, doesn't need to adjust the number of levels to the number of scores)
Image
and if you click on it it shows the same score details card as it shows for the first score right now, but the card has a pivot/tablist item at the top to switch between the various scores? [In theory, that could also overflow but there's a "with overflow" option if you check the doc link below
https://storybooks.fluentui.dev/react/?path=/docs/components-tablist--docs&globals=storybook_fluentui-react-addon_dir:rtl ]

@romanlutz Roman Lutz (romanlutz) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are GHCP generated and at least worth a look. Making another manual pass as well

Comment thread frontend/src/components/Chat/MessageList.tsx Outdated
Comment thread frontend/src/components/Chat/MessageList.tsx Outdated
Comment thread pyrit/backend/models/attacks.py
Comment thread pyrit/backend/mappers/attack_mappers.py Outdated
Comment thread frontend/src/components/Chat/MessageList.tsx Outdated
Comment thread frontend/src/components/Chat/MessageList.tsx Outdated
Comment thread frontend/src/utils/messageMapper.ts Outdated
Comment thread pyrit/backend/models/attacks.py Outdated
@romanlutz

Copy link
Copy Markdown
Contributor

I exercised the score UI across single/multiple pieces, single/multiple scores, and wide/narrow viewports. Two visual issues remain:

  1. The score tab list should not scroll horizontally. In the wide single-piece/multi-score case, each tab includes the value, scorer name, and Objective label, which forces a scrollbar. Please show only the score value in each visible tab, move scorer/objective metadata into the selected score details panel, and retain that context in a tooltip/accessible name. Cap the visible tabs at three and use the existing More menu for additional scores; the tab list itself should never scroll.

Horizontal scrollbar in the multi-score tab list

  1. The stacked-score layers need uniform geometry. The back and middle layers currently use fixed asymmetric insets while the front chip is content-sized, so short values such as 0.91 produce uneven spacing. Please size every layer from the same chip box and use consistent diagonal offsets, for example (0,0), (3,3), and (6,6), so the spacing remains even for both short and long values.

Uneven spacing in the stacked score layers

Full screenshot matrix (1440 x 900 and 390 x 844)
Scenario Wide Narrow
Single piece, single score Wide single piece, single score Narrow single piece, single score
Single piece, multiple scores Wide single piece, multiple scores Narrow single piece, multiple scores
Multiple pieces, single score Wide multiple pieces, single score Narrow multiple pieces, single score
Multiple pieces, multiple scores Wide multiple pieces, multiple scores Narrow multiple pieces, multiple scores

@jbolor21

Copy link
Copy Markdown
Contributor Author

Roman Lutz (@romanlutz) implemented your suggestions but I don't love this tab spacing/look:
image

@jbolor21

Copy link
Copy Markdown
Contributor Author

Roman Lutz (Roman Lutz (@romanlutz)) implemented your suggestions but I don't love this tab spacing/look: image

okay here's a better version I'm happier with but lmk if that is ok with you

image

@romanlutz

Copy link
Copy Markdown
Contributor

Image doesn't load for me somehow. The tablist has the advantage that overflow is handled. I'll try options and will get back to you.

@romanlutz

Copy link
Copy Markdown
Contributor

When I select any score other than the first one the shown one changes to that. In the screenshot below, the top one was 0.91 but then I selected low and so that stays visible even when I close the score popup. That seems wrong?

image

@romanlutz

Copy link
Copy Markdown
Contributor

There's no visual indication that there are more than 3 scores. There's plenty more space, though. Maybe it should fill up the space and THEN use the overflow in the ellipsis? Probably depends on what fluent element you're using there. LMK if you need input.

image

size,
pieceId: piece.id,
metadata: piece.prompt_metadata || undefined,
scores: !isOriginal && sortedScores.length > 0 ? sortedScores : undefined,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A scored media piece loses all of its scores when converted_value is empty: pieceToAttachment returns null before reaching this assignment, getTextScores excludes media, and original attachments intentionally omit scores. Please retain a score-only piece row when converted media cannot render, and add an empty-media regression test.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The score itself is preserved now, but the fix introduces a second problem: a non-renderable media piece is represented as a real MessageAttachment with url: "", and that object is also added to message.attachments. Downstream code cannot tell that it exists only to carry scores.

I reproduced this through the API/render path by blanking both media values and URLs while retaining the score: the piece rendered 0 images and 1 score control, but the message still exposed a Download image… action. handleDownload then calls fetch(""), which resolves to the current page. The copy handlers also add this object to the composer; because it retains scores, the optimistic new message can display the previous message's scores and submit an empty media value. Exports likewise list a phantom attachment.

Can we keep score presentation separate from actionable attachments—for example, let the media display piece own scores and allow it to have no attachment—or otherwise ensure score-only placeholders never enter copy/download/export paths? Please add a component/action regression test, not only the mapper-shape test.

justifyContent: 'center',
minWidth: '30px',
minHeight: '20px',
padding: `0 ${tokens.spacingHorizontalXS}`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GHCP comment: score_type="unknown" allows arbitrary strings, but both the stacked value and single-score chip are unbounded. A long unbroken value produced a 435px button in a 390px viewport and was clipped. Please constrain and ellipsize the visible value while preserving the full value in the tooltip, accessible name, and details panel.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The collapsed chip is constrained now, but this is only partially fixed: the details panel still leaves score metadata cells unbounded. Using the real API/render path with a valid 200-character unknown score at a 320px viewport, I measured 701px document scroll width, 684px panel scroll width, and a 604px value inside a 206px cell.

scoreRow and the value Badge/Text elements have no min-width: 0 or wrapping/overflow rule, so long score values—and similarly long scorer names, source labels, or categories—escape the popover. The new tests verify that the full value remains available, but not that the panel stays within the viewport. Please constrain/wrap the detail value cells and add a geometry assertion such as scrollWidth <= clientWidth at a narrow viewport.

Comment thread frontend/src/utils/messageMapper.ts Outdated
Comment thread frontend/src/components/Chat/MessageList.tsx Outdated
':hover': {
backgroundColor: tokens.colorNeutralBackground2Hover,
},
'&[aria-selected="true"]': {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should override Fluent's tab appearance here. In light mode the selected dark-blue tab renders with black text, while the objective tab's separate border/background makes it look selected too:

Light-mode score tabs with conflicting active states

Can we remove the custom pill, selected, and objective-state styling and let TabList/Tab render their native selection state? Objective status can remain in the tooltip and details panel instead of introducing a second active-looking state.

@jbolor21 jbolor21 Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch! k changed & checked it works in both light & dark adding fluent borders bc it looks odd to me w no borders. but will totally remove the borders if there's a real ui issue w them

@jbolor21 jbolor21 Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dark mode: image

light mode:
image

@romanlutz

Copy link
Copy Markdown
Contributor

One backend coverage gap remains around the core objective-score provenance path. The service test verifies argument forwarding to a mocked mapper, while the response-contract tests call MessagePieceView.from_domain with a single score. Nothing directly exercises pyrit_messages_to_dto_async(..., objective_score_id=...) with one message containing multiple pieces and multiple persisted scores.

That means a regression which attaches scores to the wrong piece, marks every score objective, or fails to mark the matching score could still leave the current tests green. Please add a mapper-level test with text and media pieces plus several scores, and assert by score ID (not list position) that:

  • every score stays on its originating piece;
  • only the score whose ID matches objective_score_id has is_objective_score=true;
  • all scores are false when objective_score_id=None.

This is the backend contract the frontend's piece-local rendering now depends on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants