Skip to content

feat(ai): invoke a project skill with / in the analyst chat - #9892

Open
dfliess wants to merge 2 commits into
rilldata:mainfrom
dfliess:dfliess/chat-skill-slash-command
Open

dfliess wants to merge 2 commits into
rilldata:mainfrom
dfliess:dfliess/chat-skill-slash-command

Conversation

@dfliess

@dfliess dfliess commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Hey @nishantmonu51,

We were working on a similar feature, but your approach in #9851 is better integrated with the developer agent, so we'd like to contribute this piece on top of it: letting the user explicitly invoke a Skill from the chat. Feel free to modify, adapt or discard it, or suggest changes.

  • Typing / in an analyst chat opens a picker with the project's Skills whose agents include analyst. A / button next to @ opens it too. / is a second suggestion on the same Mention node as @, so both share serialization.
  • Picking a skill writes <chat-reference>type="skill" skill="…"</chat-reference> into the prompt, the same tag @ already writes for other reference kinds, so it renders as a chip while typing and when the conversation is reloaded.
  • AnalystAgent scans each prompt for skill references and calls load_skill once per referenced skill before the model's turn, skipping unknown skills, skills that don't apply to the analyst, and skills already loaded in the conversation.
  • Why a tag plus a runtime load instead of splicing the skill body into the prompt: the user's text stays separate from the skill author's text, and resolution goes through the same load_skill path the model already uses, instead of depending on the model to honor the reference.
  • This makes the runtime read a tag the chat UI writes, which it didn't before (referencedSkills in runtime/ai/skill_references.go). We kept the reference in the prompt because that is what gets persisted and re-rendered as a chip, but if you'd rather have it as an explicit field on AnalystAgentContext, or want <chat-reference> specified as a shared format with the Go parser as its reference implementation, we're happy to change it.

Developed in collaboration with Claude Code

dfliess and others added 2 commits September 16, 2026 06:38
On every turn, the analyst detects skill references in the prompt
(<chat-reference>type="skill" skill="..."</chat-reference>) and
pre-invokes load_skill once per distinct referenced skill that exists
and applies to the analyst, before the model's turn. Unknown skills and
skills for other agents are ignored; always-apply skills already
pre-loaded on the first turn aren't loaded again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Typing "/" in an analyst chat (project, dashboard, and embed) opens a
picker listing the project's Skill resources that apply to the analyst,
showing name and description. Picking one writes a
<chat-reference>type="skill" skill="..."</chat-reference> tag into the
prompt text -- the same tag the runtime resolves via load_skill -- and it
renders as a chip both while typing and when the conversation is
reloaded. A dedicated / button next to the existing @ button opens the
same picker, adding a space first when the cursor sits right after a
word.

The "/" trigger is a second suggestion on the same Mention node as "@",
so both share the parsing and serialization of chat-reference tags, the
two pickers never open at once, and both follow Suggestion's default
prefixes: a slash inside a word, as in a date or a path, stays text. The
context picker takes the source of its options and whether several are
picked, so the skills picker reuses it as a single-choice list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@nishantmonu51 nishantmonu51 added Type:Feature New feature request Size:L Large change: 500-1,999 lines labels Sep 16, 2026
@nishantmonu51
nishantmonu51 self-requested a review September 17, 2026 08:57

@nishantmonu51 nishantmonu51 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Traced the tag from the Mention node through convertContextToInlinePrompt to referencedSkills and the seeded load_skill call; that path is correct, and the new Go and frontend tests pass on this head. One edge case in the "already loaded" check is worth fixing before merge, plus a smaller UX note.

On the open question in the description (tag-in-prompt vs. an explicit AnalystAgentContext field): the current approach holds up, since the same prompt text is what gets persisted and re-rendered, and referencedSkills filters to existing analyst skills so the runtime never trusts the tag beyond a name lookup. It does make chat-reference a contract between inline-context.ts and skill_references.go; the Go parser accepts attributes in any order and the frontend always emits type first, so the two agree today.

Comment on lines +40 to +47
for _, msg := range s.Messages(FilterByType(MessageTypeCall), FilterByTool(LoadSkillName)) {
content, err := s.UnmarshalMessageContent(msg)
if err != nil {
continue
}
if args, ok := content.(*LoadSkillArgs); ok {
res[args.Name] = true
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loadedSkills counts every load_skill call in the session without looking at its result, so a call whose handler errored still marks the skill as loaded (the result message is added under the call with ContentType == MessageContentTypeError, runtime/ai/ai.go:1054). Concretely, while authoring a skill in Rill Developer: the user writes "use the monthly-close skill" as plain text while skills/monthly-close/SKILL.md still has a reconcile error, the model calls load_skill("monthly-close") and gets "not found"; the user fixes the file and now picks /monthly-close from the picker. referencedSkills returns the skill, but loaded["monthly-close"] is true, so the loop at analyst_agent.go:187-191 skips it and the model never receives the body for the rest of the conversation — the chip is a silent no-op. The same happens after a transient Skills() failure. Only calls with a non-error result should count; the result is the call's child, s.Messages(FilterByParent(call.ID), FilterByType(MessageTypeResult)).

extensions: getEditorPlugins({
placeholder,
onSubmit: () => void sendMessage(),
skillOptions: skillsEnabled ? getSkillsPickerOptions : undefined,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The typed / trigger is registered whenever config.skills is set, but the / button below is gated on hasSkills. In a project with no analyst skills, or before the resources query resolves, typing / at the start of a line or after a space still opens the picker showing "No matches found", and while it is open EditorSubmitExtension swallows Enter on contextOpen (editor-plugins.svelte.ts:265-274), so "compute a /" followed by Enter does nothing until the user types a space or deletes the slash. The @ picker has the same shape of behaviour today, but @ is a much less common character in analysis prompts than /. Gating skillOptions on a non-empty store is not straightforward since the extensions are built once in onMount, so this may be acceptable as-is.

@nishantmonu51 nishantmonu51 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, AI review flagged mostly nitpick edge cases, and manual review looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Size:L Large change: 500-1,999 lines Type:Feature New feature request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants