Repository navigation
Conversation
| @@ -0,0 +1,4 @@ | |||
| interface: | |||
| display_name: "Skill Scanner" | |||
There was a problem hiding this comment.
I would prefer to have it as SkillSpector itself.
| @@ -0,0 +1,90 @@ | |||
| --- | |||
| name: skill-scanner | |||
There was a problem hiding this comment.
We should have it SkillSpector itself IMHO. The description anyways highlights it as a skill scanner
|
@edjshen - Please address the review comments. |
rng1995
left a comment
There was a problem hiding this comment.
Requesting changes for the recursive workflow bug below. The existing metadata naming threads also remain unresolved (display name, skill name), and the current head still fails the DCO check.
| 5. Report the scan mode, score, severity, recommendation, highest-impact findings with | ||
| file/line evidence, execution completeness, and material limitations. | ||
|
|
||
| Use `--recursive` when the target contains multiple immediate subdirectories that each |
There was a problem hiding this comment.
[P2] Make the recursive workflow produce the JSON that step 4 requires. Adding --recursive to the canonical command without --output does not emit combined JSON: _scan_multi_skill prints human summaries unless an output path is supplied. An agent following these instructions therefore has no execution_successful/completeness fields to inspect. Require --output <path> for recursive scans and instruct the agent to read that file, or change the launcher to honor --format json on stdout.
|
Powered by Codex: PR council review result. This is a triage signal, not a maintainer approval.
|
|
@edjshen - Please address my review comments, resolve merge conflicts and ping me for approval |
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-review: still requesting changes. The current head only merges upstream changes and leaves every requested PR-owned edit unchanged: the display/skill names remain Skill Scanner/skill-scanner, recursive JSON guidance still makes --output optional even though no JSON contract is emitted without it, and DCO still fails for 7d7b985 and ae7d8a2. In an exact-head reproduction, recursive JSON with no output file printed only the summary table and contained neither execution_successful nor analysis_completeness. Please address the three open threads and restore green DCO.
yashrajp22
left a comment
There was a problem hiding this comment.
Requesting changes for the static-mode privacy guarantee: enabled tracing can transmit scanned file contents even with --no-llm. The new finding is inline; existing review threads are not duplicated.
Verification covered 147 selected tests, 48 source/wheel corpus reports and four actual launcher cases, including concurrent outputs and Unicode paths. Testing was offline on Python 3.12; partial corpus coverage and finding-ID differences remain recorded. Live providers and cold network installation were not tested.
|
|
||
| ## Choose static or semantic analysis | ||
|
|
||
| Default to `--no-llm`. Static mode keeps file contents local, is fast, and does not require |
There was a problem hiding this comment.
Could we disable inherited tracing for static scans, or make the tracing disclosure and consent requirement explicit here? When LangSmith tracing is enabled, --no-llm still allows scanned file contents in graph state to be sent to the configured tracing endpoint. I reproduced this on the installed PR package using a synthetic loopback receiver and flushing queued traces. The local-only guarantee can therefore lead an agent to transmit contents without the approval required below.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
@edjshen thank you for packaging SkillSpector as an agent skill with a locked, production-only launcher, and sorry this has sat for so long.
Value and readiness
Still not ready. Main has resolved one earlier blocker, but the naming, privacy-claim and DCO items are unchanged. There has been no author activity on the PR since it was opened on Aug 13.
Material findings
Prior findings
- Display name
Skill Scanner(agents/openai.yaml:2): still open (maintainer asked for SkillSpector). - Skill name
skill-scanner(SKILL.md:2, and$skill-scannerindefault_prompt): still open. - Recursive JSON requires
--output(SKILL.md:30): resolved by main. #576 (fcfbcd8b, Sep 21) makes_scan_multi_skillwrite the combined JSON to stdout when--outputis absent, with progress on stderr. Step 4's fields will therefore be present after merge, and this thread can be resolved. - DCO: still open.
7d7b985andae7d8a2have noSigned-off-by. The two update-branch merges are exempt underci.yml's rule. - yashrajp22's tracing finding (
SKILL.md:37, "Static mode keeps file contents local"): still open. On main,_build_trace_configstill applies to--no-llmscans, and nothing disables inherited LangSmith tracing.
New (optional)
SKILL.md:46cites a README section Security Considerations that does not exist on main. The closest section is "Trust model and data egress".scripts/scan_skill.pylacks the SPDX header that the otherscripts/files carry.
PIC tradeoffs
- Main already ships
skills/skill-inspector/SKILL.md(#253, Aug 12), an agent skill that runsskillspector scan --no-llm --format jsonand then reviews the source. This PR adds a second, root-level skill that turns the whole repository into a skill bundle. The PIC should decide whether both are wanted. If the author does not return, closing this in favor of skill-inspector is reasonable. - The README makes the same "
--no-llmkeeps contents local" claim. Disabling tracing for--no-llmin the product may therefore be a better fix than a caveat in SKILL.md.
Verification and gaps
- Read the 142-line diff at
da2b83d7and all rng1995 and yashrajp22 reviews and threads. Commits afterae7d8a2are update merges only. git merge-treeagainst530d3c5cis clean.- Compared
_scan_multi_skillat the PR head with main, checked_build_trace_configand the README headings on main, and checked sign-offs against the DCO rule inci.yml. - Last CI ran Aug 26 (DCO failed). Nothing was executed.
What changed
skill-scannerskillSKILL.md, OpenAI UI metadata, and a portable launcheruv --frozen --no-dev--no-llminspection and preserves scanner exit codesWhy
This makes the existing SkillSpector engine directly usable as an agent skill while keeping target skills untrusted and avoiding execution of their scripts or dependencies.
Validation
quick_validate.py: valid--help: successful against SkillSpector 2.9.4mainat SkillSpector 2.9.4Static
--no-llmmode intentionally reports disabled semantic analyzers as a limitation; the launcher does not hide that completeness signal.