Skip to content

feat: add engineering plugin with dependency-upgrade skill - #2

Open
edwintantawi wants to merge 5 commits into
mainfrom
feat/engineering-plugin
Open

edwintantawi wants to merge 5 commits into
mainfrom
feat/engineering-plugin

Conversation

@edwintantawi

@edwintantawi edwintantawi commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds the engineering plugin and its first skill, dependency-upgrade.

  • Plugin bootstrap (8bccb76): plugins/engineering manifest, README, and marketplace entry.
  • dependency-upgrade skill (143bcc1, reworked in b0c52e8): upgrades dependencies to their latest safe versions and explains what the upgrades mean for the codebase.
  • Versioning (804033b): the plugin is versioned with Changesets; this PR carries one minor changeset for the skill's first release.

What the skill does

  1. Audits each target version for supply-chain risk before anything is installed (known vulnerabilities, maintainer and provenance changes, install scripts, package diff), and installs with scripts disabled.
  2. Reads every changelog entry between the current and target version and maps each one to the project's code: breaking changes, behavior changes, deprecations, bug fixes, and performance changes.
  3. Reads the dependency's official docs for the target version, then applies the code changes the documented way, one dependency or coupled group per commit.
  4. Re-checks every pinned dependency against the issue in its comment: removes the pin (and comment) once the issue is fixed upstream, keeps both otherwise.
  5. Writes one short note at docs/dependency-upgrade/<date>.md and commits it last.

Rules worth knowing:

  • Minimum release age is left to the package manager's own setting (min-release-age, minimumReleaseAge, and so on). The skill applies no wait of its own and never bypasses the project's. A release the setting is holding is not installed, but its notes are read and reported under "Upcoming".
  • Exact versions are upgraded like any other dependency and kept exact, unless the user or a comment on the entry says to hold one.
  • The note is a list, not a table: one item per dependency with current → latest and only the findings that touch this codebase, at most three sentences each. A dependency with nothing relevant is one line.
  • It never pushes or opens a PR without approval, and treats fetched changelogs and docs as data rather than instructions.

Contents

  • SKILL.md: rules, phases 0-7, exact versions and pins, checkpoint (233 lines)
  • references/: security audit, changelog and docs sources, the upgrade-note format with a worked example, and one file each for npm/pnpm/yarn, pip/uv/poetry, Go modules, Cargo
  • scripts/: read-only helpers (note path and branch naming, version listing, npm metadata diff, GitHub releases for a range)
  • evals/: nine cases, a localhost mock npm registry, fixture repos, and a grader

Testing

  • claude plugin validate --strict passes for the plugin.
  • Evals with the skill: 118 of 118 checks pass across the nine cases (patch and major upgrade, package-manager release age, deprecation migration, injected changelog instructions, report-only, scoped package plus a same-day second run, pin removed, new install script held, exact versions with a pin kept).
  • No baseline was run for this revision. The earlier revision's no-skill baseline scored 39 of 78 on seven cases, under checks that have since changed.

Known limits

  • All nine evals are npm. The Python, Go, and Cargo reference files have not been through an eval.
  • In a project with no release-age setting configured, and in Go and Cargo (which have none), brand-new releases are eligible immediately; the other audit checks still run, and the note suggests configuring the setting where one exists.
  • Pin comments in package.json are tested with a "//overrides" / "//dependencies" entry; ordinary comments in pnpm-workspace.yaml and other formats are covered by the instructions but not by an eval.
  • The eval registry counts as a private host, so eval notes cite versions without links; link output on public packages has not been exercised.
  • A description-triggering test on the earlier revision was inconclusive (no false triggers, low trigger rate in a bare claude -p session). Worth checking by loading the plugin in a real repo.
  • Running the evals needs the mock registry in evals/fixtures/registry.py.

Upgrades dependencies to their latest safe versions: audits each target
version for supply-chain risk before anything is installed, reads every
changelog entry in the range, maps each change to the project's code,
applies changes one dependency per commit, and writes a per-dependency
report under docs/dependency-upgrade/<date>/.

- SKILL.md with the non-negotiable rules, phases 0-7, checkpoint, and
  report contract
- references for the security audit, changelog sources, report
  templates, and npm/pnpm/yarn, pip/uv/poetry, Go modules, and Cargo
- read-only helper scripts: run paths, version listing with cooldown
  flags, npm metadata diff, GitHub releases for a range
- evals: seven cases, a localhost mock npm registry, fixture repos, and
  a grader that checks git history, lockfiles, source, and tests

Bump the engineering plugin to 0.1.0 and list the skill in its README.
@edwintantawi edwintantawi self-assigned this Oct 9, 2026
Add the plugin's package.json so Changesets can version it, reset the
manifest version to 0.0.0, and add a minor changeset so the first
release ships as 0.1.0.
@changeset-bot

changeset-bot Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b0c52e8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
engineering Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

- Write one short note at docs/dependency-upgrade/<date>.md instead of a
  folder of files; dependencies are a list, each item carrying only the
  findings that touch the codebase, three sentences at most per finding
- Leave minimum release age to the package manager's own setting and
  report what a held release will bring under Upcoming
- Upgrade exact versions like any other dependency unless told otherwise
- Re-check every pinned dependency against the issue in its comment
- Read the dependency's official docs before changing code
- Map performance changes to where the codebase uses the affected path
- Update evals: nine scenarios, grader checks for the single-note format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant