Repository navigation
chore: port prompt injection rules - #10
Conversation
gewenyu99
left a comment
There was a problem hiding this comment.
This will for sure be useful. There are two things I want to discuss:
-
How will these be run? I certainly think these are fragile if anyone can run random strings against these, even if it's a black box. I feel like the run and the results of these cannot be public. The ability to observe inputs and patterns of what passes and doesn't is a big enough attack surface for regex matches like these.
-
How strict should these regex's be? I think all of these are great, but very easy to skirt around. And I'm not sure if that's a problem or not. (we cannot create all powerful regexes xD)
|
@gewenyu99 > This will for sure be useful. There are two things I want to discuss:
all very good questions!!
TL;DR of this thread: it runs in process inside the consumer as a dependency (wizard: in runs, context mill: in release builds) rule secrecy isn't really the strategy, defense in depth is. v2 of the security hardening introduces agentsh + an agent layer to make that defense in depth model deeper.
the regex rules are designed to be the prevention layer of the defense in depth design, not all powerful. these rules are part of the v1 foundation, and v2 will layer agentsh. with v2 in mind, my thought was:
|
ports existing wizard prompt injection rules into 8 focused sub-rules in the Warlock, each covering a
specific attack class
Changes
prompt_injection,action
block):instruction_override,role_hijack,jailbreak_persona,chat_markup(all critical)system_prompt_leak(high)posthog_integration_attack,posthog_feature_attack(medium)base64_in_comment(critical)rules.test.tstoone
.test.tsper rule under__tests__/rules/, plus a sharedhelpers.ts. Mirrors the one-rule-per-file pattern thatsrc/scanner/rules/already uses — makes finding a rule's teststrivial and keeps files from sprawling as we add more rules. The 3
PR-1 tests were migrated to the new layout (same assertions).
Test plan
pnpm testpasses (169 tests)pnpm buildpasses, all 11.yarfiles ship todist/CONTRIBUTING.md