Skip to content

feat(ai): integrate AWS Comprehend PII redaction and observable chaining (#290) - #670

Open
tivince82 wants to merge 2 commits into
arakoodev:tsfrom
tivince82:feat/aws-comprehend-redaction
Open

tivince82 wants to merge 2 commits into
arakoodev:tsfrom
tivince82:feat/aws-comprehend-redaction

Conversation

@tivince82

@tivince82 tivince82 commented Sep 28, 2026 •

Copy link
Copy Markdown

/claim #290

Summary of Changes

This PR integrates Amazon Comprehend into the EdgeChains JavaScript/TypeScript SDK (@arakoodev/edgechains.js) as a chainable redaction utility to detect and sanitize PII before prompts reach LLM endpoints.

  1. AwsComprehendRedactor:

    • Wraps AWS SDK v3 @aws-sdk/client-comprehend (DetectPiiEntitiesCommand).
    • Supports dependency injection of the client for offline testing in CI/CD without AWS credentials.
    • Deterministic descending offset replacement to avoid index corruption.
    • Configurable masking modes (ENTITY_TYPE, REDACTED_LABEL, ASTERISK, CUSTOM).
    • Confidence thresholding (minScore) and entity allowlist/blocklists.
    • Fail-closed security mode by default to prevent leaking raw prompts if detection fails.
    • Fallback regex guards for critical identifiers (SSN, Email, Phone, Credit Cards).
  2. AwsComprehendRedactionChain & Reactive Observables:

    • Cold RxJS Observables (.redact, .redactPrompt, .chat).
    • Custom RxJS operators (redactPiiOperator(), redactChatOptionsOperator()).
    • Direct chaining with OpenAI, GeminiAI, and LlamaAI endpoints.
  3. Unit Tests:

    • Added 14 comprehensive unit tests in Vitest covering entity detection, masking formats, score filtering, Unicode handling, fail-closed behavior, and RxJS observable chaining.
  4. Example Demo:

    • Added JS/edgechains/examples/comprehend-redaction/ showcasing standalone redaction, observable streams, and endpoint chaining.

Demo Walkthrough

Demo Walkthrough


Verification

# 1. SDK Unit tests (14/14 passing):
cd JS/edgechains/arakoodev && npx vitest run src/ai/src/tests/awsComprehendRedaction.test.ts

# 2. Runnable example demo:
cd ../examples/comprehend-redaction && npm start

@github-actions

Copy link
Copy Markdown

CLA Assistant Lite bot: Thank you for your submission, we really appreciate it. Before we can accept your contribution, we ask that you sign the Arakoo Contributor License Agreement. You can sign the CLA by adding a new comment to this pull request and pasting exactly the following text.


I have read the Arakoo CLA Document and I hereby sign the CLA


absolutemadlad seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You can retrigger this bot by commenting recheck in this Pull Request

@tivince82

Copy link
Copy Markdown
Author

I have read the Arakoo CLA Document and I hereby sign the CLA

@tivince82

Copy link
Copy Markdown
Author

recheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant