diff --git a/CONTEXT.md b/CONTEXT.md index 9029b59..9a87323 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -10,14 +10,14 @@ _Avoid_: Crawler, inspector, walker **Detector**: The rule evaluator that determines if a file path is an environment file and whether it matches safe allowlist templates. -_Avoid_: Matcher, filter, classifier +_Avoid_: Matcher, filter, classifier. Not to be confused with **Secret Scanner**, which inspects file _content_ rather than file _paths_. **Configuration**: The project-level settings loaded from `.envguard.yaml`, `.envguard.yml`, or via `--config` to customize scanning and detection rules. _Avoid_: Options, settings, preferences **Allowlist**: -The collection of glob patterns for safe environment templates or sample files (e.g., `.env.example`) that should not raise security warnings. +The collection of glob patterns for safe environment templates or sample files (e.g., `.env.example`) that should not raise security warnings. Files matching the Allowlist are also skipped entirely by the **Secret Scanner**. _Avoid_: Whitelist, safe-list, permitted files **Ignore Directory**: @@ -25,7 +25,7 @@ A directory name skipped during recursive filesystem traversal (e.g., `node_modu _Avoid_: Excluded path, blacklisted folder **Severity Override**: -An explicit configuration rule that replaces the calculated severity level for files matching a specific pattern. +An explicit configuration rule that replaces the calculated severity level for files matching a specific pattern. When set on a file, it also acts as a ceiling on that file's Secret Match severities (a Secret Match's severity is `min(computed severity, file's overridden severity)`). _Avoid_: Custom rule, priority tweak **Finding**: @@ -41,7 +41,7 @@ The process of stripping secret values from environment definitions while preser _Avoid_: Masking, redacting, cleaning **Fixer**: -The remediation component responsible for automatically generating and applying non-destructive `.gitignore` rules for unignored environment files. +The remediation component responsible for automatically generating and applying non-destructive `.gitignore` rules for unignored environment files. Scoped strictly to `.gitignore` edits; it never invokes the Secret Scanner and has no awareness of Secret Matches. _Avoid_: Patcher, autofixer, corrector, remediator **Hook Manager**: @@ -51,3 +51,11 @@ _Avoid_: Hook installer, hook script, hook handler **Hook Runner**: The fast execution mode invoked during Git pre-commit lifecycle to inspect staged repository files and prevent accidental commits of environment secrets. _Avoid_: Commit watcher, stage scanner, commit blocker + +**Secret Scanner**: +The content-inspection component that examines the text inside environment files for embedded secrets, using known-provider signature patterns (AWS, Stripe, GitHub, etc.) and entropy-based heuristics as independent checks. Skips any file covered by the Allowlist and any file that fails a binary/size check. +_Avoid_: Detector, content matcher, secret detector + +**Secret Match**: +A single detected secret instance found by the Secret Scanner within a Finding's file: line number, key name (if identifiable), detection method (pattern or entropy), provider (if pattern-matched), and its own severity, independent of the parent Finding's severity. +_Avoid_: Secret finding, leak, hit diff --git a/docs/adr/0005-secret-scanning-scope-and-defaults.md b/docs/adr/0005-secret-scanning-scope-and-defaults.md new file mode 100644 index 0000000..10fafa3 --- /dev/null +++ b/docs/adr/0005-secret-scanning-scope-and-defaults.md @@ -0,0 +1,32 @@ +# 0005: Secret Scanning Scope, Detection Strategy, and Defaults + +To implement the v0.3.0 roadmap goal of content-based secret scanning, `envguard` introduces a **Secret Scanner** that inspects the text inside environment files for embedded secrets, producing **Secret Match** records distinct from the existing **Finding** concept. + +## Context & Problem + +Filename/path-level detection (the existing `Detector`) tells you a file _might_ contain secrets, but not whether it actually does. The roadmap calls for two capabilities: known-provider signature matching (AWS, Stripe, GitHub, etc.) and entropy-based heuristics for unrecognized secret formats. These have very different false-positive profiles, and the feature needed to be scoped without breaking the `hook run` `<10ms` performance budget or the "never print secret values" design principle. + +## Decision + +1. **Scope**: The Secret Scanner only inspects files already classified as environment files (the `Detector`'s existing territory). It does not scan arbitrary repository files (source code, general config) in v0.3.0 — that's a larger surface with a different false-positive character, deferred to a later version. + +2. **Detection strategy**: Pattern matching (known-provider signatures) and entropy calculation are independent checks, both able to produce `Secret Match` records on the same file. Pattern matching ships **on by default**; entropy-based detection is **opt-in** via `detector.entropy_scan`, since its higher false-positive rate risks alert fatigue and broken CI before the heuristic is tuned. + +3. **Command boundaries**: The Secret Scanner runs in `scan` and `check` only. It is excluded from `hook run` to preserve its `<10ms` budget, and `fix` remains completely unaware of it — `fix` is scoped strictly to `.gitignore` edits and has no remediation path for a real secret (rotation isn't automatable). + +4. **Secret Match is a distinct concept from Finding**, not a merge into it: a `Finding` (env file, severity from git tracked/staged status) can contain zero or more `Secret Match` records, each with its own severity floor (`HIGH` when untracked, `CRITICAL` when tracked/staged) — independent of the parent `Finding`'s severity. `check`'s exit-code aggregation takes the max severity across a file's `Finding` and its `Secret Match` records. + +5. **Existing features extend to cover it**: files on the `Allowlist` (e.g. `.env.example`) are skipped entirely by the Secret Scanner. A file's `Severity Override` (e.g. `.env.test` → `warning`) acts as a ceiling on that file's `Secret Match` severities, so intentionally fake secrets in test fixtures don't trigger `CRITICAL`. A separate `detector.secret_ignore` list suppresses specific false-positive matches by key name/pattern regardless of file. + +6. **Output stays zero-value-exposure**: a `Secret Match` is reported as key name (if identifiable) + file + line number only — never a value fragment, consistent with envguard's existing guarantee. + +## Considered Options + +- **Scanning all repository files, not just env files** — rejected for v0.3.0: much larger surface, higher false-positive risk from code that legitimately references key _names_, and a bigger traversal/performance question. Left as a future expansion. +- **Entropy detection on by default alongside pattern matching** — rejected: shipping a high-false-positive heuristic on-by-default in a security tool risks alert fatigue and broken CI runs before it's proven in the wild. +- **Folding Secret Match into the existing Finding object** — rejected: a `Finding`'s severity is driven by git exposure status, while a `Secret Match`'s severity is driven by the secret itself; conflating them would have made the severity model harder to reason about and prevented the `Severity Override` ceiling from applying cleanly. + +## Consequences + +- Widening scope later (e.g. scanning all repository files) will require revisiting the traversal and ignore-directory logic that today is scoped to already-detected env files. +- Enabling entropy detection by default in a future release is a config default change, not an architectural one — low cost to reverse. diff --git a/internal/cli/check.go b/internal/cli/check.go index 293c0f8..5a40c41 100644 --- a/internal/cli/check.go +++ b/internal/cli/check.go @@ -72,7 +72,7 @@ func runCheckCommand(args []string, stdout, stderr io.Writer, scannerInstance *s minRank := severityRank(minSev) filtered := make([]scanner.Finding, 0, len(result.Findings)) for _, f := range result.Findings { - if severityRank(f.Severity) >= minRank { + if severityRank(f.EffectiveSeverity()) >= minRank { filtered = append(filtered, f) } } diff --git a/internal/cli/scan.go b/internal/cli/scan.go index 9018035..ad370a6 100644 --- a/internal/cli/scan.go +++ b/internal/cli/scan.go @@ -115,7 +115,7 @@ func runScanCommand(args []string, stdout, stderr io.Writer, scannerInstance *sc minRank := severityRank(minSev) filtered := make([]scanner.Finding, 0, len(result.Findings)) for _, f := range result.Findings { - if severityRank(f.Severity) >= minRank { + if severityRank(f.EffectiveSeverity()) >= minRank { filtered = append(filtered, f) } } diff --git a/internal/config/config.go b/internal/config/config.go index 0ae18eb..4284e79 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -7,6 +7,7 @@ import ( "path/filepath" "strings" + "github.com/joaooncode/envguard/internal/secretscanner" "gopkg.in/yaml.v3" ) @@ -33,6 +34,12 @@ type DetectorConfig struct { CustomPatterns []string `yaml:"custom_patterns,omitempty"` Allowlist []string `yaml:"allowlist,omitempty"` SeverityOverrides []SeverityOverride `yaml:"severity_overrides,omitempty"` + // EntropyScan enables the Secret Scanner's entropy-based heuristic (opt-in; off by default). + EntropyScan bool `yaml:"entropy_scan,omitempty"` + // SecretProviders restricts the Secret Scanner's pattern matching to these providers. Empty means all shipped providers. + SecretProviders []string `yaml:"secret_providers,omitempty"` + // SecretIgnore suppresses Secret Matches whose key equals or glob-matches one of these entries. + SecretIgnore []string `yaml:"secret_ignore,omitempty"` } // SeverityOverride overrides the calculated severity for files matching a pattern. @@ -52,6 +59,8 @@ func NewDefault() *Config { CustomPatterns: make([]string, 0), Allowlist: make([]string, 0), SeverityOverrides: make([]SeverityOverride, 0), + SecretProviders: make([]string, 0), + SecretIgnore: make([]string, 0), }, } } @@ -130,5 +139,12 @@ func (c *Config) Validate() error { return fmt.Errorf("severity_overrides[%d]: invalid severity %q (supported: info, warning, high, critical)", i, override.Severity) } } + + for i, provider := range c.Detector.SecretProviders { + if !secretscanner.IsValidProvider(provider) { + return fmt.Errorf("detector.secret_providers[%d]: unknown provider %q (supported: %s)", i, provider, strings.Join(secretscanner.ValidProviders(), ", ")) + } + } + return nil } diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 095c59b..b41cf19 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -60,6 +60,45 @@ detector: } } +func TestParseYAML_SecretScanning(t *testing.T) { + yamlContent := ` +detector: + entropy_scan: true + secret_providers: + - aws + - stripe + secret_ignore: + - "TEST_*" +` + cfg, err := Parse([]byte(yamlContent)) + if err != nil { + t.Fatalf("unexpected error parsing valid yaml: %v", err) + } + + if !cfg.Detector.EntropyScan { + t.Errorf("expected EntropyScan = true") + } + if len(cfg.Detector.SecretProviders) != 2 { + t.Errorf("expected 2 secret providers, got %d", len(cfg.Detector.SecretProviders)) + } + if len(cfg.Detector.SecretIgnore) != 1 || cfg.Detector.SecretIgnore[0] != "TEST_*" { + t.Errorf("expected secret_ignore to contain 'TEST_*', got %v", cfg.Detector.SecretIgnore) + } +} + +func TestParseYAML_InvalidSecretProvider(t *testing.T) { + invalidProviderYAML := ` +detector: + secret_providers: + - aws + - awss +` + _, err := Parse([]byte(invalidProviderYAML)) + if err == nil { + t.Fatal("expected error on unknown secret_providers entry, got nil") + } +} + func TestParseYAML_InvalidSyntax(t *testing.T) { invalidYAML := ` scanner: diff --git a/internal/reporter/json.go b/internal/reporter/json.go index a704227..30271d7 100644 --- a/internal/reporter/json.go +++ b/internal/reporter/json.go @@ -58,12 +58,13 @@ func (j *JSONReporter) Render(result *scanner.Result, w io.Writer) error { sanitizedFindings := make([]scanner.Finding, len(findings)) for i, f := range findings { sanitizedFindings[i] = scanner.Finding{ - Path: f.Path, - Severity: f.Severity, - Message: f.Message, - Suggestions: f.Suggestions, - GitStatus: f.GitStatus, - IsAllowed: f.IsAllowed, + Path: f.Path, + Severity: f.Severity, + Message: f.Message, + Suggestions: f.Suggestions, + GitStatus: f.GitStatus, + IsAllowed: f.IsAllowed, + SecretMatches: f.SecretMatches, } } diff --git a/internal/reporter/reporter_test.go b/internal/reporter/reporter_test.go index e394428..e730fb7 100644 --- a/internal/reporter/reporter_test.go +++ b/internal/reporter/reporter_test.go @@ -22,6 +22,9 @@ func sampleResult() *scanner.Result { Severity: scanner.SeverityCritical, Message: "Environment file is tracked by Git (committed in repository history).", Suggestions: []string{"Remove file from git tracking: git rm --cached .env", "Add to .gitignore", "Rotate any leaked credentials"}, + SecretMatches: []scanner.SecretMatch{ + {Line: 3, Key: "AWS_ACCESS_KEY_ID", Method: "pattern", Provider: "aws", Severity: scanner.SeverityCritical}, + }, }, { Path: ".env.local", @@ -180,6 +183,14 @@ func TestTerminalReporter_Render(t *testing.T) { if !strings.Contains(out, "FAILED") { t.Errorf("expected FAILED status in summary, got:\n%s", out) } + + // Verify nested secret match line, and that no value is ever printed + if !strings.Contains(out, "AWS_ACCESS_KEY_ID") || !strings.Contains(out, "line 3") { + t.Errorf("expected nested secret match with key and line number, got:\n%s", out) + } + if strings.Contains(out, "AKIA") { + t.Errorf("secret value must never be printed, got:\n%s", out) + } }) t.Run("Render with ANSI colors enabled", func(t *testing.T) { @@ -269,6 +280,18 @@ func TestJSONReporter_Render(t *testing.T) { t.Errorf("unexpected first finding: %+v", report.Findings[0]) } + if len(report.Findings[0].SecretMatches) != 1 { + t.Fatalf("expected 1 secret match on first finding, got %d: %+v", len(report.Findings[0].SecretMatches), report.Findings[0].SecretMatches) + } + sm := report.Findings[0].SecretMatches[0] + if sm.Line != 3 || sm.Key != "AWS_ACCESS_KEY_ID" || sm.Provider != "aws" || sm.Severity != scanner.SeverityCritical { + t.Errorf("unexpected secret match: %+v", sm) + } + + if len(report.Findings[1].SecretMatches) != 0 { + t.Errorf("expected 0 secret matches on second finding, got %+v", report.Findings[1].SecretMatches) + } + if report.Summary.Critical != 1 || report.Summary.High != 1 || report.Summary.Warning != 1 || report.Summary.Info != 1 { t.Errorf("unexpected summary counts: %+v", report.Summary) } diff --git a/internal/reporter/terminal.go b/internal/reporter/terminal.go index 96e690c..1d5ff76 100644 --- a/internal/reporter/terminal.go +++ b/internal/reporter/terminal.go @@ -113,6 +113,13 @@ func (t *TerminalReporter) Render(result *scanner.Result, w io.Writer) error { sb.WriteString(fmt.Sprintf(" • %s\n", sug)) } } + if len(f.SecretMatches) > 0 { + sb.WriteString(fmt.Sprintf(" %s\n", t.c.cyan("Secrets:"))) + for _, sm := range f.SecretMatches { + _, badge := t.severityBadge(sm.Severity) + sb.WriteString(fmt.Sprintf(" • %s %s at line %d\n", badge, t.secretLabel(sm), sm.Line)) + } + } sb.WriteString("\n") } } @@ -138,6 +145,18 @@ func (t *TerminalReporter) Render(result *scanner.Result, w io.Writer) error { return err } +// secretLabel formats a SecretMatch's key/provider without ever including its value. +func (t *TerminalReporter) secretLabel(sm scanner.SecretMatch) string { + label := sm.Key + if label == "" { + label = "(unnamed key)" + } + if sm.Provider != "" { + label = fmt.Sprintf("%s (%s)", label, sm.Provider) + } + return label +} + func (t *TerminalReporter) severityBadge(sev scanner.Severity) (icon string, badge string) { switch sev { case scanner.SeverityCritical: diff --git a/internal/scanner/finding.go b/internal/scanner/finding.go index 680c100..2493861 100644 --- a/internal/scanner/finding.go +++ b/internal/scanner/finding.go @@ -20,14 +20,25 @@ const ( SeverityInfo Severity = "info" ) +// SecretMatch represents a single detected secret instance within a Finding's file. +// Its Severity is independent of the parent Finding's severity. +type SecretMatch struct { + Line int `json:"line"` + Key string `json:"key,omitempty"` + Method string `json:"method"` + Provider string `json:"provider,omitempty"` + Severity Severity `json:"severity"` +} + // Finding represents a detected environment file and its Git security posture. type Finding struct { - Path string `json:"path"` - Severity Severity `json:"severity"` - Message string `json:"message"` - Suggestions []string `json:"suggestions,omitempty"` - GitStatus git.FileStatus `json:"git_status"` - IsAllowed bool `json:"is_allowed"` + Path string `json:"path"` + Severity Severity `json:"severity"` + Message string `json:"message"` + Suggestions []string `json:"suggestions,omitempty"` + GitStatus git.FileStatus `json:"git_status"` + IsAllowed bool `json:"is_allowed"` + SecretMatches []SecretMatch `json:"secret_matches,omitempty"` } // Summary aggregates finding counts categorized by severity. @@ -49,11 +60,26 @@ type Result struct { Summary Summary `json:"summary"` } +// EffectiveSeverity returns the maximum severity between the Finding itself and any +// of its SecretMatches. Per ADR 0005 item 4, a Secret Match's severity is independent +// of its parent Finding's severity, and exit-code aggregation must take the max of +// the two — otherwise a real secret in a properly ignored file (Finding severity +// Info) would be silently dropped from the summary. +func (f Finding) EffectiveSeverity() Severity { + max := f.Severity + for _, m := range f.SecretMatches { + if severityRank(m.Severity) > severityRank(max) { + max = m.Severity + } + } + return max +} + // CalculateSummary computes metrics for a slice of findings. func CalculateSummary(findings []Finding) Summary { var s Summary for _, f := range findings { - switch f.Severity { + switch f.EffectiveSeverity() { case SeverityCritical: s.Critical++ case SeverityHigh: diff --git a/internal/scanner/scanner.go b/internal/scanner/scanner.go index ed090da..b526ce0 100644 --- a/internal/scanner/scanner.go +++ b/internal/scanner/scanner.go @@ -1,6 +1,7 @@ package scanner import ( + "bytes" "fmt" "os" "path/filepath" @@ -9,6 +10,7 @@ import ( "github.com/joaooncode/envguard/internal/config" "github.com/joaooncode/envguard/internal/detector" "github.com/joaooncode/envguard/internal/git" + "github.com/joaooncode/envguard/internal/secretscanner" ) // IgnoredDirectories contains the directory names skipped during recursive scanning. @@ -24,10 +26,11 @@ var IgnoredDirectories = map[string]bool{ // Scanner coordinates filesystem traversal, environment detection, and Git status inspection. type Scanner struct { - gitClient git.Client - detector *detector.Detector - cfg *config.Config - ignoreDirs map[string]bool + gitClient git.Client + detector *detector.Detector + cfg *config.Config + ignoreDirs map[string]bool + secretScanner *secretscanner.Scanner } // New creates a new Scanner instance with the provided git client and detector. @@ -60,6 +63,11 @@ func NewWithConfig(gitClient git.Client, det *detector.Detector, cfg *config.Con detector: det, cfg: cfg, ignoreDirs: ignoreMap, + secretScanner: secretscanner.NewWithOptions(secretscanner.Options{ + EntropyScan: cfg.Detector.EntropyScan, + Providers: cfg.Detector.SecretProviders, + Ignore: cfg.Detector.SecretIgnore, + }), } } @@ -128,6 +136,13 @@ func (s *Scanner) Scan(dir string) (*Result, error) { } finding := s.classifyFinding(relPath, status, isAllowed) + if !isAllowed { + secretMatches, err := s.scanSecrets(absDir, relPath, status) + if err != nil { + return err + } + finding.SecretMatches = secretMatches + } result.Findings = append(result.Findings, finding) return nil }) @@ -178,6 +193,22 @@ func (s *Scanner) classifyFinding(relPath string, status git.FileStatus, isAllow } // Check if any severity override matches this file path or base name + if overrideSeverity, ok := s.matchSeverityOverride(relPath); ok { + severity = overrideSeverity + } + + return Finding{ + Path: relPath, + Severity: severity, + Message: message, + Suggestions: suggestions, + GitStatus: status, + IsAllowed: isAllowed, + } +} + +// matchSeverityOverride returns the configured severity override for relPath, if any. +func (s *Scanner) matchSeverityOverride(relPath string) (Severity, bool) { baseName := filepath.Base(relPath) for _, override := range s.cfg.Detector.SeverityOverrides { patternLower := strings.ToLower(override.Pattern) @@ -193,31 +224,110 @@ func (s *Scanner) classifyFinding(relPath string, status git.FileStatus, isAllow matched = true } - if matched { - switch strings.ToLower(override.Severity) { - case "info": - severity = SeverityInfo - case "warning", "warn": - severity = SeverityWarning - case "high": - severity = SeverityHigh - case "critical": - severity = SeverityCritical - } - break + if !matched { + continue + } + + // Stop at the first pattern match regardless of whether its severity is + // recognized, mirroring the original inline loop's unconditional break. + // An unrecognized severity here means "no override" rather than "try + // the next override" - callers programmatically building a Config + // without Validate() can otherwise see a later, unintended override win. + switch strings.ToLower(override.Severity) { + case "info": + return SeverityInfo, true + case "warning", "warn": + return SeverityWarning, true + case "high": + return SeverityHigh, true + case "critical": + return SeverityCritical, true + default: + return "", false } } + return "", false +} - return Finding{ - Path: relPath, - Severity: severity, - Message: message, - Suggestions: suggestions, - GitStatus: status, - IsAllowed: isAllowed, +// severityRank orders severities from least (0) to most (3) severe. +func severityRank(sev Severity) int { + switch sev { + case SeverityInfo: + return 0 + case SeverityWarning: + return 1 + case SeverityHigh: + return 2 + case SeverityCritical: + return 3 + default: + return 0 } } +// scanSecrets inspects relPath's content for embedded secrets and returns the +// resulting SecretMatches, with severity floored by git status and capped by +// any configured Severity Override for the file. +func (s *Scanner) scanSecrets(absDir, relPath string, status git.FileStatus) ([]SecretMatch, error) { + fullPath := filepath.Join(absDir, relPath) + + content, ok := readScannableContent(fullPath) + if !ok { + return nil, nil + } + + rawMatches, err := s.secretScanner.Scan(content) + if err != nil { + return nil, fmt.Errorf("secret scan failed for %s: %w", relPath, err) + } + if len(rawMatches) == 0 { + return nil, nil + } + + floor := SeverityHigh + if status.IsTracked || status.IsStaged { + floor = SeverityCritical + } + + if override, ok := s.matchSeverityOverride(relPath); ok && severityRank(override) < severityRank(floor) { + floor = override + } + + matches := make([]SecretMatch, 0, len(rawMatches)) + for _, rm := range rawMatches { + matches = append(matches, SecretMatch{ + Line: rm.Line, + Key: rm.Key, + Method: string(rm.Method), + Provider: rm.Provider, + Severity: floor, + }) + } + return matches, nil +} + +const maxScannableFileSize = 1 << 20 // 1MB + +// readScannableContent reads fullPath's content, returning ok=false for files +// that don't exist, exceed the size cap, or look binary (contain a null byte). +func readScannableContent(fullPath string) ([]byte, bool) { + info, err := os.Stat(fullPath) + if err != nil || info.Size() > maxScannableFileSize { + return nil, false + } + + content, err := os.ReadFile(fullPath) + if err != nil { + return nil, false + } + + if bytes.IndexByte(content, 0) != -1 { + return nil, false + } + + return content, true +} + // DefaultScanner is the package-level default scanner instance. var DefaultScanner = NewDefault() diff --git a/internal/scanner/scanner_test.go b/internal/scanner/scanner_test.go index 1b26bae..56b0daa 100644 --- a/internal/scanner/scanner_test.go +++ b/internal/scanner/scanner_test.go @@ -74,6 +74,82 @@ func TestCalculateSummary(t *testing.T) { } } +func TestCalculateSummary_SecretMatchSeverityEscalatesFinding(t *testing.T) { + // A real secret in a properly ignored file: Finding.Severity is Info + // (IsIgnored=true), but its SecretMatches contain a High-severity match. + // Per ADR 0005 item 4, the summary must reflect the max severity across + // the Finding and its SecretMatches, so this must NOT report Passed. + findings := []Finding{ + { + Severity: SeverityInfo, + SecretMatches: []SecretMatch{ + {Line: 3, Key: "AWS_SECRET_ACCESS_KEY", Method: "pattern", Provider: "aws", Severity: SeverityHigh}, + }, + }, + } + + summary := CalculateSummary(findings) + + if summary.High != 1 { + t.Errorf("expected High = 1, got %d", summary.High) + } + if summary.Info != 0 { + t.Errorf("expected Info = 0 (escalated to High), got %d", summary.Info) + } + if summary.Passed { + t.Errorf("expected Passed = false when a SecretMatch has High severity, got true") + } +} + +func TestFinding_EffectiveSeverity(t *testing.T) { + tests := []struct { + name string + finding Finding + expected Severity + }{ + { + name: "no secret matches keeps finding severity", + finding: Finding{Severity: SeverityWarning}, + expected: SeverityWarning, + }, + { + name: "secret match lower than finding severity keeps finding severity", + finding: Finding{ + Severity: SeverityCritical, + SecretMatches: []SecretMatch{{Severity: SeverityHigh}}, + }, + expected: SeverityCritical, + }, + { + name: "secret match higher than finding severity wins", + finding: Finding{ + Severity: SeverityInfo, + SecretMatches: []SecretMatch{{Severity: SeverityCritical}}, + }, + expected: SeverityCritical, + }, + { + name: "max is taken across multiple secret matches", + finding: Finding{ + Severity: SeverityInfo, + SecretMatches: []SecretMatch{ + {Severity: SeverityWarning}, + {Severity: SeverityHigh}, + }, + }, + expected: SeverityHigh, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := tt.finding.EffectiveSeverity(); got != tt.expected { + t.Errorf("expected EffectiveSeverity = %s, got %s", tt.expected, got) + } + }) + } +} + func TestClassifyFinding(t *testing.T) { s := NewDefault() @@ -344,6 +420,264 @@ func TestScanNonGitDirectory(t *testing.T) { } } +func TestScanPopulatesSecretMatches(t *testing.T) { + repoDir := setupGitRepo(t) + + // Untracked file with a real-looking secret -> Secret Match floor is HIGH. + untrackedFile := filepath.Join(repoDir, ".env.local") + if err := os.WriteFile(untrackedFile, []byte("AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\n"), 0644); err != nil { + t.Fatal(err) + } + + // Tracked file with a real-looking secret -> Secret Match escalates to CRITICAL. + trackedFile := filepath.Join(repoDir, ".env.production") + if err := os.WriteFile(trackedFile, []byte("STRIPE_SECRET_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dc\n"), 0644); err != nil { + t.Fatal(err) + } + runGit(t, repoDir, "add", ".env.production") + runGit(t, repoDir, "commit", "-m", "chore: add prod env") + + s := NewDefault() + res, err := s.Scan(repoDir) + if err != nil { + t.Fatalf("Scan failed: %v", err) + } + + findingsMap := make(map[string]Finding) + for _, f := range res.Findings { + findingsMap[f.Path] = f + } + + untracked, ok := findingsMap[".env.local"] + if !ok { + t.Fatalf("expected finding for .env.local") + } + if len(untracked.SecretMatches) != 1 { + t.Fatalf("expected 1 secret match for .env.local, got %d: %+v", len(untracked.SecretMatches), untracked.SecretMatches) + } + if untracked.SecretMatches[0].Severity != SeverityHigh { + t.Errorf("untracked secret match severity = %s, want %s", untracked.SecretMatches[0].Severity, SeverityHigh) + } + if untracked.SecretMatches[0].Provider != "aws" { + t.Errorf("untracked secret match provider = %s, want aws", untracked.SecretMatches[0].Provider) + } + + tracked, ok := findingsMap[".env.production"] + if !ok { + t.Fatalf("expected finding for .env.production") + } + if len(tracked.SecretMatches) != 1 { + t.Fatalf("expected 1 secret match for .env.production, got %d: %+v", len(tracked.SecretMatches), tracked.SecretMatches) + } + if tracked.SecretMatches[0].Severity != SeverityCritical { + t.Errorf("tracked secret match severity = %s, want %s", tracked.SecretMatches[0].Severity, SeverityCritical) + } +} + +func TestScanSkipsSecretScanningForAllowlistedFiles(t *testing.T) { + tempDir := t.TempDir() + + exampleFile := filepath.Join(tempDir, ".env.example") + if err := os.WriteFile(exampleFile, []byte("AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\n"), 0644); err != nil { + t.Fatal(err) + } + + res, err := Scan(tempDir) + if err != nil { + t.Fatalf("Scan failed: %v", err) + } + + if len(res.Findings) != 1 { + t.Fatalf("expected 1 finding, got %d", len(res.Findings)) + } + if len(res.Findings[0].SecretMatches) != 0 { + t.Errorf("expected allowlisted file to have 0 secret matches, got %d: %+v", len(res.Findings[0].SecretMatches), res.Findings[0].SecretMatches) + } +} + +func TestScanSecretMatchSeverityCappedBySeverityOverride(t *testing.T) { + repoDir := setupGitRepo(t) + + // Tracked file with a real-looking secret would normally floor at CRITICAL, + // but a Severity Override of "warning" should cap it there. + trackedFile := filepath.Join(repoDir, ".env.test") + if err := os.WriteFile(trackedFile, []byte("AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\n"), 0644); err != nil { + t.Fatal(err) + } + runGit(t, repoDir, "add", ".env.test") + runGit(t, repoDir, "commit", "-m", "chore: add test env fixture") + + cfg := &config.Config{ + Detector: config.DetectorConfig{ + SeverityOverrides: []config.SeverityOverride{ + {Pattern: ".env.test", Severity: "warning"}, + }, + }, + } + + s := NewWithConfig(nil, nil, cfg) + res, err := s.Scan(repoDir) + if err != nil { + t.Fatalf("Scan failed: %v", err) + } + + var finding Finding + for _, f := range res.Findings { + if f.Path == ".env.test" { + finding = f + } + } + + if len(finding.SecretMatches) != 1 { + t.Fatalf("expected 1 secret match, got %d: %+v", len(finding.SecretMatches), finding.SecretMatches) + } + if finding.SecretMatches[0].Severity != SeverityWarning { + t.Errorf("secret match severity = %s, want %s (capped by severity override)", finding.SecretMatches[0].Severity, SeverityWarning) + } +} + +func TestMatchSeverityOverrideStopsAtFirstMatchEvenWhenUnrecognized(t *testing.T) { + // Config built programmatically (bypassing config.Validate()), as several + // tests in this package already do via NewWithConfig. The first override + // matches but has an unrecognized severity, so it must win with "no + // override" rather than let the loop fall through to the second, + // unrelated override. + cfg := &config.Config{ + Detector: config.DetectorConfig{ + SeverityOverrides: []config.SeverityOverride{ + {Pattern: ".env.production", Severity: "super-critical"}, + {Pattern: "*.production", Severity: "info"}, + }, + }, + } + + s := NewWithConfig(nil, nil, cfg) + + sev, ok := s.matchSeverityOverride(".env.production") + if ok { + t.Errorf("matchSeverityOverride() = (%v, true), want ok=false since the first matching override's severity is unrecognized", sev) + } +} + +func TestScanSkipsSecretScanningForBinaryFiles(t *testing.T) { + tempDir := t.TempDir() + + // A binary-looking file that happens to match an env filename pattern. + binaryContent := append([]byte("AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\x00"), make([]byte, 10)...) + binaryFile := filepath.Join(tempDir, ".env.binary") + if err := os.WriteFile(binaryFile, binaryContent, 0644); err != nil { + t.Fatal(err) + } + + res, err := Scan(tempDir) + if err != nil { + t.Fatalf("Scan failed: %v", err) + } + + if len(res.Findings) != 1 { + t.Fatalf("expected 1 finding, got %d", len(res.Findings)) + } + if len(res.Findings[0].SecretMatches) != 0 { + t.Errorf("expected binary file to be skipped (0 secret matches), got %d: %+v", len(res.Findings[0].SecretMatches), res.Findings[0].SecretMatches) + } +} + +func TestScanSkipsSecretScanningForOversizedFiles(t *testing.T) { + tempDir := t.TempDir() + + oversized := make([]byte, maxScannableFileSize+1) + for i := range oversized { + oversized[i] = 'a' + } + secretLine := []byte("\nAWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\n") + oversized = append(oversized, secretLine...) + + oversizedFile := filepath.Join(tempDir, ".env.big") + if err := os.WriteFile(oversizedFile, oversized, 0644); err != nil { + t.Fatal(err) + } + + res, err := Scan(tempDir) + if err != nil { + t.Fatalf("Scan failed: %v", err) + } + + if len(res.Findings) != 1 { + t.Fatalf("expected 1 finding, got %d", len(res.Findings)) + } + if len(res.Findings[0].SecretMatches) != 0 { + t.Errorf("expected oversized file to be skipped (0 secret matches), got %d: %+v", len(res.Findings[0].SecretMatches), res.Findings[0].SecretMatches) + } +} + +func TestScannerAppliesEntropyScanAndSecretIgnoreConfig(t *testing.T) { + tempDir := t.TempDir() + + content := "NODE_ENV=production\n" + + "RANDOM_TOKEN=K7mP9xQ2vL8nR4tY6wZ1aB3cD5eF0gHj\n" + + "IGNORED_TOKEN=K7mP9xQ2vL8nR4tY6wZ1aB3cD5eF0gHj\n" + envFile := filepath.Join(tempDir, ".env") + if err := os.WriteFile(envFile, []byte(content), 0644); err != nil { + t.Fatal(err) + } + + cfg := &config.Config{ + Detector: config.DetectorConfig{ + EntropyScan: true, + SecretIgnore: []string{"IGNORED_TOKEN"}, + }, + } + + s := NewWithConfig(nil, nil, cfg) + res, err := s.Scan(tempDir) + if err != nil { + t.Fatalf("Scan failed: %v", err) + } + + if len(res.Findings) != 1 { + t.Fatalf("expected 1 finding, got %d", len(res.Findings)) + } + matches := res.Findings[0].SecretMatches + if len(matches) != 1 { + t.Fatalf("expected 1 secret match, got %d: %+v", len(matches), matches) + } + if matches[0].Key != "RANDOM_TOKEN" { + t.Errorf("Key = %q, want %q", matches[0].Key, "RANDOM_TOKEN") + } + if matches[0].Method != "entropy" { + t.Errorf("Method = %q, want %q", matches[0].Method, "entropy") + } +} + +func TestScannerAppliesSecretProvidersConfig(t *testing.T) { + tempDir := t.TempDir() + + content := "AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\n" + envFile := filepath.Join(tempDir, ".env") + if err := os.WriteFile(envFile, []byte(content), 0644); err != nil { + t.Fatal(err) + } + + cfg := &config.Config{ + Detector: config.DetectorConfig{ + SecretProviders: []string{"github"}, // aws disabled + }, + } + + s := NewWithConfig(nil, nil, cfg) + res, err := s.Scan(tempDir) + if err != nil { + t.Fatalf("Scan failed: %v", err) + } + + if len(res.Findings) != 1 { + t.Fatalf("expected 1 finding, got %d", len(res.Findings)) + } + if len(res.Findings[0].SecretMatches) != 0 { + t.Errorf("expected 0 secret matches with aws disabled, got %d: %+v", len(res.Findings[0].SecretMatches), res.Findings[0].SecretMatches) + } +} + func TestScanInvalidDirectory(t *testing.T) { s := NewDefault() diff --git a/internal/secretscanner/secretscanner.go b/internal/secretscanner/secretscanner.go new file mode 100644 index 0000000..c70b177 --- /dev/null +++ b/internal/secretscanner/secretscanner.go @@ -0,0 +1,231 @@ +package secretscanner + +import ( + "bufio" + "bytes" + "math" + "path/filepath" + "regexp" +) + +// Method identifies which detection technique produced a Match. +type Method string + +const ( + // MethodPattern indicates a match found via a known-provider signature. + MethodPattern Method = "pattern" + // MethodEntropy indicates a match found via the entropy heuristic. + MethodEntropy Method = "entropy" +) + +// Match represents a single detected secret instance within scanned content. +type Match struct { + Line int + Key string + Method Method + Provider string +} + +type providerPattern struct { + provider string + regex *regexp.Regexp +} + +var providerPatterns = []providerPattern{ + {provider: "aws", regex: regexp.MustCompile(`AKIA[0-9A-Z]{16}`)}, + {provider: "stripe", regex: regexp.MustCompile(`sk_(live|test)_[0-9a-zA-Z]{16,}`)}, + {provider: "github", regex: regexp.MustCompile(`gh[pousr]_[0-9A-Za-z]{36}`)}, + {provider: "pem", regex: regexp.MustCompile(`-----BEGIN ((RSA|EC|OPENSSH|DSA) )?PRIVATE KEY-----`)}, + {provider: "bearer-token", regex: regexp.MustCompile(`Bearer [A-Za-z0-9\-_.]{20,}`)}, +} + +// keyValueLine extracts the key name from a "KEY=VALUE" style line, if present. +var keyValueLine = regexp.MustCompile(`^\s*([A-Za-z_][A-Za-z0-9_]*)\s*=`) + +// Options configures which detection techniques a Scanner runs. +type Options struct { + // EntropyScan enables the entropy-based heuristic (opt-in; higher false-positive rate). + EntropyScan bool + // Providers restricts pattern matching to the named providers. Empty/nil means all shipped providers. + Providers []string + // Ignore suppresses matches whose Key equals or glob-matches one of these entries, regardless of provider. + Ignore []string +} + +// ValidProviders returns the names of all shipped secret providers. +func ValidProviders() []string { + names := make([]string, len(providerPatterns)) + for i, pp := range providerPatterns { + names[i] = pp.provider + } + return names +} + +// IsValidProvider reports whether provider is one of the shipped provider names. +func IsValidProvider(provider string) bool { + for _, pp := range providerPatterns { + if pp.provider == provider { + return true + } + } + return false +} + +func (o Options) providerEnabled(provider string) bool { + if len(o.Providers) == 0 { + return true + } + for _, p := range o.Providers { + if p == provider { + return true + } + } + return false +} + +func (o Options) keyIgnored(key string) bool { + if key == "" { + return false + } + for _, pattern := range o.Ignore { + if pattern == key { + return true + } + if matched, err := filepath.Match(pattern, key); err == nil && matched { + return true + } + } + return false +} + +const ( + entropyMinValueLength = 16 + entropyThreshold = 3.5 +) + +// Scanner inspects file content for embedded secrets using known-provider +// signature patterns and, optionally, an entropy-based heuristic. +type Scanner struct { + opts Options +} + +// New creates a Scanner with default (pattern-only) detection enabled. +func New() *Scanner { + return &Scanner{} +} + +// NewWithOptions creates a Scanner with the given detection Options. +func NewWithOptions(opts Options) *Scanner { + return &Scanner{opts: opts} +} + +// Scan inspects content line by line and returns every detected Match. +// +// The scanner's buffer is sized to cover content in full: bufio.Scanner's +// default 64KB per-line limit would otherwise make Scan() stop silently +// (bufio.ErrTooLong) on any single line larger than that — e.g. a PEM +// certificate or JSON credentials blob assigned to one env var — dropping +// every match on subsequent lines without reporting an error. Since no line +// can be longer than content itself, sizing the buffer to len(content) +// guarantees no legitimate line ever hits the limit. Any other scanning +// error is returned rather than swallowed. +func (s *Scanner) Scan(content []byte) ([]Match, error) { + matches := make([]Match, 0) + + scanner := bufio.NewScanner(bytes.NewReader(content)) + maxTokenSize := len(content) + if maxTokenSize < bufio.MaxScanTokenSize { + maxTokenSize = bufio.MaxScanTokenSize + } + scanner.Buffer(make([]byte, 0, bufio.MaxScanTokenSize), maxTokenSize) + + lineNum := 0 + for scanner.Scan() { + lineNum++ + line := scanner.Text() + + matchedPattern := false + for _, pp := range providerPatterns { + if !s.opts.providerEnabled(pp.provider) { + continue + } + if pp.regex.MatchString(line) { + key := extractKey(line) + matchedPattern = true + if s.opts.keyIgnored(key) { + continue + } + matches = append(matches, Match{ + Line: lineNum, + Key: key, + Method: MethodPattern, + Provider: pp.provider, + }) + } + } + + if !matchedPattern && s.opts.EntropyScan { + if key, value, ok := extractKeyValue(line); ok && looksLikeSecret(value) && !s.opts.keyIgnored(key) { + matches = append(matches, Match{ + Line: lineNum, + Key: key, + Method: MethodEntropy, + }) + } + } + } + + return matches, scanner.Err() +} + +// extractKeyValue splits a "KEY=VALUE" line into its key and value, if present. +func extractKeyValue(line string) (key string, value string, ok bool) { + idx := bytes.IndexByte([]byte(line), '=') + if idx < 0 { + return "", "", false + } + key = extractKey(line) + if key == "" { + return "", "", false + } + value = line[idx+1:] + return key, value, true +} + +// looksLikeSecret reports whether value is long enough and random-looking enough +// (by Shannon entropy) to be flagged as a likely secret. +func looksLikeSecret(value string) bool { + if len(value) < entropyMinValueLength { + return false + } + return shannonEntropy(value) >= entropyThreshold +} + +// shannonEntropy computes the Shannon entropy (bits per character) of s. +func shannonEntropy(s string) float64 { + if s == "" { + return 0 + } + + counts := make(map[rune]int) + for _, r := range s { + counts[r]++ + } + + length := float64(len(s)) + var entropy float64 + for _, c := range counts { + p := float64(c) / length + entropy -= p * math.Log2(p) + } + return entropy +} + +// extractKey returns the key name from a "KEY=VALUE" line, or "" if not present. +func extractKey(line string) string { + m := keyValueLine.FindStringSubmatch(line) + if m == nil { + return "" + } + return m[1] +} diff --git a/internal/secretscanner/secretscanner_test.go b/internal/secretscanner/secretscanner_test.go new file mode 100644 index 0000000..bcf64a9 --- /dev/null +++ b/internal/secretscanner/secretscanner_test.go @@ -0,0 +1,237 @@ +package secretscanner + +import ( + "bufio" + "bytes" + "testing" +) + +func TestScanDetectsAWSAccessKey(t *testing.T) { + content := []byte("AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\n") + + s := New() + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 1 { + t.Fatalf("expected 1 match, got %d: %+v", len(matches), matches) + } + + m := matches[0] + if m.Line != 1 { + t.Errorf("Line = %d, want 1", m.Line) + } + if m.Key != "AWS_ACCESS_KEY_ID" { + t.Errorf("Key = %q, want %q", m.Key, "AWS_ACCESS_KEY_ID") + } + if m.Method != MethodPattern { + t.Errorf("Method = %q, want %q", m.Method, MethodPattern) + } + if m.Provider != "aws" { + t.Errorf("Provider = %q, want %q", m.Provider, "aws") + } +} + +func TestScanDetectsStripeSecretKey(t *testing.T) { + content := []byte("STRIPE_SECRET_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dc\n") + + s := New() + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 1 { + t.Fatalf("expected 1 match, got %d: %+v", len(matches), matches) + } + + m := matches[0] + if m.Provider != "stripe" { + t.Errorf("Provider = %q, want %q", m.Provider, "stripe") + } + if m.Key != "STRIPE_SECRET_KEY" { + t.Errorf("Key = %q, want %q", m.Key, "STRIPE_SECRET_KEY") + } +} + +func TestScanDetectsGitHubToken(t *testing.T) { + content := []byte("GITHUB_TOKEN=ghp_16C7e42F292c6912E7710c838347Ae178B4a\n") + + s := New() + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 1 { + t.Fatalf("expected 1 match, got %d: %+v", len(matches), matches) + } + + m := matches[0] + if m.Provider != "github" { + t.Errorf("Provider = %q, want %q", m.Provider, "github") + } + if m.Key != "GITHUB_TOKEN" { + t.Errorf("Key = %q, want %q", m.Key, "GITHUB_TOKEN") + } +} + +func TestScanDetectsPEMPrivateKeyHeader(t *testing.T) { + content := []byte("PRIVATE_KEY=\"-----BEGIN RSA PRIVATE KEY-----\n") + + s := New() + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 1 { + t.Fatalf("expected 1 match, got %d: %+v", len(matches), matches) + } + + m := matches[0] + if m.Provider != "pem" { + t.Errorf("Provider = %q, want %q", m.Provider, "pem") + } +} + +func TestScanDetectsBearerToken(t *testing.T) { + content := []byte("AUTH_HEADER=Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.abcdef\n") + + s := New() + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 1 { + t.Fatalf("expected 1 match, got %d: %+v", len(matches), matches) + } + + m := matches[0] + if m.Provider != "bearer-token" { + t.Errorf("Provider = %q, want %q", m.Provider, "bearer-token") + } +} + +func TestScanTracksLineNumbersAndIgnoresPlainLines(t *testing.T) { + content := []byte("NODE_ENV=production\nAWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\nPORT=3000\nSTRIPE_SECRET_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dc\n") + + s := New() + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 2 { + t.Fatalf("expected 2 matches, got %d: %+v", len(matches), matches) + } + if matches[0].Line != 2 { + t.Errorf("matches[0].Line = %d, want 2", matches[0].Line) + } + if matches[1].Line != 4 { + t.Errorf("matches[1].Line = %d, want 4", matches[1].Line) + } +} + +func TestScanEntropyHeuristicDisabledByDefault(t *testing.T) { + content := []byte("SECRET_TOKEN=K7mP9xQ2vL8nR4tY6wZ1aB3cD5eF0gHj\n") + + s := New() + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 0 { + t.Fatalf("expected 0 matches with entropy scan disabled, got %d: %+v", len(matches), matches) + } +} + +func TestScanEntropyHeuristicOptIn(t *testing.T) { + content := []byte("NODE_ENV=production\nSECRET_TOKEN=K7mP9xQ2vL8nR4tY6wZ1aB3cD5eF0gHj\n") + + s := NewWithOptions(Options{EntropyScan: true}) + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 1 { + t.Fatalf("expected 1 match, got %d: %+v", len(matches), matches) + } + + m := matches[0] + if m.Method != MethodEntropy { + t.Errorf("Method = %q, want %q", m.Method, MethodEntropy) + } + if m.Key != "SECRET_TOKEN" { + t.Errorf("Key = %q, want %q", m.Key, "SECRET_TOKEN") + } + if m.Line != 2 { + t.Errorf("Line = %d, want 2", m.Line) + } +} + +func TestScanRespectsProviderAllowlist(t *testing.T) { + content := []byte("AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\nSTRIPE_SECRET_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dc\n") + + s := NewWithOptions(Options{Providers: []string{"stripe"}}) + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 1 { + t.Fatalf("expected 1 match, got %d: %+v", len(matches), matches) + } + if matches[0].Provider != "stripe" { + t.Errorf("Provider = %q, want %q", matches[0].Provider, "stripe") + } +} + +func TestScanHandlesLinesLargerThan64KB(t *testing.T) { + // A line larger than bufio.Scanner's default 64KB token limit (e.g. a PEM + // certificate or JSON credentials blob assigned to one env var) must not + // truncate the scan and hide secrets on later lines. + hugeValue := bytes.Repeat([]byte("a"), bufio.MaxScanTokenSize+1024) + var content bytes.Buffer + content.WriteString("BIG_BLOB=") + content.Write(hugeValue) + content.WriteString("\nAWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\n") + + s := New() + matches, err := s.Scan(content.Bytes()) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 1 { + t.Fatalf("expected 1 match, got %d: %+v", len(matches), matches) + } + if matches[0].Line != 2 { + t.Errorf("Line = %d, want 2", matches[0].Line) + } + if matches[0].Provider != "aws" { + t.Errorf("Provider = %q, want %q", matches[0].Provider, "aws") + } +} + +func TestScanRespectsIgnoreList(t *testing.T) { + content := []byte("AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\nSTRIPE_SECRET_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dc\n") + + s := NewWithOptions(Options{Ignore: []string{"AWS_ACCESS_KEY_ID"}}) + matches, err := s.Scan(content) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if len(matches) != 1 { + t.Fatalf("expected 1 match, got %d: %+v", len(matches), matches) + } + if matches[0].Key != "STRIPE_SECRET_KEY" { + t.Errorf("Key = %q, want %q", matches[0].Key, "STRIPE_SECRET_KEY") + } +}