Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,22 @@ _Avoid_: Crawler, inspector, walker

**Detector**:
The rule evaluator that determines if a file path is an environment file and whether it matches safe allowlist templates.
_Avoid_: Matcher, filter, classifier
_Avoid_: Matcher, filter, classifier. Not to be confused with **Secret Scanner**, which inspects file _content_ rather than file _paths_.

**Configuration**:
The project-level settings loaded from `.envguard.yaml`, `.envguard.yml`, or via `--config` to customize scanning and detection rules.
_Avoid_: Options, settings, preferences

**Allowlist**:
The collection of glob patterns for safe environment templates or sample files (e.g., `.env.example`) that should not raise security warnings.
The collection of glob patterns for safe environment templates or sample files (e.g., `.env.example`) that should not raise security warnings. Files matching the Allowlist are also skipped entirely by the **Secret Scanner**.
_Avoid_: Whitelist, safe-list, permitted files

**Ignore Directory**:
A directory name skipped during recursive filesystem traversal (e.g., `node_modules`, `.git`).
_Avoid_: Excluded path, blacklisted folder

**Severity Override**:
An explicit configuration rule that replaces the calculated severity level for files matching a specific pattern.
An explicit configuration rule that replaces the calculated severity level for files matching a specific pattern. When set on a file, it also acts as a ceiling on that file's Secret Match severities (a Secret Match's severity is `min(computed severity, file's overridden severity)`).
_Avoid_: Custom rule, priority tweak

**Finding**:
Expand All @@ -41,7 +41,7 @@ The process of stripping secret values from environment definitions while preser
_Avoid_: Masking, redacting, cleaning

**Fixer**:
The remediation component responsible for automatically generating and applying non-destructive `.gitignore` rules for unignored environment files.
The remediation component responsible for automatically generating and applying non-destructive `.gitignore` rules for unignored environment files. Scoped strictly to `.gitignore` edits; it never invokes the Secret Scanner and has no awareness of Secret Matches.
_Avoid_: Patcher, autofixer, corrector, remediator

**Hook Manager**:
Expand All @@ -51,3 +51,11 @@ _Avoid_: Hook installer, hook script, hook handler
**Hook Runner**:
The fast execution mode invoked during Git pre-commit lifecycle to inspect staged repository files and prevent accidental commits of environment secrets.
_Avoid_: Commit watcher, stage scanner, commit blocker

**Secret Scanner**:
The content-inspection component that examines the text inside environment files for embedded secrets, using known-provider signature patterns (AWS, Stripe, GitHub, etc.) and entropy-based heuristics as independent checks. Skips any file covered by the Allowlist and any file that fails a binary/size check.
_Avoid_: Detector, content matcher, secret detector

**Secret Match**:
A single detected secret instance found by the Secret Scanner within a Finding's file: line number, key name (if identifiable), detection method (pattern or entropy), provider (if pattern-matched), and its own severity, independent of the parent Finding's severity.
_Avoid_: Secret finding, leak, hit
32 changes: 32 additions & 0 deletions docs/adr/0005-secret-scanning-scope-and-defaults.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# 0005: Secret Scanning Scope, Detection Strategy, and Defaults

To implement the v0.3.0 roadmap goal of content-based secret scanning, `envguard` introduces a **Secret Scanner** that inspects the text inside environment files for embedded secrets, producing **Secret Match** records distinct from the existing **Finding** concept.

## Context & Problem

Filename/path-level detection (the existing `Detector`) tells you a file _might_ contain secrets, but not whether it actually does. The roadmap calls for two capabilities: known-provider signature matching (AWS, Stripe, GitHub, etc.) and entropy-based heuristics for unrecognized secret formats. These have very different false-positive profiles, and the feature needed to be scoped without breaking the `hook run` `<10ms` performance budget or the "never print secret values" design principle.

## Decision

1. **Scope**: The Secret Scanner only inspects files already classified as environment files (the `Detector`'s existing territory). It does not scan arbitrary repository files (source code, general config) in v0.3.0 — that's a larger surface with a different false-positive character, deferred to a later version.

2. **Detection strategy**: Pattern matching (known-provider signatures) and entropy calculation are independent checks, both able to produce `Secret Match` records on the same file. Pattern matching ships **on by default**; entropy-based detection is **opt-in** via `detector.entropy_scan`, since its higher false-positive rate risks alert fatigue and broken CI before the heuristic is tuned.

3. **Command boundaries**: The Secret Scanner runs in `scan` and `check` only. It is excluded from `hook run` to preserve its `<10ms` budget, and `fix` remains completely unaware of it — `fix` is scoped strictly to `.gitignore` edits and has no remediation path for a real secret (rotation isn't automatable).

4. **Secret Match is a distinct concept from Finding**, not a merge into it: a `Finding` (env file, severity from git tracked/staged status) can contain zero or more `Secret Match` records, each with its own severity floor (`HIGH` when untracked, `CRITICAL` when tracked/staged) — independent of the parent `Finding`'s severity. `check`'s exit-code aggregation takes the max severity across a file's `Finding` and its `Secret Match` records.

5. **Existing features extend to cover it**: files on the `Allowlist` (e.g. `.env.example`) are skipped entirely by the Secret Scanner. A file's `Severity Override` (e.g. `.env.test` → `warning`) acts as a ceiling on that file's `Secret Match` severities, so intentionally fake secrets in test fixtures don't trigger `CRITICAL`. A separate `detector.secret_ignore` list suppresses specific false-positive matches by key name/pattern regardless of file.

6. **Output stays zero-value-exposure**: a `Secret Match` is reported as key name (if identifiable) + file + line number only — never a value fragment, consistent with envguard's existing guarantee.

## Considered Options

- **Scanning all repository files, not just env files** — rejected for v0.3.0: much larger surface, higher false-positive risk from code that legitimately references key _names_, and a bigger traversal/performance question. Left as a future expansion.
- **Entropy detection on by default alongside pattern matching** — rejected: shipping a high-false-positive heuristic on-by-default in a security tool risks alert fatigue and broken CI runs before it's proven in the wild.
- **Folding Secret Match into the existing Finding object** — rejected: a `Finding`'s severity is driven by git exposure status, while a `Secret Match`'s severity is driven by the secret itself; conflating them would have made the severity model harder to reason about and prevented the `Severity Override` ceiling from applying cleanly.

## Consequences

- Widening scope later (e.g. scanning all repository files) will require revisiting the traversal and ignore-directory logic that today is scoped to already-detected env files.
- Enabling entropy detection by default in a future release is a config default change, not an architectural one — low cost to reverse.
2 changes: 1 addition & 1 deletion internal/cli/check.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ func runCheckCommand(args []string, stdout, stderr io.Writer, scannerInstance *s
minRank := severityRank(minSev)
filtered := make([]scanner.Finding, 0, len(result.Findings))
for _, f := range result.Findings {
if severityRank(f.Severity) >= minRank {
if severityRank(f.EffectiveSeverity()) >= minRank {
filtered = append(filtered, f)
}
}
Expand Down
2 changes: 1 addition & 1 deletion internal/cli/scan.go
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ func runScanCommand(args []string, stdout, stderr io.Writer, scannerInstance *sc
minRank := severityRank(minSev)
filtered := make([]scanner.Finding, 0, len(result.Findings))
for _, f := range result.Findings {
if severityRank(f.Severity) >= minRank {
if severityRank(f.EffectiveSeverity()) >= minRank {
filtered = append(filtered, f)
}
}
Expand Down
16 changes: 16 additions & 0 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"path/filepath"
"strings"

"github.com/joaooncode/envguard/internal/secretscanner"
"gopkg.in/yaml.v3"
)

Expand All @@ -33,6 +34,12 @@ type DetectorConfig struct {
CustomPatterns []string `yaml:"custom_patterns,omitempty"`
Allowlist []string `yaml:"allowlist,omitempty"`
SeverityOverrides []SeverityOverride `yaml:"severity_overrides,omitempty"`
// EntropyScan enables the Secret Scanner's entropy-based heuristic (opt-in; off by default).
EntropyScan bool `yaml:"entropy_scan,omitempty"`
// SecretProviders restricts the Secret Scanner's pattern matching to these providers. Empty means all shipped providers.
SecretProviders []string `yaml:"secret_providers,omitempty"`
// SecretIgnore suppresses Secret Matches whose key equals or glob-matches one of these entries.
SecretIgnore []string `yaml:"secret_ignore,omitempty"`
}

// SeverityOverride overrides the calculated severity for files matching a pattern.
Expand All @@ -52,6 +59,8 @@ func NewDefault() *Config {
CustomPatterns: make([]string, 0),
Allowlist: make([]string, 0),
SeverityOverrides: make([]SeverityOverride, 0),
SecretProviders: make([]string, 0),
SecretIgnore: make([]string, 0),
},
}
}
Expand Down Expand Up @@ -130,5 +139,12 @@ func (c *Config) Validate() error {
return fmt.Errorf("severity_overrides[%d]: invalid severity %q (supported: info, warning, high, critical)", i, override.Severity)
}
}

for i, provider := range c.Detector.SecretProviders {
if !secretscanner.IsValidProvider(provider) {
return fmt.Errorf("detector.secret_providers[%d]: unknown provider %q (supported: %s)", i, provider, strings.Join(secretscanner.ValidProviders(), ", "))
}
}

return nil
}
39 changes: 39 additions & 0 deletions internal/config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,45 @@ detector:
}
}

func TestParseYAML_SecretScanning(t *testing.T) {
yamlContent := `
detector:
entropy_scan: true
secret_providers:
- aws
- stripe
secret_ignore:
- "TEST_*"
`
cfg, err := Parse([]byte(yamlContent))
if err != nil {
t.Fatalf("unexpected error parsing valid yaml: %v", err)
}

if !cfg.Detector.EntropyScan {
t.Errorf("expected EntropyScan = true")
}
if len(cfg.Detector.SecretProviders) != 2 {
t.Errorf("expected 2 secret providers, got %d", len(cfg.Detector.SecretProviders))
}
if len(cfg.Detector.SecretIgnore) != 1 || cfg.Detector.SecretIgnore[0] != "TEST_*" {
t.Errorf("expected secret_ignore to contain 'TEST_*', got %v", cfg.Detector.SecretIgnore)
}
}

func TestParseYAML_InvalidSecretProvider(t *testing.T) {
invalidProviderYAML := `
detector:
secret_providers:
- aws
- awss
`
_, err := Parse([]byte(invalidProviderYAML))
if err == nil {
t.Fatal("expected error on unknown secret_providers entry, got nil")
}
}

func TestParseYAML_InvalidSyntax(t *testing.T) {
invalidYAML := `
scanner:
Expand Down
13 changes: 7 additions & 6 deletions internal/reporter/json.go
Original file line number Diff line number Diff line change
Expand Up @@ -58,12 +58,13 @@ func (j *JSONReporter) Render(result *scanner.Result, w io.Writer) error {
sanitizedFindings := make([]scanner.Finding, len(findings))
for i, f := range findings {
sanitizedFindings[i] = scanner.Finding{
Path: f.Path,
Severity: f.Severity,
Message: f.Message,
Suggestions: f.Suggestions,
GitStatus: f.GitStatus,
IsAllowed: f.IsAllowed,
Path: f.Path,
Severity: f.Severity,
Message: f.Message,
Suggestions: f.Suggestions,
GitStatus: f.GitStatus,
IsAllowed: f.IsAllowed,
SecretMatches: f.SecretMatches,
}
}

Expand Down
23 changes: 23 additions & 0 deletions internal/reporter/reporter_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ func sampleResult() *scanner.Result {
Severity: scanner.SeverityCritical,
Message: "Environment file is tracked by Git (committed in repository history).",
Suggestions: []string{"Remove file from git tracking: git rm --cached .env", "Add to .gitignore", "Rotate any leaked credentials"},
SecretMatches: []scanner.SecretMatch{
{Line: 3, Key: "AWS_ACCESS_KEY_ID", Method: "pattern", Provider: "aws", Severity: scanner.SeverityCritical},
},
},
{
Path: ".env.local",
Expand Down Expand Up @@ -180,6 +183,14 @@ func TestTerminalReporter_Render(t *testing.T) {
if !strings.Contains(out, "FAILED") {
t.Errorf("expected FAILED status in summary, got:\n%s", out)
}

// Verify nested secret match line, and that no value is ever printed
if !strings.Contains(out, "AWS_ACCESS_KEY_ID") || !strings.Contains(out, "line 3") {
t.Errorf("expected nested secret match with key and line number, got:\n%s", out)
}
if strings.Contains(out, "AKIA") {
t.Errorf("secret value must never be printed, got:\n%s", out)
}
})

t.Run("Render with ANSI colors enabled", func(t *testing.T) {
Expand Down Expand Up @@ -269,6 +280,18 @@ func TestJSONReporter_Render(t *testing.T) {
t.Errorf("unexpected first finding: %+v", report.Findings[0])
}

if len(report.Findings[0].SecretMatches) != 1 {
t.Fatalf("expected 1 secret match on first finding, got %d: %+v", len(report.Findings[0].SecretMatches), report.Findings[0].SecretMatches)
}
sm := report.Findings[0].SecretMatches[0]
if sm.Line != 3 || sm.Key != "AWS_ACCESS_KEY_ID" || sm.Provider != "aws" || sm.Severity != scanner.SeverityCritical {
t.Errorf("unexpected secret match: %+v", sm)
}

if len(report.Findings[1].SecretMatches) != 0 {
t.Errorf("expected 0 secret matches on second finding, got %+v", report.Findings[1].SecretMatches)
}

if report.Summary.Critical != 1 || report.Summary.High != 1 || report.Summary.Warning != 1 || report.Summary.Info != 1 {
t.Errorf("unexpected summary counts: %+v", report.Summary)
}
Expand Down
19 changes: 19 additions & 0 deletions internal/reporter/terminal.go
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,13 @@ func (t *TerminalReporter) Render(result *scanner.Result, w io.Writer) error {
sb.WriteString(fmt.Sprintf(" • %s\n", sug))
}
}
if len(f.SecretMatches) > 0 {
sb.WriteString(fmt.Sprintf(" %s\n", t.c.cyan("Secrets:")))
for _, sm := range f.SecretMatches {
_, badge := t.severityBadge(sm.Severity)
sb.WriteString(fmt.Sprintf(" • %s %s at line %d\n", badge, t.secretLabel(sm), sm.Line))
}
}
sb.WriteString("\n")
}
}
Expand All @@ -138,6 +145,18 @@ func (t *TerminalReporter) Render(result *scanner.Result, w io.Writer) error {
return err
}

// secretLabel formats a SecretMatch's key/provider without ever including its value.
func (t *TerminalReporter) secretLabel(sm scanner.SecretMatch) string {
label := sm.Key
if label == "" {
label = "(unnamed key)"
}
if sm.Provider != "" {
label = fmt.Sprintf("%s (%s)", label, sm.Provider)
}
return label
}

func (t *TerminalReporter) severityBadge(sev scanner.Severity) (icon string, badge string) {
switch sev {
case scanner.SeverityCritical:
Expand Down
40 changes: 33 additions & 7 deletions internal/scanner/finding.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,25 @@ const (
SeverityInfo Severity = "info"
)

// SecretMatch represents a single detected secret instance within a Finding's file.
// Its Severity is independent of the parent Finding's severity.
type SecretMatch struct {
Line int `json:"line"`
Key string `json:"key,omitempty"`
Method string `json:"method"`
Provider string `json:"provider,omitempty"`
Severity Severity `json:"severity"`
}

// Finding represents a detected environment file and its Git security posture.
type Finding struct {
Path string `json:"path"`
Severity Severity `json:"severity"`
Message string `json:"message"`
Suggestions []string `json:"suggestions,omitempty"`
GitStatus git.FileStatus `json:"git_status"`
IsAllowed bool `json:"is_allowed"`
Path string `json:"path"`
Severity Severity `json:"severity"`
Message string `json:"message"`
Suggestions []string `json:"suggestions,omitempty"`
GitStatus git.FileStatus `json:"git_status"`
IsAllowed bool `json:"is_allowed"`
SecretMatches []SecretMatch `json:"secret_matches,omitempty"`
}

// Summary aggregates finding counts categorized by severity.
Expand All @@ -49,11 +60,26 @@ type Result struct {
Summary Summary `json:"summary"`
}

// EffectiveSeverity returns the maximum severity between the Finding itself and any
// of its SecretMatches. Per ADR 0005 item 4, a Secret Match's severity is independent
// of its parent Finding's severity, and exit-code aggregation must take the max of
// the two — otherwise a real secret in a properly ignored file (Finding severity
// Info) would be silently dropped from the summary.
func (f Finding) EffectiveSeverity() Severity {
max := f.Severity
for _, m := range f.SecretMatches {
if severityRank(m.Severity) > severityRank(max) {
max = m.Severity
}
}
return max
}

// CalculateSummary computes metrics for a slice of findings.
func CalculateSummary(findings []Finding) Summary {
var s Summary
for _, f := range findings {
switch f.Severity {
switch f.EffectiveSeverity() {
case SeverityCritical:
s.Critical++
case SeverityHigh:
Expand Down
Loading
Loading