Skip to content

feat(obd_ai): add policy guardrails and audit logging - #15

Open
virtuscyber wants to merge 1 commit into
feature/5-obd-ai-serverfrom
feature/6-obd-ai-guardrails
Open

virtuscyber wants to merge 1 commit into
feature/5-obd-ai-serverfrom
feature/6-obd-ai-guardrails

Conversation

@virtuscyber

Copy link
Copy Markdown
Owner

Refs #6

Stacked on top of #5.

Includes:

  • policy classes and decisions for LLM-facing OBD actions
  • approval-required handling for sensitive/destructive operations
  • explicit blocking for raw/custom commands and force=True bypasses
  • structured audit logging for tool invocations and policy decisions
  • tests and docs for guardrail behavior

@augmentcode

augmentcode Bot commented Apr 11, 2026

Copy link
Copy Markdown
🤖 Augment PR Summary

Summary: This PR introduces a policy guardrail layer and structured audit logging around the LLM-facing OBD tool surface to prevent unsafe or destructive operations.

Changes:

  • Added obd_ai.policy with policy classes, allow/deny/approval decisions, and a default policy catalog
  • Extended approved command catalog entries with policy metadata (policy_class, approval_required, guard_tags) and surfaced it via serializers
  • Integrated the policy engine into OBDAISession / OBDAISessionManager and the read-only tool surface
  • Blocked raw/custom command construction fields and force=True bypass attempts via policy evaluation
  • Added obd_ai.audit (audit events + sinks) and emitted tool-invocation and policy-decision audit events
  • Updated docs to describe policy handling and audit emission
  • Added/updated tests validating approval-required behavior (e.g., CLEAR_DTC), blocked actions, and audit event emission

Technical Notes: The tool surface now returns structured error payloads for policy blocks/approval requirements and logs both the decision and the final tool outcome via an injectable audit sink.

🤖 Was this summary useful? React with 👍 or 👎

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 4 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread obd_ai/policy.py
) -> Optional[PolicyDecision]:
request_payload = dict(payload or {})

if request_payload.get("force"):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obd_ai/policy.py:133 — request_payload.get("force") treats any truthy value as a bypass attempt, so a caller passing something like "false" (string) could be denied even though it’s semantically false. Consider requiring a real boolean True (and/or rejecting non-bools) to prevent accidental policy blocks.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Comment thread obd_ai/policy.py
mutable_by_key[action.key] = action
ordered.append(action)

if action.obd_command_name is not None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obd_ai/policy.py:76-78 — _by_obd_command_name silently overwrites entries when multiple actions share the same obd_command_name, which can make find_action() resolve to the wrong policy. It may be safer to detect duplicates and raise so the catalog stays deterministic.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Comment thread obd_ai/audit.py
self._logger = logger or logging.getLogger("obd_ai.audit")

def record(self, event: AuditEvent) -> None:
self._logger.info("obd_ai_audit %s", json.dumps(event.to_dict(), sort_keys=True))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obd_ai/audit.py:57 — json.dumps(event.to_dict()) can raise TypeError if request/policy contains non-JSON types (e.g., programmatic callers could pass a Python bytes value for the bytes field), which would make tool calls fail while logging. Consider making logging resilient (e.g., default=str or pre-sanitizing) so auditing never becomes a new failure mode.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Comment thread obd_ai/tools.py
request=request_payload,
decision="success" if result_payload.get("ok") else "error",
result="success" if result_payload.get("ok") else "error",
error_code=result_payload.get("error", {}).get("code"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obd_ai/tools.py:1053 — tool_invocation.error_code is pulled only from top-level payload["error"], but tools like read_sensors can return ok=False with only per-item errors and no top-level error, leaving audit events with decision=error but error_code=None. Consider including an aggregated/representative error code (or counts by code) in the audit metadata so audits can be filtered reliably.

Severity: low

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant