Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
15 changes: 5 additions & 10 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@

- `fp-cloud-cli`'s Click shim survives typer 0.27.2, which moved `Abort` out of its vendored Click. `_click_compat` wrapped all six vendored imports in one `try: … except ImportError: from click import …`, so that single missing name rebound **every** symbol to pip Click — the exact silent failure the module exists to prevent. Typer catches only its own Click's exceptions, so every typed error escaped uncaught: `fp alerts show ghost` exited 1 with an empty stderr instead of 6 with a message, and the same for exits 2, 3, 4 and 5. 105 tests went red on the dependabot bump that first installed 0.27.2. The Click is now chosen once — on whether `typer._click` exists at all — and each symbol imported from that choice, so a name that goes missing raises at import (a CLI that will not start) rather than silently downgrading every error to exit 1. `Abort` alone is resolved from `typer.Abort`, which tracks the move by construction: pip Click's before typer 0.26, the vendored class through 0.27.1, `typer.exceptions.Abort` from 0.27.2 (#771)

### Docs

- The 1.0.2 documentation overhaul is reverted: `docs/` and `README.md` go back byte-for-byte to the commit #756 merged onto, along with the 14 generated locales and 14 translated READMEs #759 regenerated from those English sources. The release half of #756 stays — `package.json` and the Cargo workspace are untouched, since they have moved on to 1.0.4-beta.0 and the release tag the CLI builds its daemon download URL from is that npm version. Under `## 1.0.2` the heading and its release narrative stay, because 1.0.2 did ship; the `### Docs` entries underneath described the overhaul and go with it. Leaving the locales in place was the alternative considered and rejected: the nightly translate job is content-hash cached, so pages whose pre-overhaul English hashes it had already seen would have been skipped rather than repaired, stranding every non-English reader on a translation of text that no longer exists (#773)
- The landing page no longer opens with the harness paragraph claiming that "the same events, the same policies, and the same session history apply to every one" of the twelve. Removed from `docs/index.mdx` and all 14 locales (#773)

### Dependencies

- `fp-cloud-cli`: typer 0.27.1 → 0.27.2, click 8.4.2 → 8.5.0, posthog 7.42.0 → 7.44.2 (#771)
Expand Down Expand Up @@ -54,16 +59,6 @@ pinned by the digest of its artifact and by the commit it was built from, and a
pack that cannot be loaded DENIES within the scope it declared rather than
disappearing quietly.

### Docs

- The documentation is rewritten against the shipped 1.0.2 CLI. An audit of all 68 English pages plus the README found 219 factual errors, and the whole of `start/` documented a setup path that no longer exists: bare `failproofai config` — the verb that installs the daemon and wires every supported agent CLI — appeared on no page, and neither did `failproofai policies add FailproofAI/policies`, so a reader who followed the quickstart end to end finished with hook entries, no daemon, and one enforcing policy. The retired `pack`/`policy` spellings, `pack add core`, `--bundled` and `config --connect <url> --token <key>` are gone; the local no-account path (the dashboard on `localhost:8020`, local findings from `failproofai audit`, and separately documented telemetry controls) is documented for the first time; and `start/integrations.mdx`, which five pages linked to and nothing in the navigation reached, is wired in (#756)
- Observe mode is documented as what it is: the policy is EVALUATED for real, under the same timeout and error handling as an enforcing one, and its verdict recorded — only the enforcement is withheld. Both layers that set it are named, `failproofai publish --effect observe` for a pack and `fp fleet deploy <machine> --add <id>:observe` for a Cloud deployment, and every page showing `--add` now says that a bare one ENFORCES immediately, which is how a shadow rollout turns into a production incident. `policies/deploy.mdx` had described the workflow entirely as dashboard clicks and named no `fp` command at all; the CLI lane — `fp fleet list/show/deploy/diff/history/rollback`, `fp guardrails summary|timeline`, and `fp policies test`, which decides a policy locally with no server and no auth — is now covered (#756)
- Enforcement is no longer described as uniform across the twelve harnesses. `docs/index.mdx` claimed "the same events, the same policies, and the same session history apply to every one" — the exact claim `src/hooks/enforcement-capability.ts` exists to stop drifting. A `PreToolUse` deny is verified to stop the tool on all twelve; a `Stop` deny is verified on eight, is `observe` on Pi, and is unverified on OpenCode, Hermes and Goose. Goose has no `Stop` event and we install none on Hermes, so the five `require-*-before-stop` builtins never fire on either. A pair with no row in that file now reads as unverified rather than as blocking (#756)
- The install no longer claims that policies come with it. "39 built-in policies activate immediately" and "eleven are on when you accept the defaults" were both false: `configure-wizard.ts` enables nothing, so `enabledPolicies` is empty and `builtin-policies.ts` registers only what carries `alwaysOn` — exactly one policy, `block-failproofai-commands`. The eleven `defaultEnabled` flags are catalog metadata that decide nothing until a pack is installed (#756)
- The Policy Hub at befailproof.ai/policy-hub is documented, including that `failproofai publish` does not set the `failproofai-policies` topic the hub indexes on — so publishing is publish-then-tag-by-hand until it does (#756)
- Two claims copied out of the CLI's own `--help` are corrected, because the help is wrong. Scope support is not "Codex, Copilot, Cursor, OpenCode and Pi take user or project only": `types.ts` makes Hermes and OpenClaw **user-only** and gives `local` to Claude alone. And `backfill --help` names `~/.failproofai/config.toml`, a layout-2 file no current build writes; the file is `config.json` (#756)
- Six fabricated commands and flags were caught by an adversarial verification pass before they shipped, among them `fp fleet rename --name` (the command takes two positionals and declares no options), a `tool` filter on the local dashboard's activity view (`HookActivityFilters` has six keys and tool is not one), and `failproofai config --token <key> --machine-label <name>` as a setup one-liner — which sets nothing up, since any invocation carrying `--machine-label` without `--connect` routes straight to the rename path and exits 1 on a machine that is not yet connected (#756)

### Fixes

- The mirror of that, on the success path: a shared artifact that imports FINE but registers less than one of its packs selected. Registration is recorded per pack id, and every hook from a collapsed load carries only the id the collapse kept — so a policy declared by the non-winning record's manifest and absent from the artifact left that pack in neither the failure map nor the registered map, and `missingGuards` skips a pack in neither. Nothing registered the policy and nothing denied on its behalf: the machine reported itself enforcing a scope that was running nothing. Registrations are propagated to every pack id behind the artifact now, so the pack that selected the missing policy is measured against what actually loaded (#738)
Expand Down
71 changes: 13 additions & 58 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@

**Translations:** [简体中文](./docs/i18n/README.zh.md) · [日本語](./docs/i18n/README.ja.md) · [한국어](./docs/i18n/README.ko.md) · [Español](./docs/i18n/README.es.md) · [Português](./docs/i18n/README.pt-br.md) · [Deutsch](./docs/i18n/README.de.md) · [Français](./docs/i18n/README.fr.md) · [Русский](./docs/i18n/README.ru.md) · [हिन्दी](./docs/i18n/README.hi.md) · [Türkçe](./docs/i18n/README.tr.md) · [Tiếng Việt](./docs/i18n/README.vi.md) · [Italiano](./docs/i18n/README.it.md) · [العربية](./docs/i18n/README.ar.md) · [עברית](./docs/i18n/README.he.md)

**See what your agents do. Stop known failures before they repeat.**
Failproof AI works wherever your agents run: coding tools like Claude Code and
Codex, chat gateways like Hermes, self-hosted assistants like OpenClaw, and agents
you instrument yourself. It records each run and can block dangerous tool calls
before they execute.
**Observability and enforcement for every harness your agents run in.**
Wherever your agents run, we see it — and we can say no. Failproof hooks 12 agent
harnesses — coding CLIs like Claude Code and Codex, chat gateways like Hermes,
self-hosted assistants like OpenClaw — capturing every run and blocking dangerous
tool calls before they execute. 39 built-in policies. Zero latency. Runs locally.

</div>

Expand All @@ -30,9 +30,9 @@ before they execute.

## Supported harnesses

Twelve harnesses in two classes are supported: ten coding CLIs, plus two
gateways: Hermes, OpenClaw. The policy API and session history are shared; which
events can block varies by harness.
Twelve harnesses in two classes — ten coding CLIs, and two chat and assistant
gateways (Hermes, OpenClaw). Same events, same policies, same session history,
Comment thread
nk-ag marked this conversation as resolved.
whichever one your agent runs in.

Agents that run in none of them report through the [Python SDK](https://docs.befailproof.ai/reference/custom-agents),
which gives you tracing, sessions and audits. Enforcement there needs a hook in
Expand Down Expand Up @@ -134,31 +134,13 @@ your own runtime — [talk to us](mailto:support@befailproof.ai) and we'll map i

## Install

Give a compatible agent the Failproof AI skill if you want it to guide setup,
inspect the machine, and route policy, audit, session, and Cloud work correctly:

```sh
npx skills add FailproofAI/skills
```

This installs the umbrella skill and its specialist siblings. To install only the
umbrella, add `--skill failproofai`. Skills supply operating instructions; install
and configure the product itself with:

```sh
npm install -g failproofai
failproofai config
failproofai policies add FailproofAI/policies
failproofai # dashboard on localhost:8020
failproofai policies --install # or just run `failproofai` and accept the first-run prompt
failproofai
```

Setup connects supported agents and installs the background service. It chooses no
policy pack: before you add one, only `block-failproofai-commands` runs to stop an
agent disabling Failproof AI.

Connect Cloud without prompts with `failproofai config --token <machine-key>`. On a
shared machine or in CI, set `FAILPROOFAI_CLOUD_TOKEN` and run `failproofai config`
so the key does not appear in command history.
39 built-in policies activate immediately. Dashboard at `localhost:8020`. Disable the first-run prompt with `FAILPROOFAI_NO_FIRST_RUN=1`.
Comment thread
nk-ag marked this conversation as resolved.

---

Expand All @@ -175,8 +157,8 @@ so the key does not appear in command history.
| `block-rm-rf` | Recursive file deletion |
| `block-force-push` / `block-push-master` | `git push --force`, direct pushes to `main` |

These policies protect files, credentials, infrastructure, databases, and agent
workflows. Exact enforcement support varies by harness and event.
The first five apply to any agent that can call a tool. The last three are the
developer favourites — coding CLIs are the harness class we cover deepest.

→ [All 39 built-in policies](https://docs.befailproof.ai/policies/builtin)

Expand Down Expand Up @@ -213,33 +195,6 @@ Three decisions available to every policy:

---

## Policy packs

A policy pack is a versioned set of policies published from a public GitHub
repository. Inspect one before installing it:

```sh
failproofai policies show FailproofAI/policies
failproofai policies add FailproofAI/policies
```

Anything with a slash is a pack source; anything without one is a policy name.
You can install selected categories or policies, and pin a release when needed.

```sh
failproofai policies add FailproofAI/policies --category git,database
failproofai policies add owner/repo@a1b2c3d4e5f6
```

Browse published packs in the [Policy Hub](https://befailproof.ai/policy-hub/), or
run `failproofai publish --init` to start your own. Observe mode lets a pack record
what it would have done without blocking: `failproofai publish --effect observe`.

→ [Policy packs](https://docs.befailproof.ai/policies/packs) ·
[Publish a pack](https://docs.befailproof.ai/policies/publish-a-pack)

---

## Observability

Enforcement is one half. The other half is seeing what the agent actually did.
Expand Down
82 changes: 51 additions & 31 deletions docs/admin/keys-and-permissions.mdx
Original file line number Diff line number Diff line change
@@ -1,54 +1,74 @@
---
title: "Keys and permissions"
description: "Create credentials for people, automation, and machines."
description: "Create scoped API keys for machines, automation, and operators."
icon: "key-round"
---

Use a separate key for each person, machine, or automation job. Grant only the permissions it needs.
API keys belong to an organization and carry explicit permissions. Use separate keys for agent ingestion, policy delivery, evaluators, CI automation, and administrative scripts.

## Create and rotate a key

Use **Admin → Keys**, or the Cloud CLI:
<Tabs>
<Tab title="Dashboard">
1. Go to **Administration → Keys**, select **new key**, and enter a workload name.
2. Choose a permission set and adjust individual permissions only when the preset is insufficient.
3. Create the key and copy its one-time secret immediately.
4. Open the key later to update grants, disable it, or regenerate the secret.

```bash
fp keys list
fp keys create "audit automation" --add audits:read
fp keys disable "audit automation"
```
The creation drawer is where you choose the narrowest grants required by the workload.

Store the secret when it is created; it is not shown again. Rotate by creating a replacement, updating the consumer, then disabling the old key.
![The new API key drawer with permission presets and individual grants.](/images/dashboard/key-create.png)

## Machine keys
After creation, the Keys page shows the persistent metadata and management actions. The one-time secret is not shown again.

A machine key connects the local service to Cloud:
![The API Keys page showing key permissions, creation time, and regenerate and disable actions.](/images/dashboard/api-keys.png)

```bash
export FAILPROOFAI_CLOUD_TOKEN="<key>"
failproofai config
```
Use this list to review grants regularly and disable keys that no longer map to an active workload.
</Tab>
<Tab title="CLI">
```bash
fp keys create production-agents \
--add events:add \
--add policies:pull
fp keys show production-agents
fp keys update production-agents --add events:read
fp keys regenerate production-agents --yes
fp keys disable production-agents
```

A connected machine may need two capabilities:
Redirect or capture create/regenerate output securely; the secret is returned once.
</Tab>
</Tabs>

- `policies:pull` to receive Cloud-managed policies.
- `events:add` to send decisions and sessions.
The two permissions required by a connected Failproof AI machine are independent:

Status reports these separately because one may work while the other does not.
- `events:add` sends events and session data.
- `policies:pull` retrieves assigned policy deployments.

## Common permissions
Key secrets are shown when created or regenerated. Store them in a secret manager and rotate them without reusing an operator's interactive credentials.

| Permission | Allows |
## Permission catalog

| Area | Permissions |
| --- | --- |
| `events:add` | Send events |
| `events:read` | Read events and errors |
| `evaluations:read` | Read sessions and evaluations |
| `audits:read` / `audits:write` | Review or manage audits |
| `policies:read` / `policies:write` | Review or deploy policies |
| `policies:pull` | Pull machine policy assignments |
| `keys:create` / `keys:disable` | Create or disable keys |
| `orgs:admin` | Instance-level organization administration; not assignable to an organization key |
| Events | `events:add`, `events:read` |
| Keys | `keys:create`, `keys:read`, `keys:disable`, `keys:regenerate`; `keys:update` is human-session only |
| Users | `users:create`, `users:read`, `users:update`, `users:delete` |
| Evaluations | `evaluations:read`, `evaluations:trigger` |
| Dashboards | `dashboards:read`, `dashboards:write`, `dashboards:delete` |
| Queries | `queries:read`, `queries:write`, `queries:delete`, `queries:run` |
| Assistant | `agent:use` |
| Settings | `settings:read`, `settings:write` |
| Alerts | `alerts:read`, `alerts:write` |
| Issues | `issues:read`, `issues:create`, `issues:close` |
| Audits | `audits:read`, `audits:write` |
| Policies | `policies:read`, `policies:write`, `policies:pull` |
| Usage | `usage:read` |

`orgs:admin` is reserved for the instance operator and cannot be granted to an organization key or ordinary member. Retired `incidents:*` and `alerts:ack` tokens are accepted for compatibility and normalize to current `issues:*` permissions.

Some administrative `fp fleet` and `fp guardrails` commands require a signed-in user session rather than an API key. Their help text states this before making a request.
Builtin permission sets are `read-only`, `standard`, and `admin`. `standard` adds evaluation triggering, query execution, issue response, and assistant use to read permissions. Key creation strips human-only grants even when a permission set contains them.

<Warning>
Never reuse ingest credentials such as `AGENTEYE_KEY` or `AGENTEYE_API_KEY` as an `FP_API_KEY`. They serve different systems and permissions.
Instance-scoped keys can select an organization with the `X-AgentEye-Org` header. Set it explicitly on multi-organization deployments; omission may select the default organization.
</Warning>
Loading
Loading