security-benchmark

Open-source, 100% reproducible AI Agent Runtime Security Benchmark & Sandbox Environment (RFC-010 Draft Protocol).

Open, privacy-bounded assurance for AI agents: containment provenance, identity passports, authorization twins, OTel evidence, CI gates, and OSCAL.

Open deterministic security tests for unsafe multi-agent handoffs and authority escalation.

Deterministic security benchmark for tool-using AI agents

FreightSkillBench is a reproducible benchmark for evaluating document-to-transaction integrity, prompt-injection risk, and security controls in AI-enabled shipping and logistics workflows.

Production-grade microservices security benchmark featuring OWASP Top 10 logic exploits, automated remediation, custom Semgrep SAST rules, and CI/CD DevSecOps gates.

Open AI-for-security validation benchmark: non-LLM scorer + a SOTA-validation loop. Labeled positive corpus withheld pending coordinated disclosure.

Smart contract vulnerability detection and benchmarking framework for Solidity, including SDB and enhanced DeFi security test suites.

The public adversarial evaluation suite for Signetry: measures attack success rate (ASR) and utility-under-defense for coding-agent prompt injection, skill/MCP poisoning, and memory-injection threats, governed by signetry-core.

ReplayBench-IoT: reproducible IoT replay-defense benchmark with Monte Carlo sweeps, CI, static demo, and hardware-validation artifacts.

Security evaluation input | Labeled Python injection, deserialization, crypto and trust-boundary cases | labeled-positive-negative

An AI assistant that can't be hijacked by the documents it reads. The part that decides what to do never sees untrusted text, and every value is tagged with its origin, checked at each action. In a 1440-run offline benchmark: a normal agent was hijacked 73/73, this one 0/73.

Automated adversarial security testing for AI agents. Deploys an LLM-powered attacker against tool-using systems, validates violations via deterministic oracles, and produces reproducible vulnerability reports with causal attack graphs.

Open reference guard + G-G parameter-risk vector pool: 400 frozen published vectors (200-vector holdout sealed), 35.5% -> 95.0% interception at 0 FPR. Elastic-2.0 (code), CC-BY-SA-4.0 (data).

Security evaluation input | Python AWS CDK, generated infrastructure intent, encryption and network permissions | intentional-vulnerabilities

Synthetic AI security benchmark for claim integrity across bilingual agent handoffs

Open-source agent runtime security benchmark harness

Security evaluation input | Verilog RTL, timing leakage, register clearing and functional controls | labeled-positive-negative

Security evaluation input | LLM application trust boundaries, prompt/data separation and model-output handling | intentional-vulnerabilities

Benchmark LLM providers and self-hosted models on security analysis and prompt-injection robustness. Claude, OpenAI, Kimi, GLM, and local vLLM/Ollama through one code path.