Repository navigation

added why.md for the environments - #23

Open

ygxio wants to merge 98 commits into

sys-intelligence:mainfrom

ygxio:pr/why-md

Commits on Nov 6, 2025

Rename it "System Intelligence Benchmark"
xuafeng
authored
6d24e69

Commits on Nov 7, 2025

Init: Initialize SysMoBench benchmark integration
Qian-Cheng-nju
committed
87db1e9
feat: Add gitigore
Qian-Cheng-nju
committed
69f4cb5
feat: Add prototype for phase 1&2
Qian-Cheng-nju
committed
843f031

Commits on Nov 9, 2025

feat: Distinguish evaluator and model API keys in env.toml
Qian-Cheng-nju
committed
0d2b38f
feat: Add validation for required evaluator API keys
Qian-Cheng-nju
committed
b2acaa7

Commits on Nov 10, 2025

doc: update README.md
Qian-Cheng-nju
committed
ca7e72e
initial ArtEval commit
bastoica
committed
ec7b57f
Merge pull request #2 from systemintelligence/feat/distinguish-api-keys
xuafeng
authored
a607e73

Commits on Nov 11, 2025

feat: Add test
Qian-Cheng-nju
committed
60d30e0
featr: Add install.sh
Qian-Cheng-nju
committed
ff96313

Commits on Nov 12, 2025

adding overview and contributor's guide
bastoica
committed
1799370
skeleton ArtEval agent implementation
bastoica
committed
2054314
adding sosp24 wasabi
bastoica
committed
6303aa5

Commits on Nov 13, 2025

docs: add arteval to main README
tareknaser
committed
a5358dc
feat(ci): add GH Actions workflow for running benchmarks tests
tareknaser
committed
904374e
feat: add issue and pull request templates
tareknaser
committed
40ccf1f
fix(ci): add a test for example_bench
tareknaser
committed
4130c7a
fix: shell scripts to be executable
tareknaser
committed
3af5b70
docs: update README with instructions for running a single benchmark
tareknaser
committed
156c77c
docs: a note on docker image arch support
tareknaser
committed
a0557f9
meta: add outputs directories to gitignore
tareknaser
committed
868da59
feat(ci): add release trigger to workflow
tareknaser
committed
ea9b54d

Commits on Nov 14, 2025

fix: Use tla_specification instead of generated_text to adapt upstream changes
Qian-Cheng-nju
committed
5ef835f
Merge commit '04900168e10834f3aa5eef4d13b318e1efcdac24' as 'benchmarks/sysmobench/sysmobench_core'
Qian-Cheng-nju
committed
c5dfbb1
fix: Add gpt-4o config and fix cross-device link issue in setup_tools
Qian-Cheng-nju
committed
a68e171
fix: Convert GenerationOutput to GenerationResult for evaluators
Qian-Cheng-nju
committed
984336a
docs: Update README and install script for Git Subtree integration
Qian-Cheng-nju
committed
68025ca
feat: Add docker file
Qian-Cheng-nju
committed
25c6af8
fix: Add env.toml
Qian-Cheng-nju
committed
97c1c3c
feat: Add tasks
Qian-Cheng-nju
committed
76eb459
Merge branch 'main' into sysmobench
Qian-Cheng-nju
authored
321320a
refactor(course_exam_bench): eliminate data duplication and simplify
tareknaser
committed
627a634
fix(ci): only load config file when needed
tareknaser
committed
32b406f

Commits on Nov 15, 2025

fix: Separate the executor and evaluator for further extensibility
Qian-Cheng-nju
committed
061b7ad

Commits on Nov 17, 2025

doc: Add porting benchmark guide
Qian-Cheng-nju
committed
72b6916
doc: small updates
Qian-Cheng-nju
authored
1a7f1b1
chore(course_exam_bench): update 6_1810 exam_id for consistency
tareknaser
committed
4d18658
Merge pull request #11 from systemintelligence/docs_course_exam_bench
xuafeng
authored
1577aca
Merge pull request #10 from systemintelligence/sysmobench
xuafeng
authored
faa1ff3
Revise README for clarity and contribution instructions
xuafeng
authored
6a2e118
Rename custom_benchmark.md to creating_benchmark.md
xuafeng
authored
164fded
Update link to custom benchmark guidelines
xuafeng
authored
2a2efa8
Enhance formatting and clarity in porting_benchmark.md
xuafeng
authored
6645b7a
Update links to creating_benchmark.md in documentation
xuafeng
authored
5fb70de
Rename SysCap-CLI to Benchmark-CLI
xuafeng
authored
8d8cd34
fix: addressing compilation issues and some format compatibility issues
bastoica
committed
d79f851
Revise guide for creating custom benchmarks
xuafeng
authored
8e07579
Mark project as under development in README
xuafeng
authored
3ddc038
Mark Cache Benchmark as under development
xuafeng
authored
d74f3d1

Commits on Nov 18, 2025

Rename cache_bench to algo_cache_bench
xuafeng
committed
39f79db
Rename course_project_bench to course_lab_bench
xuafeng
committed
bc2ce10
Update links for changed benchmark names
xuafeng
committed
ccfc983
Merge pull request #12 from systemintelligence/update_doc
xuafeng
authored
b839fb2
Fix the github org name change
xuafeng
committed
f88182d
Merge pull request #13 from sys-intelligence/fix
xuafeng
authored
73fc32c
Merge commit '38fb9c7a5fd40ba2176ad62cc8f1059f7f005959' into sysmobench
Qian-Cheng-nju
committed
2aec74c
Merge branch 'main' of https://github.com/systemintelligence/system_intelligence_benchmark into sysmobench
Qian-Cheng-nju
committed
bc35289
feat: update interface
Qian-Cheng-nju
committed
a8f3f26
Rename to cache_algo_bench
xuafeng
committed
f1216c7
Merge pull request #14 from sys-intelligence/fix_cache_algo
xuafeng
authored
f52f896
fix: update benchmark title in README
xuafeng
committed
8b1a8cb
Clean couser_lab_bench
xuafeng
committed
a1a6bb6
fix: correct benchmark references in creating_benchmark.md
xuafeng
committed
1b84b81
fix: update framework name in README for consistency
xuafeng
committed
d0cfe49
fix: update Slack channel link in contribution guidelines
xuafeng
committed
fd06274
fix: update repository links in documentation for consistency
xuafeng
committed
55221f7

Commits on Nov 19, 2025

feature: consolidated AI agent oracles
bastoica
committed
f9c3552
feature: add .gitignore
bastoica
committed
949a3da
fix: cleaning up _agent_eval directory
bastoica
committed
dd49df2
fix: adapt Wasabi's README file to exclude git instructions
bastoica
committed
41e9ee1
fix: refactor JSON containing task description
bastoica
committed
66e0107
fix: refactor the AI agent evaluator bundle to return the final score
bastoica
committed
d593df2
fix: unitialized variable
bastoica
committed
07935ff
refactor: major refactoring of the ArtEval agent scaffolding; also adding several popular code-focused AI agents
bastoica
committed
51fac0f
fix: correcting the name of the benchmark in the initial log message
bastoica
committed
0fba806
feature: improving and expanding the contributor's guide
bastoica
committed
ddac095
fix: rebase the repository's top-level README
bastoica
committed
ffedcc0
docs(course_lab_bench): add instructions for extending the benchmark
tareknaser
committed
10f3d07
Merge pull request #16 from sys-intelligence/docs_course_lab_bench
xuafeng
authored
f7d3cbd
Merge pull request #15 from sys-intelligence/arteval_benchmark
xuafeng
authored
535a78f

Commits on Nov 20, 2025

Commits on Nov 22, 2025

fix: apply suggestions to WHY.md

bastoica
and
tareknaser
authored
c72230a
fix: a few typos in WHY.md

bastoica
and
tareknaser
authored
77b6e7f

Commits on Nov 23, 2025

Commits on Nov 24, 2025

Commits on Dec 6, 2025

why md for benchmarks
ygxio
committed
3928b04

Commits on Dec 12, 2025

doc: Revise benchmark system count and types in WHY.md
Qian-Cheng-nju
authored
c783e5c