Skip to content

Submit R**2: Reasonometry Router - #191

Open
sashakolpakov wants to merge 1 commit into
RouteWorks:mainfrom
sashakolpakov:submission-r2-reasonometry-router-little-delta
Open

Submit R**2: Reasonometry Router#191
sashakolpakov wants to merge 1 commit into
RouteWorks:mainfrom
sashakolpakov:submission-r2-reasonometry-router-little-delta

Conversation

@sashakolpakov

Copy link
Copy Markdown

R**2: Reasonometry Router

Prediction-only submission for a private, query-only, single-round router.

  • Router policy was frozen before RouterArena evaluation-data access.
  • Training used xRouteBench TRAIN only.
  • No RouterArena prompts, labels, responses, scores, or per-example outcomes were used for training, fitting, tuning, or policy selection.
  • RouterArena prompts were used only to execute the frozen inference policy.
  • Full predictions: 8,400/8,400 successful generated results.
  • Robustness predictions: 420/420.
  • Maximum reported output tokens: 2,048.
  • The submission exposes only the required config and prediction artifacts; implementation remains private.

Both current official config/prediction validators pass for full and robustness.

@sashakolpakov

Copy link
Copy Markdown
Author

/evaluate

@github-actions

Copy link
Copy Markdown

Router Evaluation Results

Router: r2-reasonometry-router
Dataset Split: full

RouterArena Metrics

Metric Value
RouterArena Score 0.6790
Accuracy 68.31%
Total Cost $1.744850
Avg Cost per Query $0.000208
Avg Cost per 1K Queries $0.2077
Number of Queries 8400
Abnormal Entries 0
Robustness Score 0.8738

Evaluation completed by RouterArena automated workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant