Skip to content

Evaluation new models #36

Open
Open
Evaluation new models#36

Description

@elmoBG8
elmoBG8
opened on May 2, 2024

Hello, I see you added new supported models. Can you provide an evaluation of them on SWE-bench so that it can be compared with the evaluations already done?

Thank you

Activity

  1. qrdlgit commented on May 3, 2024

    @qrdlgit
    qrdlgit
    on May 3, 2024

    Yes, the entire engineering world is holding their breath :)

    SWE-bench is probably the best eval out there right now, and this is one of the best ways to evaluate swe-bench!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions