Skip to content

Latest commit

 

History

History
7 lines (5 loc) · 658 Bytes

adding_a_benchmark.md

File metadata and controls

7 lines (5 loc) · 658 Bytes

Adding a benchmark

The MTEB Leaderboard is available here and we encourage additions of new benchmarks. To add a new benchmark:

  1. Add your benchmark to benchmark.py as a Benchmark object, and select the MTEB tasks that will be in the benchmark. If some of the tasks do not exist in MTEB, follow the "add a dataset" instructions to add them.
  2. Open a PR at https://github.com/embedding-benchmark/results with results of models on your benchmark.
  3. When PRs are merged, your benchmark will be added to the leaderboard automatically after the next workflow trigger.