The MTEB Leaderboard is available here and we encourage additions of new benchmarks. To add a new benchmark:
- Add your benchmark to benchmark.py as a
Benchmark
object, and select the MTEB tasks that will be in the benchmark. If some of the tasks do not exist in MTEB, follow the "add a dataset" instructions to add them. - Open a PR at https://github.com/embedding-benchmark/results with results of models on your benchmark.
- When PRs are merged, your benchmark will be added to the leaderboard automatically after the next workflow trigger.