KazTEB Leaderboard 🏆

Kazakh language extension for the Massive Text Embedding Benchmark

This is a new and ongoing project dedicated to a comprehensive evaluation of existing text embedding models on datasets designed for Kazakh language tasks. Link to the project code.

Currently, the leaderboard supports only 3 tasks: retrieval, classification, and bitext mining, based on existing human-annotated datasets. The aim of this project is to extend the list to 8 tasks proposed in MTEB and cover multiple domains within each task. The test datasets are planned to be acquired from real data sources, without using synthetic samples.


1	intfloat/multilingual-e5-large-instruct	0.6435	8192	560M	4096	0.6435


1	Qwen/Qwen3-Embedding-8B	0.6435	32K	7B	4096	0.6435
2	intfloat/multilingual-e5-large-instruct	0.6416	512	560M	1024	0.6416
3	jinaai/jina-embeddings-v3	0.6321	8192	572M	1024	0.6321
4	Qwen/Qwen3-Embedding-4B	0.6153	32K	4B	2560	0.6153
5	intfloat/multilingual-e5-large	0.6139	512	560M	1024	0.6139
6	intfloat/multilingual-e5-base	0.5631	512	278M	768	0.5631
7	intfloat/multilingual-e5-small	0.5356	512	118M	384	0.5356
8	Qwen/Qwen3-Embedding-0.6B	0.5045	32K	595M	1024	0.5045

📋 TODO:

API-based Model Evaluation: Adding results of closed-source models such as Google's Gemini embeddings.
Dynamic Data Loading: Switching to API-based result fetching for real-time updates without manual JSON uploads.

📧 Contact: arysbatyr@gmail.com