KazTEB Leaderboard 🏆

Kazakh language extension for the Massive Text Embedding Benchmark

This is a new and ongoing project dedicated to a comprehensive evaluation of existing text embedding models on datasets designed for Kazakh language tasks. Link to the project code.

Currently, the leaderboard supports only 3 tasks: retrieval, classification, and bitext mining, based on existing human-annotated datasets. The aim of this project is to extend the list to 8 tasks proposed in MTEB and cover multiple domains within each task. The test datasets are planned to be acquired from real data sources, without using synthetic samples.


10	intfloat/multilingual-e5-large-instruct	0.7174	2048	Unknown	3072	0.7174


1	google/gemini-embedding-001	0.7174	2048	Unknown	3072	0.7174
2	Qwen/Qwen3-Embedding-8B	0.6435	32K	7B	4096	0.6435
3	intfloat/multilingual-e5-large-instruct	0.6416	512	560M	1024	0.6416
4	jinaai/jina-embeddings-v3	0.6321	8192	572M	1024	0.6321
5	Qwen/Qwen3-Embedding-4B	0.6153	32K	4B	2560	0.6153
6	intfloat/multilingual-e5-large	0.6139	512	560M	1024	0.6139
7	intfloat/multilingual-e5-base	0.5631	512	278M	768	0.5631
8	jinaai/jina-embeddings-v4	0.5431	32K	3B	2048	0.5431
9	intfloat/multilingual-e5-small	0.5356	512	118M	384	0.5356
10	Qwen/Qwen3-Embedding-0.6B	0.5045	32K	595M	1024	0.5045
11	bm25s	0.4900	NA	NA	NA	0.4900
12	sentence-transformers/LaBSE	0.3948	512	471M	768	0.3948

📋 TODO:

Dynamic Data Loading: Switching to API-based result fetching for real-time updates without manual JSON uploads.

📧 Contact: arysbatyr@gmail.com