The French observatory Compar:IA recently unveiled a user-generated ranking of the artificial intelligence models most popular among French-speaking internet users.1 Presented as a collective evaluation effort, this initiative has sparked widespread interest… but also significant criticism. While it reflects the public’s growing curiosity about generative AI, it also raises a key question: can we truly measure a model’s quality based on user votes? This ranking, which places Mistral Medium 3.1 at the top ahead of models from Google and Alibaba, stands out more for its participatory nature than for its methodological rigor.
\n\n\n\nUnlike technical benchmarks (such as MMLU or GSM8K), which measure the consistency, accuracy, or robustness of models,2 the Compar:IA ranking is based on subjective votes. Each user is presented with two anonymous responses from two different AIs and then chooses the one they find clearest or most convincing. While this method has the merit of involving the general public, it has several limitations:
\n\n\n\nCompar:IA's 2025 rankings crown the French Mistral Medium 3.1 model, followed by Google's Gemini 2.5 Flash and Gemini 2.0 Flash models.3 Trailing behind are Alibaba’s Qwen 3 Max and DeepSeek v3 from the Chinese company DeepSeek.
While these results are symbolically encouraging for the French and European scenes, they do not necessarily reflect the reality of technical performance. Indeed:
The “blind” voting system implemented by Compar:IA was designed to reduce name-recognition bias. However, several limitations remain. The tests do not account for the factual accuracy of responses or measure generative biases (ideological, cultural, or linguistic). AI researchers point out that a model can be “likable” without being effective: a persuasive or empathetic AI is not necessarily accurate.
Linguistic bias is also significant: models trained on French-language corpora, such as those from Mistral, have a clear advantage. Furthermore, emotional perception often influences preference: a warmer tone, a familiar turn of phrase, or a narrative style can be perceived as indicators of quality.
In short, Compar:IA helps us understand the public’s expectations regarding generative AI, but it does not measure their actual performance. 4
On the aivancity blog, the General AI rankings are based on a radically different approach: scientific, comparative, and measurable.5 These analyses evaluate models based on objective criteria such as:
\n\n\n\nThese rigorous evaluations help identify models that are truly effective and suitable for professional or academic use. Unlike Compar:IA, they do not aim to measure popularity, but rather to rank models based on facts, not perceptions.
\n\n\n\nAll General AI rankings can be viewed in the AI Tools, which regularly compares the latest models based on technical, ethical, and energy-related criteria.
\n\n\n\nThe success of the Compar:IA rankings highlights a broader phenomenon: the general public’s growing desire to understand and compare AI systems. But it also underscores the need for digital literacy training regarding evaluation tools.
Rather than pitting a citizen-driven approach against scientific expertise, it would be more effective to integrate the two. A participatory evaluation can enrich the perception of use, while a technical analysis ensures the reliability and reproducibility of results.
Ultimately, European institutions, through the AI Act, will encourage the creation of hybrid indicators: trust barometers that integrate technical performance, sustainability, and model transparency.6
The Compar:IA rankings reflect a genuine interest in artificial intelligence and a collective desire to understand its applications. However, in the absence of a scientific methodology, they cannot be considered a reliable measurement tool. They provide a snapshot of user preferences, not a ranking of the best AI systems.
\n\n\n\nFor a rigorous analysis of models, aivancity’s Generative AI rankings remain the most comprehensive and unbiased benchmark available in the French-speaking market. By combining academic expertise, technical criteria, and ethical considerations, they go beyond mere popularity to provide a truly informed view of artificial intelligence performance.
\n\n\n\nFor further reading on the topics of evaluation and sovereignty in the field of artificial intelligence:
\n\n\n\n1. Compar:IA. (2025). Citizen Observatory on Artificial Intelligence – 2025 Rankings. » target= »_blank »> https://comparia.fr
\n \n
2. Hendrycks, D. et al. (2021). Measuring Massive Multitask Language Understanding (MMLU). arXiv. \n \n
\n\n\n\n3. Le Journal du Net. (2025). Mistral beats Google and OpenAI in the Compar:IA rankings. » target= »_blank »> https://www.journaldunet.com
\n
4. Ministry of Culture. (2025). Note on the methodology for participatory evaluations of AI models.
https://www.culture.gouv.fr \n \n
5. aivancity. (2025). General AI Rankings – AI Tools Category.
https://www.aivancity.ai/blog/category/outils-ia/ \n \n
6. European Commission. (2024). AI Act – European Regulation on Artificial Intelligence.
https://digital-strategy.ec.europa.eu \n
The French observatory Compar:IA recently unveiled a user-generated ranking of the artificial intelligence models most popular among French-speaking internet users.1 Presented as a collective evaluation effort, this initiative has sparked widespread interest… but also…
\n","protected":false},"author":2,"featured_media":425987,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[130,40],"tags":[59],"class_list":{"0":"post-425968","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-etudes-ia","8":"category-ia-responsable-durable","9":"tag-parlonsia"},"yoast_head":"\n