Artificial intelligence
HyperCLOVA X surpasses GPT-4 in Korean AI evaluation
It got higher scores in tests composed in Korean compared to OpenAI's GPT-3.5-Turbo and Google's Gemini-Pro
By Feb 27, 2024 (Gmt+09:00)
1
Min read
Most Read
![HyperCLOVA X surpasses GPT-4 in Korean AI evaluation](https://www.kedglobal.com/data/ked/image/2024/02/27/ked202402270004.647x.0.jpg)
South Korea's Naver Cloud, the cloud computing arm of Naver Corp., announced on Tuesday that its HyperCLOVA X has scored higher than OpenAI and Google's generative AIs in the Korean AI performance evaluation system Measuring Massive Multitask Language Understanding in Korean (KMMLU).
KMMLU is an AI performance evaluation metric construction project led by the renowned domestic open-source language model research team HAE-RAE.
It consists of 35,030 questions asking for expert-level knowledge in 45 fields, including humanities, sociology, science, and technology.
About 80% of the questions ask for broad knowledge that can be applied worldwide, such as mathematical reasoning ability, while 20% evaluate the ability to solve Korea-specific problems, such as the geography of the Korean Peninsula and domestic law.
Composed of test questions in Korean, KMMLU allows for a more accurate assessment of AI's Korean language understanding capabilities, measuring both universal abilities and local knowledge to comprehensively judge AI solutions useful for Korean users, according to Naver.
According to the KMMLU research paper, HyperCLOVA X recorded higher scores than OpenAI's GPT-3.5-Turbo and Google's Gemini-Pro and even surpassed OpenAI's GPT-4 in terms of Korea-specific knowledge.
Naver Cloud plans to develop HyperCLOVA X into a Sovereign AI solution equipped with both security and performance, based on the competitive performance proven through KMMLU.
Write to Ju-Hyun Lee at deep@hankyung.com
More to Read
-
Tech, Media & TelecomNaver to develop HyperCLOVA X-based edu service
Feb 16, 2024 (Gmt+09:00)
1 Min read -
Cloud computingNaver Cloud, NHN Cloud to co-work for hyperscale AI
Feb 05, 2024 (Gmt+09:00)
1 Min read -
Artificial intelligenceNaver, NHIS to boost data services with HyperCLOVA X
Dec 18, 2023 (Gmt+09:00)
1 Min read -
Tech, Media & TelecomNaver raises S.Korean market lead thanks to LLM HyperCLOVA X
Sep 05, 2023 (Gmt+09:00)
1 Min read -
Artificial intelligenceNaver eyes global market with new LLM HyperCLOVA X
Aug 24, 2023 (Gmt+09:00)
4 Min read -
Tech, Media & TelecomKorea's Hancom, Naver Cloud agree to cooperate in AI field
Mar 30, 2023 (Gmt+09:00)
1 Min read
Comment 0
LOG IN