Bae Kyung-hoon, Deputy Prime Minister and Minister of the Ministry of Science and ICT, said he would review improving the evaluation method for the second assessment results of the "domestic artificial intelligence (AI) foundation model (Dokpamo)," noting there were "regrettable parts."
Bae, the deputy prime minister, said this at the second "science and technology-artificial intelligence future strategy meeting" held at the President Hotel in Jung-gu, Seoul, on the 27th.
Bae, the deputy prime minister, said, "It is time for a fundamental reflection on whether the evaluations and competition methods established a year and a half ago are still valid amid AI trends that change every three to six months," adding, "To build a world-class model that anyone can recognize, we will review supplementing how government investment is focused and the evaluation system." He emphasized, "We must compete globally by producing results of such a world-class level that no one can dispute."
Motif Technologies, which was eliminated in the second Dokpamo evaluation, issued a statement the same day calling for disclosure of the detailed evaluation criteria and a review. The second Dokpamo evaluation consists of 40 points for benchmark evaluation (25 points for AAII, 15 points for NIA), 35 points for expert evaluation, and 25 points for user evaluation, and among these, Motif received the highest score among Dokpamo participants in AAII.
Kim Kyung-man, Deputy Minister for artificial intelligence policy at the Ministry of Science and ICT, told reporters after the meeting that they would handle Motif's objection to the second evaluation results fairly in accordance with established procedures. Kim said, "Objections focus on whether there were procedural or formal unfairness in the evaluation," adding, "Under the rules, we plan to conduct a review within 15 days and make a final decision through a designated agency."
Kim said, "It is not enough to strengthen a model's uniqueness; how it is actually used matters," explaining, "The objectives of the Dokpamo project include not only performance targets but also service targets." He added, "In the expert evaluation, we raised the usability score weight from 10 points to 15 and reflected 25 points for real-user evaluation to uphold the purpose of the Dokpamo project," noting, "These criteria were not unilaterally changed by the government but had already been disclosed after reaching agreement with participating corporations."
On the demand to disclose scores, Kim said, "We were deeply concerned about the stigma and side effects experienced by corporations eliminated after the first announcement," adding, "We will decide after internal review, considering fairness and the impact on the ecosystem."
Regarding the criteria for the upcoming third Dokpamo evaluation, Kim said, "While conducting the second evaluation, global AI development trends such as AI agents, advanced reasoning, and coding shifted rapidly," adding, "For the third round, we will set moving targets by discussing development direction and evaluation methods with participating corporations."