Among the five elite teams participating in the government's "national flagship AI model project," Naver Cloud and NC AI were eliminated in the first round on the 15th. The NCAI elite team was cut in the first-stage evaluation, and the Naver Cloud elite team was eliminated on the criterion of originality. The national flagship AI model project is a core government initiative that concentrates support such as graphics processing units (GPUs) and data, with the goal of building a uniquely Korean AI model to rank among the world's top three AI nations.
Second Vice Minister Ryu Je-myeong of the Ministry of Science and ICT said this at a briefing on the "independent AI foundation model project first-stage evaluation results" held at Government Complex Seoul on the afternoon of the 15th. With this, the elite teams from Upstage, SK Telecom, and LG AI Research Institute advanced to the second stage.
The first-stage evaluation was conducted with △ benchmark (40 points) △ expert evaluation (35 points) △ user evaluation (35 points). In the three categories, excluding NC AI, the elite teams from LG AI Research Institute, Naver Cloud, SK Telecom, and Upstage met the requirements. However, the Naver Cloud elite team fell short on originality.
The Ministry of Science and ICT said, "As a result of evaluating the independent AI foundation model project standards from technical, policy, and ethical perspectives, the AI model of the Naver Cloud elite team was judged not to meet the originality standard," and explained, "Expert evaluators also raised issues about the limits of originality, leading to the decision to eliminate the Naver Cloud elite team."
The Ministry of Science and ICT added that, by eliminating two teams instead of one in the first round, it plans to select one additional elite team. Eligible applicants for the additional elite team include the consortia that applied to the initial project call, the consortia not included among the elite teams after this first-stage evaluation (Naver Cloud and NC AI consortia), and other corporations. The ministry said, "The one additional elite team to be selected will be given an opportunity to take on the development of the independent AI foundation model project through support such as graphics processing units (GPUs) and data and the conferment of the 'K-AI company' designation, and we will swiftly proceed with an additional call for one elite team through administrative procedures."
◇ LG AI Research Institute tops benchmark, expert, and user evaluations... NC AI eliminated
In the first-stage evaluation, LG AI Research Institute earned the highest scores across all areas.
First, for the "benchmark" score, the government conducted evaluations by the National Information Society Agency (NIA) with △ NIA benchmark evaluation (10 points) △ global common benchmark evaluation (20 points) △ global individual benchmark evaluation (10 points). The NIA benchmark evaluation considered not only math, knowledge, and long-form understanding but also reliability and safety in collaboration with the AI Safety Research Institute. For the global common benchmark evaluation, 13 globally recognized benchmarks (spanning numerous areas including agents, math, knowledge/reasoning, and instruction following) were selected for assessment. For the global individual benchmark evaluation, five benchmarks comparable to each elite team's global target model (SOTA level) were assessed.
In the NIA benchmark evaluation, SK Telecom and LG AI Research Institute each scored a highest 9.2 out of 10. In the global common benchmark evaluation, LG AI Research Institute scored a highest 14.4 out of 20. In the global individual benchmark evaluation, Upstage and LG AI Research Institute each scored a highest 10.0 out of 10. Aggregating these, LG AI Research Institute achieved the top benchmark score of 33.6. The five-team elite average was 30.4.
For the expert evaluation, a committee of 10 external AI experts from industry, academia, and research was formed to conduct an in-depth evaluation over an extended period based on materials submitted by each team. They said they comprehensively assessed △ development strategy and technology △ development performance and plans △ ripple effects and contribution plans.
The evaluation committee analyzed each elite team's publicly released technical report, AI model training state log files, and more to assess the teams' technology development process and technological capabilities (including originality). LG AI Research Institute earned the highest expert evaluation score of 31.6 out of 35. The five-team elite average was 28.56.
For the user evaluation, 49 AI expert users, including AI startup CEOs, participated. Based on AI-use websites built with the elite teams' AI models, they conducted an in-depth analysis of practical on-site applicability and inference cost efficiency. In user evaluation scores as well, LG AI Research Institute achieved the highest, 25.0 out of 25. The average score was 20.76.
◇ Naver Cloud eliminated for failing to meet originality requirements
The government also conducted an originality evaluation encompassing △ technical △ policy △ ethical perspectives.
On the technical side, the government aims for "independent implementation of the AI model," carrying out end-to-end training by designing an original AI model architecture, independently securing and processing large-scale data, and applying proprietary training algorithm techniques. The government noted that while it is a general trend in the global AI ecosystem to use open source, it regarded the basic condition for an independent AI model, as commonly accepted across the domestic and international AI industry and academia, to be developing the AI model by training after initializing weights. Therefore, even if elite teams strategically use proven open source to leverage verified technologies, ensure alignment with the global AI ecosystem, and expand globally, the minimum requirement to secure model originality is to initialize weights before training and development.
On the policy side, to address potential concerns such as national secret leaks or threats to national security when using foreign AI models for defense, diplomacy, security, and national infrastructure (power grids, transportation, communications networks), the government aims to secure the capacity to independently develop and advance AI models at any time (sovereignty) and to autonomously control the operation and use of AI models under any circumstances (control). In other words, AI models must be developed entirely with Korea's own technology, or be self-developed and advanced using open source with no licensing restrictions, and they must remain free from external control or interference resulting from open source use, the government said.
On the ethical side, given today's AI ecosystem where using open source is common, the government said there is a need to promote sound development by complying with license policies such as reference notices for AI models developed using open source, thereby building trust in the AI ecosystem, strengthening public verification, and enhancing transparency.