The government eliminated Naver Cloud and NC AI in the first round among the five elite teams that took part in the "national AI model project." The government initially planned to narrow the five teams to four, but two teams were eliminated at once. The NC AI elite team fell short in benchmarks, expert reviews, and user evaluations, and Naver Cloud failed to meet the originality requirement, dropping out of the competition. With only Upstage, SK Telecom, and LG AI Research Institute remaining as elite teams, the government said it would hold a repechage. Concerns over fairness and equity are expected.
Vice Minister Ryu Je-myung of the Ministry of Science and ICT said at a briefing on the afternoon of the 15th at Government Complex Seoul announcing the "independent AI foundation model project first-stage evaluation results" that "the intention behind the compressed competition among a small number is not so much to select two corporations in the end, but to design a structure that creates the most intense competitive environment so we can achieve many results in a short period." The national AI model project is a core government initiative that concentrates resources such as graphics processing units (GPUs) and data with the goal of building an AI model unique to Korea.
The government said it would give another chance not only to the Naver Cloud and NC AI elite teams eliminated in the first evaluation, but also to Motif Technologies, Kakao, KT, Konan Technology, the KAIST consortium, and all other corporations that were not selected as elite teams. Additional elite teams selected will be allowed to use GPUs and receive data support the same as those that passed stage one, and can also use the K-AI corporate designation.
Some say the government created a pretext for Naver Cloud. That is because the government did not announce the rankings by corporation in the detailed evaluation. Earlier this month, Naver Cloud faced allegations that the image and audio encoders for its "HyperCLOVA X Seed 32B Sync" model copied Alibaba's "Qwen 2.5 ViT" from China. Naver said, "It is not because of a lack of technological self-sufficiency; we strategically adopted external encoders to enhance the overall model's completeness and stability by using standardized, high-performance modules." Naver Cloud said that day, "We respect the Ministry of Science and ICT's decision and will not consider another attempt."
The following is a summary of key questions and answers with Vice Minister Ryu Je-myung of the Ministry of Science and ICT and Kim Kyung-man, director general for AI policy at the ministry.
◇ "We will proceed quickly with the new elite team call... we will give the same GPU and data period"
—What are the criteria and timing for recruiting one additional team?
"Because we faced the unexpected situation of the fourth spot becoming vacant, we will complete the administrative procedures for soliciting a new corporation to join stage two as quickly as possible. We will give opportunities not only to the (eliminated) corporations that could not join stage two, but also to the 10 consortia that could not join the stage-one evaluation and to capable corporations that newly form consortia."
—This is essentially a repechage, but any additional elite team selected will have a time gap. What advantages will you give?
"We will design and provide the GPUs and data supported by the government so that corporations additionally participating can use them for the entire project period. Even if the evaluation period or the project end point differs, we will have them finish stage two within the same period. Even if there is a sequence difference in timing, we plan to manage the period difference flexibly."
—What is the schedule for the second evaluation and the roadmap ahead?
"The three teams (that passed the first-stage evaluation) will be able to start stage two immediately. The government has leased GPUs and is providing them for participating corporations to use, but if the three elite teams (that passed stage one) have to wait for the additional elite team, the leased GPU resources would sit idle, which would waste the budget. (However, for the additionally selected elite team) we are designing it so that the project participation period and the total amount of GPUs provided can be under the same conditions as the three teams that started earlier."
◇ "Naver fell short of technical requirements; explanatory materials sent after the project ended"
—Please explain in more detail why Naver Cloud was eliminated.
"We analyzed the problematic encoder in detail from technical, policy, and ethical perspectives. The project call guidelines included the basic conditions that an independent AI foundation model project must meet. What we essentially aim for in an independent AI foundation model project is to design and train from scratch under any conditions, and (Naver Cloud) did not meet that, which became the reason for elimination. Many evaluators also pointed out that Naver Cloud fell short of the project's technical requirements. While using external encoders is a method commonly used in development, in this case the encoder was not in a form where the weights could be updated but was frozen, so using an external encoder and weights as-is made it difficult to recognize it as an independent foundation model, according to our internal judgment."
—Did Naver make a prior inquiry regarding the encoder issue?
"No. For reference, after the controversy arose, Naver sent an explanatory statement. Depending on various judgment issues and perspectives, there were parts open to different interpretations. The important point is that the explanation did not take place before the project ended on Dec. 31 last year, and it was sent while the evaluation was underway, so it was not reflected. We determined there would be various procedural problems in reflecting (the explanatory materials) received after the process had already ended into the post-project evaluation. Naver said it has its own encoders and that the encoder used now accounts for a fairly small portion of the project. There are differences in views, and experts see it differently."
—Detailed criteria such as whether using an external encoder is allowed likely could not be included in the first call. Will you provide clearer guidelines for judging originality going forward?
"Globally, including global corporations, it is fair to say there are no corporations that do not use open source. We should not see open source as a sin. Appropriately and most strategically using open source at each stage in accordance with licensing terms is natural in the AI ecosystem. However, what we are trying to do in the independent AI foundation model project is to design the model ourselves, and even if we use open source, using already trained weights as-is is, in a way, free-riding on others' experience. We want to newly undertake the training experience itself. That way, even when we use open source going forward, we can build more competitive AI models. Because the pace of AI competition is fast, rather than starting such a project after everything is certain, we started quickly despite uncertainties and have been improving as we go. In the development process, the government has managed it as we continuously aligned, and the operators have communicated with the government to proceed. Regarding the evaluation, we also sought consensus that could be derived mutually with the elite teams on the criteria and methods, and did it to the greatest extent aligned and agreed."
◇ Repechage expected to spark fairness concerns
—Was there originally a minimum passing score? You said Naver Cloud and NC AI can apply again and come back; if they return or a new team joins, will there be any penalties in stage two?
"The intention behind the compressed competition among a small number is not so much to select two corporations in the end, but to design a structure that creates the most intense competition so we can achieve many results in a short period. We judged that even corporations not directly participating would be spurred to catch up technologically. For instance, Motif Technologies and KT did not advance to the first-round finals, but they accelerated R&D and achieved high rankings at the global AI performance evaluation organization Artificial Analysis. In that spirit, we intend to allow a fresh start and a renewed challenge so that stage-one results do not affect stage two at all."
—Won't the repechage raise fairness concerns for other elite teams that participated? It can be seen as giving an opportunity to someone among the two operators that fell. Isn't selecting additional elite teams wasteful?
"For corporations that could not participate in stage one, we have created foundation model projects in specialized fields using available resources and are running two consortia. Our GPU resources and budget are limited, but the background of the stage-one evaluation was to let as many AI corporations as possible use GPUs one way or another under resource constraints and to create conditions for them to participate in AI R&D. We gained a lot in the process. This is by no means an approach hastily crafted to accommodate a particular corporation or for a particular corporation. We have individually consulted with all participating corporations, and the consensus formed is that the achievements generated by these corporations should not become the property of any particular corporation. They should contribute as open source that many AI service corporations in Korea can use. Instead of using the term 'repechage' now, please describe it as a re-leap, catch-up, or re-challenge program."
—Naver Cloud was not the only one with a from-scratch issue. Were the same standards applied to other elite teams, and was there any disagreement?
"In the case of Upstage, evaluators pointed out issues with unmentioned references. When we set standards for independent foundation models, we believed Korea's AI ecosystem would advance further if, even when using open-source software made public, we could confidently state how we used it and what parts we modified, and undergo technical verification. From that perspective, Upstage's unmentioned reference issue can indeed be seen as not meeting our standards. However, experts did not consider it such a serious defect as to determine pass or fail. SK Telecom also received some criticism in that regard. But it was not an absolute evaluation criterion."
—Will the evaluation criteria be the same for stage two?
"The broad framework of benchmark, expert, and user evaluations will likely remain unchanged. However, regarding the from-scratch portion, we plan to incorporate opinions from academia, industry, and experts to make differentiated scoring and weighting by degree more specific. Still, there will be uncertainties. If we secure additional GPUs, we can supply more. It is not set, but we could raise the target. It may be good to adapt to and accept the competitive situation among global AI corporations and proceed dynamically."