The artificial intelligence (AI) inference market is already on track and I believe it will become the main battleground of the AI industry going forward. Beyond simple chatbots that just give answers, once AI agents that handle complex tasks become common, demand for inference computation will inevitably grow exponentially.
Jeong Yoon-seok, chief strategy officer (CSO) at Rebellions, met with ChosunBiz at the company's headquarters in Seongnam, Gyeonggi, on the 8th and said, The center of gravity in the AI industry is rapidly shifting from training to inference. After the training stage of building AI models, inference—where AI provides answers to users' questions or commands—has become the center of the market. As AI services like ChatGPT become part of everyday life, inference computation runs 24/7 inside data centers.
Jeong said AI infrastructure for training and inference requires different approaches, adding, In inference, cost efficiency (TCO, total cost of ownership) and low latency (performance to reduce response delay) are key. Jeong said, It is a reality that global AI corporations are bearing massive infrastructure expense, and added, Demand for dedicated infrastructure optimized for inference by lowering compute expense will grow explosively.
Rebellions has been working with Arm and SK Telecom (SKT) since April to develop next-generation AI infrastructure aimed at this market shift. The core is to mount Arm's data center CPU, the Arm AGI CPU, and Rebellions' AI accelerator RebelCard in a single server. The inference servers built this way will be applied in SKT's AI data centers to validate performance and stability in real operating environments.
The three companies plan to boost performance by optimizing not only the hardware coupling but also software across the board, including firmware (basic software that controls hardware). Jeong said, When operating AI services in production, a range of processing—data input/output, network communication, memory management, job scheduling—must occur simultaneously in addition to computation, adding, The CPU plays the role of a control tower that oversees system operation, while the Neural Processing Unit (NPU), a semiconductor specialized for AI computation, is dedicated to AI inference computation.
◇ Rebellions puts sovereign AI front and center
Rebellions is a homegrown fabless (semiconductor design) company whose main business is designing, developing, and supplying AI Semiconductor chips. Launched in Sept. 2020, it is focusing on attacking the inference infrastructure market with its commercial product ATOM-Max and its next-generation data center AI Semiconductor, Rebel100. The company is valued at 3.4 trillion won, with cumulative fundraising of 1.3 trillion won. Arm, SKT, SK hynix, Samsung, the Public Growth Fund, and Aramco are among the major investors.
Rebellions has emphasized sovereign AI since its launch. Sovereign AI refers to a structure in which a nation or corporations do not overly depend on a specific overseas cloud or a single big tech company and can control the necessary AI infrastructure and data on their own. As AI spreads into industrial, administrative, security, and communications infrastructure, where and on what semiconductors and servers a country operates its own data and AI services has become a critical issue.
It is difficult to fully meet each government's and institution's demands for Data Sovereignty, expense, power efficiency, and localization with general-purpose GPU servers alone. Rebellions believes it can address this structure through cooperation with Arm and SKT. Jeong said, We will build sovereign AI in the truest sense and prepare tailored solutions for global telecom companies, adding, It is meaningful in that we can supply an integrated next-generation AI infrastructure standard to the global market that safeguards each country's Data Sovereignty.
Jeong said sovereign AI should not be understood as a closed structure that uses only a specific country's technology. He said, Many understand sovereign AI simply as using only domestic technology, adding, The essence of the problem is falling into a state where, as dependence on a specific country or a single big tech company grows, you lose bargaining power and cannot shape the paradigm of the AI industry yourself. He added, Sovereign AI ultimately comes down to leadership in setting the rules and to bargaining power.
To the question of whether partnering with an overseas company like Arm runs counter to sovereign AI, he answered, On the contrary, it aligns. Jeong said, If the problem is being subordinated to a single big tech company, the solution is to build a new ecosystem with diverse global partners, adding, Rebellions forming an alliance with Arm and SKT is in the same vein.
◇ Shift from training to inference… GPU alone carries a heavy expense burden
In the early market when Generative AI services emerged, there was fierce competition among big tech companies to secure training GPUs to build bigger models. But as AI services spread into industry and everyday life, how cheaply, quickly, and reliably you can operate the model has become the more important issue.
Jeong cited AI agents—AI that autonomously performs multiple tasks—as an example of this shift. AI agents go beyond answering a single question to perform a series of tasks such as search, coding, document creation, and scheduling. He said, For developers, coding agents are already essential goods, and added, As agents that handle complex tasks and assignments become common, far more inference computation will be needed than now.
Jeong explained that this is why demand for NPUs is also surging. While GPUs have strengths in general-purpose AI computation, NPUs are products focused on handling repetitive AI inference tasks with low power. He said, The differentiator of Rebellions' NPU is delivering GPU-level usability while achieving higher energy efficiency in AI inference, adding, Through cooperation with Arm and SKT, we will be able to respond more proactively to changes in the AI infrastructure market.
◇ Ecosystems matter more than competitors… AI infrastructure is a comprehensive art
RebelCard is the key component of this collaboration. RebelCard refers to a card-type product built with Rebellions' next-generation AI Semiconductor, Rebel100. Rebel100 is designed by combining four NPU chiplets (a structure that bundles small chips to use as one) with fifth-generation high-bandwidth memory (HBM3E, high-performance memory for AI computation) to perform large-scale AI inference functions.
Jeong said, Combining Rebellions' NPU specialized for AI inference with Arm CPUs offers the advantage of lower energy expense for the same performance—or higher performance for the same energy—compared with existing GPU servers.
SKT plans to validate this server at its own AI data centers. Rebellions has already run SKT's independently developed foundation model, A.X K1, on a Rebellions server equipped with RebelCard.
Jeong said, Not only the performance of the hardware itself but also the completeness of the serving software that ensures stable operation in real service environments is important, adding, Based on infrastructure capabilities comparable to high-performance GPU servers, our experience efficiently running ultra-large models on a single server will be applied to building next-generation AI infrastructure. Rebellions acquired SqueezeBits, a company specializing in AI model compression, in June to strengthen its serving software capabilities.
Jeong graduated from Seoul National University's Department of Electrical and Computer Engineering and earned a master's in statistics and a Ph.D. in applied physics from Harvard University. After serving as a director at Bain & Company, Jeong now oversees Rebellions' global business strategy and growth direction.