SK hynix presented the development path of "CPO (co-packaged optics)," a next-generation optical connection technology aimed at reducing data transfer bottlenecks between artificial intelligence (AI) accelerators and servers. CPO is a technology that places an optical transceiver in the same package as the processor to send and receive data with light instead of traditional copper wiring.
SK hynix said on the 20th that it published, with global researchers, a paper analyzing CPO technology for AI and high-performance computing (HPC) titled "Co-packaged optics for high-performance computing and artificial intelligence" in the international journal Nature Electronics.
In the study, Hong Seung-hun, SK hynix AI infrastructure Head of Team, and Lee Gyu-sang, a professor in the Department of Electrical and Computer Engineering at the University of Virginia, served as corresponding authors. Research teams from the University of Illinois Urbana-Champaign (UIUC), Nanyang Technological University (NTU), the Massachusetts Institute of Technology (MIT), and Yonsei University also took part in the joint research.
The researchers focused on CPO because, as AI systems grow, a "bandwidth wall" is emerging in which data movement speeds fall behind computational performance. Bandwidth refers to the amount of data that can be transmitted over a given period.
According to the paper, computational performance is increasing about threefold every two years, while data transfer bandwidth is growing only about 1.4 times over the same period. In large-scale AI systems that connect thousands of graphics processing units (GPUs) and high bandwidth memory (HBM), overall performance is determined not only by the computational capability of the chips themselves but also by how quickly data can be moved between the chips and the servers.
Data centers currently use copper wiring mainly for short-distance data transfer. But as transmission distance and speed increase, signals weaken, and the power consumption and latency required to compensate for that grow.
Professor Lee said, "Even if compute-chip performance improves, the system's overall performance will not rise if the ability to move data between chips cannot keep up," adding, "Replacing copper wiring, which faces physical limits, with light to move data is the most promising scaling path."
To reduce these problems, CPO places the optical transceiver that converts the electrical signals of data into light close to the processor. It shortens the distance high-speed electrical signals must travel and connects the rest of the path with optical signals.
Head of Team Hong explained CPO as "a technology that puts an optical transceiver (TRx) in the same package as the processor so chips can send and receive data with light instead of long electrical wiring."
The researchers said that using CPO can expand data transfer ranges and reduce power consumption not only between chips but also between racks where servers are installed and between systems composed of multiple racks. A rack is a standard structure for installing data center equipment such as GPUs and servers.
The paper also presented target CPO performance for next-generation AI infrastructure. The goal is to transmit more than 100 terabits per second (Tb/s) of data from a single computing node while reducing energy consumption per bit to below 1 picojoule (pJ) and cutting inter-chip data transfer latency to under 10 nanoseconds (ns).
The packaging structure is also expected to evolve from today's planar placement toward bringing chips closer together. The researchers projected that CPO will advance from conventional two-dimensional (2D) packaging, which places chips on the same substrate, to 2.5-dimensional (2.5D), which mounts multiple chips on an interposer as an intermediate connection board, and to three-dimensional (3D) heterogeneous integration, which stacks chips with different functions vertically.
Over the long term, they also proposed extending optical connections not only to processors but also to memory. This is the so-called "optics-centric" architecture. It directly connects processors and memory with optical signals to reduce wiring and spatial constraints that currently arise within semiconductor packages.
Applying this architecture would allow multiple AI accelerators to form a "memory pool" that shares a single large memory resource. Instead of placing all the required memory around a specific accelerator, multiple accelerators would share it, enabling more flexible expansion of memory capacity as AI models grow in size.
Professor Lee said, "Extending optical connections to memory can bypass the physical constraints around compute chips and overcome limits on memory capacity and the number of interconnects," adding, "Multiple accelerators will be able to share a massive memory pool, responding much more flexibly to the scaling up of AI models."
There are still challenges to resolve for commercialization. Power consumption of optical devices must be reduced, and different semiconductors and optical devices must be reliably integrated into a single package. In mass production, issues such as Production yield, cost, cooling, and reliability must also be addressed.
Professor Lee said, "The relevant technologies have already moved out of the lab and entered the early stages of industrialization," adding, "There are many challenges to solve, from integrating low-power optical devices to coherence protocols and reliability technologies, but ultimately the key is co-designing memory devices and controllers, optical devices, and packages as a single system."
Head of Team Hong said, "It is meaningful that academia's expertise and industry's on-site experience came together to present a future direction for AI infrastructure," adding, "We will continue to expand open collaboration that can deliver tangible value to customers and the industrial ecosystem."