Advances in optical interconnections are vital for disaggregated datacenters

In early March, Ayar Labs, a provider of optical connectivity solutions for data-intensive applications, gave a public demonstration of what it claims is the first compact CMOS chiplet with 4 terabits per second (Tbps) transfer capability. The company now provides integrated optical I/O solutions with latency below 10 nanoseconds and less than 10 watts of power consumption to major customers in semiconductor, Artificial Intelligence, high-performance computing (HPC), and aerospace. Optical I/O systems like this are critical for datacenters, among other use cases.

Optical interconnectivity for datacenters may become increasingly vital in today’s data explosion landscape, hungry for high-speed, high-bandwidth connections. More and more users, more Internet of Things (IoT) devices or not, more services and more accesses could not result in anything other than high demand for performance-, energy- and cost-efficient connectivity, characteristics offered by optical interconnectivity solutions.

In parallel, datacenters are also being composed of more and more heterogeneous and customized hardware capable of meeting different computing needs. As a result, we are seeing a movement of many large datacenters toward disaggregated architectures in which different hardware resources such as memory and storage are organized independently and connected to a mesh of networks, the so-called disaggregated datacenters.

The approach of unbundling datacenter components can be a way to enable more processing power for massive workloads, while ensuring more flexibility, scalability, and energy efficiency. With disaggregation, datacenter architecture can be reorganized to make better use of often underutilized computing resources, for example by allocating only the processing power, memory, and storage system required for a specific job. The remaining resources can be allocated to other tasks.

Resource unbundling is opposed to the resource utilization method employed in most modern datacenters, which relies on a fixed amount of connected elements (e.g. CPUs and memory) in blade servers.

In today’s datacenters, thousands of servers are deployed in different racks and connected to switches by network cards. Each server has a fixed amount of resources, the static configuration of which leads to pictures of underutilization of processing power. For example, a task that makes intensive use of video processing can consume a lot of CPU resource, while much of the memory on the same server cannot be allocated to other tasks. Also, integrating resources into a single chassis can make it difficult to change or modernize the configuration of the servers.

In the disaggregated scenario, the critical element is connectivity, which must meet the bandwidth and latency requirements of datacenter communication. Recent advances in optical systems enable high-bandwidth, low-power-per-bit reconfigurable interconnections for disaggregated datacenters.

Datacenter disaggregation has been in use for a few years now, but CPU and memory resources are still generally coupled. More recently, the concept of a fully disaggregated architecture has proposed to migrate from using ‘boxes’ integrating different types of resources to using the same types of resources shape a unit (a blade, a rack, or a resource cluster). Such units are interconnected to allow communication between the different types of resources.

In particular, CPU-to-memory interconnections in disaggregated datacenters require ultra-low latency and ultra-high throughput bandwidth to avoid performance degradation when processing workloads. To get an idea, latency requirements for different types of resources range from milliseconds (for example, between CPU and storage systems) to nanoseconds (between CPU and memory). For this reason, communication via optical systems is a promising technique for providing high throughput and low latency.

The main benefits of optical interconnections for disaggregated data centers are:

1. High bandwidth that allows fast transfer of large volumes of data between different datacenter components.

2. More flexibility and scalability.

3. Low-power consumption, as optical interconnects consume less power than copper ones.

4. Better reliability and redundancy, since disaggregated data centers can use different types of interconnections for different components.

5. Economy, because optical interconnects are cheaper than copper ones.

Revenue generated by optical I/O systems for high-performance computing (HPC) was about $5 million in 2022 and is expected to reach $2.3 billion by 2033, with a compound annual growth rate of 74% over the period, according to Yole Group.