Consortium to develop Ultra Ethernet standard

A new consortium is bringing together major companies to develop an Ethernet-based communications architecture for high-performance networks. The Ultra Ethernet Consortium (UEC) recognizes that Artificial Intelligence (AI) and High Performance Computing (HPC) workloads are evolving rapidly and demanding more functionality, performance, interoperability, and total cost of ownership. The Ultra Ethernet solution stack will exploit the ubiquity and flexibility of today’s Ethernet to address these and other types of performance-demanding workloads while remaining cost-effective.

“It’s not about redesigning Ethernet, but tuning it to increase efficiency for workloads with specific performance request. We are analyzing every layer, from physical to software, to find a way to improve efficiency and performance outcomes at scale,” explains J. Metz, chairman of the Ultra Ethernet Consortium.

The core founding members of the Ultra Ethernet Consortium are AMD, Arista, Broadcom, Cisco, Eviden (Atos Business), HPE, Intel, Meta, and Microsoft. The group will work to minimize changes to the communications stack, as well as maintain and promote Ethernet interoperability. The technical objective of the consortium is to develop specifications, APIs, and source code to define:

• Protocols, electrical and optical signalling characteristics, APIs and/or data structures for Ethernet communication;
• Transport protocols at the end-to-end link level to extend or replace existing transport an
• Congestion, telemetry and signalling mechanisms at the end-to-end link level, each suitable for AI, machine learning and high-performance computing environments.
• Software builds, storage, management and security to facilitate diverse workloads and operating environments.

The consortium will follow a systematic approach with modular, compatible and interoperable layers with tight integration to result in a holistic improvement for performance demanding workloads. UEC founding members are making highly valuable contributions in four working groups: Physical Layer, Link Layer, Transport Layer and Software Layer.

The UEC is a Joint Development Foundation project supported by The Linux Foundation. Applications for new members will begin to be accepted in Q4 2023 at ultraethernet.org.

Ethernet advantage According to the UEC, many large clusters with hyperscale deployments of GPUs used for AI training are already operating over Ethernet networks, exploiting advantages such as:

• Extensive vendor ecosystem of interoperable switches, NICs, cables, transceivers, optical systems, management tools and software;

• IP network addressing and routing that enables the construction of networks at various scales, from racks to data centres;

• Broad spectrum of tools to efficiently test, measure, deploy and operate Ethernet networks;

• Cost reduction based on a competitive ecosystem and economies of scale;

• IEEE Ethernet standards with the ability to move quickly and regularly across many physical and optical layers.

Even given these characteristics of today’s Ethernet, these networks are being required to deliver unprecedented levels of performance, scale, and bandwidth. According to the UEC, “tail latency” must be minimized and to achieve this, the specification needs to be significantly improved by addressing the following requirements:

• Multipathing and pulverisation of packages
• Flexible delivery requests
• More modern congestion control mechanisms
• End-to-end telemetry
• More scale, stability and reliability

The consortium expects the first products compatible with the new Ethernet standard to hit the market in 2024.