Ultra Ethernet 1.1 extends the specification to scale-up networks and adds programmable congestion control
Network World ·
The Ultra Ethernet Consortium (UEC) is pushing forward to meet evolving networking needs with the release of UEC 1.1 on October 12. The UEC got its start in 2023 to help improve Ethernet for the growing needs of AI and high-performance computing (HPC) networks. The consortium published its 1.0 specification in June 2025. That release defined a transport protocol, congestion control mechanisms and a modern remote direct memory access (RDMA) approach. With UEC 1.1 , the specification is adding scale-up support, in-network collectives, and programmable congestion control. Specifically, the improvements in UEC 1.1 include: Scale-up networks. The Unified Local profile covers short-range interconnects between accelerators, which often use one tier of switches or none. Packet efficiency. The Unified Forwarding Header and optional idle reduction cut per-packet overhead for small messages. In-network collectives (INC). Switches perform reduction operations such as allreduce for a group of endpoints. Congestion Signaling (CSIG). A packet tag carries congestion measurements from the bottleneck switch back to the sender. Programmable Congestion Management (PCM). Congestion control and load balancing algorithms run as programs loaded onto the endpoint. RDMA verbs mapping. Extensions to the RDMA verbs programming interface expose Ultra Ethernet transport to applications. “A big new focus of UEC 1.1 is extending support to scale-up networks,” Tom Emmons , Chair, UEC Technical Advisory Committee, told Network World . “This includes a simplified transport protocol appropriate for scale-up networks, along with a number of options to improve packet efficiency.” Scale-up networks need less overhead per packet Scale-up networks connect the accelerators inside a node, such as the GPUs in one server. Scale-out networks connect nodes across a cluster. Scale-up traffic consists of small messages at high packet rates, so per-packet overhead matters. A standard Ethernet frame carrying IPv4 and UDP data has 66 bytes of overhead before any transport header or payload. UEC 1.1 adds three critical updates to improve scale-up networking: Unified Local profile. This is a simplified transport for local networks. Requests in one direction also carry acknowledgment information for the other, which can reduce state and acknowledgment packets. It is designed for a lossless fabric built with priority flow control (PFC) or credit-based flow control (CBFC). Unified Forwarding Header (UFH). This optional header replaces the separate IP header. It carries addressing in the existing 12-byte Ethernet media access control (MAC) header and can share a link with standard Ethernet frames. A standard IPv4 header uses 20 bytes. Idle reduction. This optional feature shrinks the required gap between frames. Minimum frame spacing falls from an 84-byte average to a constant 80 bytes. UEC is working with the Open Compute Project (OCP) to bring these scale-up technologies into the next version of its Ethernet for Scale-up Networking (ESUN) specification. Congestion control gets better signals and programmable algorithms Congestion control is how a sender decides how fast to transmit. When several senders target the same link, switch queues fill and packets are delayed or dropped. The sender needs a signal to slow down and an algorithm that decides by how much. UEC 1.0 included congestion control mechanisms, though there were still at least two limits that needed to be addressed. First, the signal carries little information. An explicit congestion notification (ECN) mark is a single bit. It says a queue somewhere on the path is filling, but not where or by how much. Second, the algorithm that reacts to the signal is fixed in the NIC. It cannot be updated as AI workloads change. UEC 1.1 adds two changes to help address those limits. CSIG. Congestion Signaling adds a 4-byte or 8-byte tag to packets, and every switch on the path can update it. The tag holds one value, such as the longest queueing delay or the least available bandwidth seen so far. The receiver returns the value to the sender in an acknowledgment, so the sender sees the worst condition on the path. PCM. Programmable Congestion Management replaces the fixed algorithms with programs. UEC’s two baseline algorithms, Network Signal-based Congestion Control (NSCC) and Receiver-Credit Congestion Control (RCCC), remain and can use CSIG signals. “The goal of PCM is to ensure congestion control isn’t baked into the nic, but instead fully software configurable,” Emmons said. “This allows continued development on congestion control over the NIC’s full lifetime. AI workloads are constantly evolving, so there is a strong desire for Congestion Control Algorithms to be able to keep up.” In-network collectives move reductions into the switch AI and HPC jobs split work across many accelerators that must coordinate as a group. A collective operation is a step that involves the whole group. Allreduce is one example. Each accelerator contributes values, the values are combined into one result, and every accelerator receives that result. UEC 1.1 adds In-Network Collectives (INC), which lets a switch combine the values instead of the endpoints. An INC manager, which is a daemon, groups endpoints into a communicator. Each endpoint sends its data to the switch. The switch waits until every endpoint has contributed, then returns the results. The specification says this can cut the bandwidth needed for large reductions by a factor of two. In UEC’s example, each endpoint sends 1 GB instead of 2 GB to produce a 1 GB result. From specification to products As part of the 1.1 update, UEC worked on what vendors need to build products. Emmons described that work as, “dotting the i’s and crossing the t’s to enable companies to build UEC NICs.” “This includes the management plane, compliance and test support, fixing bugs and clarifying the spec, and adding additional minor features where necessary,” Emmons said. The specification covers each of those areas. The management chapter describes a controller that manages endpoints and switches. Servers expose a Redfish data model and switches expose a YANG data model. Vendors claim compliance by self-attestation. They must name the features and profiles they support and run the UEC checklist tests. Emmons said UEC cannot comment directly on interoperable products and that the matter is up to its member companies. “I believe all of the features in UEC 1.1 require some degree of hardware support,” Emmons said. “However, because these features have been in progress for quite some time, I expect companies to announce support for them on existing hardware via software and firmware updates.”
The Ultra Ethernet Consortium (UEC) is pushing forward to meet evolving networking needs with the release of UEC 1.1 on October 12. The UEC got its start in 2023 to help improve Ethernet for the growing needs of AI and high-performance computing (HPC) networks. The consortium published its 1.0 specification in June 2025. That release defined a transport protocol, congestion control mechanisms and a modern remote direct memory access (RDMA) approach. With UEC 1.1 , the specification is adding scale-up support, in-network collectives, and programmable congestion control. Specifically, the improvements in UEC 1.1 include: Scale-up networks. The Unified Local profile covers short-range interconnects between accelerators, which often use one tier of switches or none. Packet efficiency. The Unified Forwarding Header and optional idle reduction cut per-packet overhead for small messages. In-network collectives (INC). Switches perform reduction operations such as allreduce for a group of endpoints. Congestion Signaling (CSIG). A packet tag carries congestion measurements from the bottleneck switch back to the sender. Programmable Congestion Management (PCM). Congestion control and load balancing algorithms run as programs loaded onto the endpoint. RDMA verbs mapping. Extensions to the RDMA verbs programming interface expose Ultra Ethernet transport to applications. “A big new focus of UEC 1.1 is extending support to scale-up networks,” Tom Emmons , Chair, UEC Technical Advisory Committee, told Network World . “This includes a simplified transport protocol appropriate for scale-up networks, along with a number of options to improve packet efficiency.” Scale-up networks need less overhead per packet Scale-up networks connect the accelerators inside a node, such as the GPUs in one server. Scale-out networks connect nodes across a cluster. Scale-up traffic consists of small messages at high packet rates, so per-packet overhead matters. A standard Ethernet frame carrying IPv4 and UDP data has 66 bytes of overhead before any transport header or payload. UEC 1.1 adds three critical updates to improve scale-up networking: Unified Local profile. This is a simplified transport for local networks. Requests in one direction also carry acknowledgment information for the other, which can reduce state and acknowledgment packets. It is designed for a lossless fabric built with priority flow control (PFC) or credit-based flow control (CBFC). Unified Forwarding Header (UFH). This optional header replaces the separate IP header. It carries addressing in the existing 12-byte Ethernet media access control (MAC) header and can share a link with standard Ethernet frames. A standard IPv4 header uses 20 bytes. Idle reduction. This optional feature shrinks the required gap between frames. Minimum frame spacing falls from an 84-byte average to a constant 80 bytes. UEC is working with the Open Compute Project (OCP) to bring these scale-up technologies into the next version of its Ethernet for Scale-up Networking (ESUN) specification. Congestion control gets better signals and programmable algorithms Congestion control is how a sender decides how fast to transmit. When several senders target the same link, switch queues fill and packets are delayed or dropped. The sender needs a signal to slow down and an algorithm that decides by how much. UEC 1.0 included congestion control mechanisms, though there were still at least two limits that needed to be addressed. First, the signal carries little information. An explicit congestion notification (ECN) mark is a single bit. It says a queue somewhere on the path is filling, but not where or by how much. Second, the algorithm that reacts to the signal is fixed in the NIC. It cannot be updated as AI workloads change. UEC 1.1 adds two changes to help address those limits. CSIG. Congestion Signaling adds a 4-byte or 8-byte tag to packets, and every switch on the path can update it. The tag holds one value, such as the longest queueing delay or the least available bandwidth seen so far. The receiver returns the value to the sender in an acknowledgment, so the sender sees the worst condition on the path. PCM. Programmable Congestion Management replaces the fixed algorithms with programs. UEC’s two baseline algorithms, Network Signal-based Congestion Control (NSCC) and Receiver-Credit Congestion Control (RCCC), remain and can use CSIG signals. “The goal of PCM is to ensure congestion control isn’t baked into the nic, but instead fully software configurable,” Emmons said. “This allows continued development on congestion control over the NIC’s full lifetime. AI workloads are constantly evolving, so there is a strong desire for Congestion Control Algorithms to be able to keep up.” In-network collectives move reductions into the switch AI and HPC jobs split work across many accelerators that must coordinate as a group. A collective operation is a step that involves the whole group. Allreduce is one example. Each accelerator contributes values, the values are combined into one result, and every accelerator receives that result. UEC 1.1 adds In-Network Collectives (INC), which lets a switch combine the values instead of the endpoints. An INC manager, which is a daemon, groups endpoints into a communicator. Each endpoint sends its data to the switch. The switch waits until every endpoint has contributed, then returns the results. The specification says this can cut the bandwidth needed for large reductions by a factor of two. In UEC’s example, each endpoint sends 1 GB instead of 2 GB to produce a 1 GB result. From specification to products As part of the 1.1 update, UEC worked on what vendors need to build products. Emmons described that work as, “dotting the i’s and crossing the t’s to enable companies to build UEC NICs.” “This includes the management plane, compliance and test support, fixing bugs and clarifying the spec, and adding additional minor features where necessary,” Emmons said. The specification covers each of those areas. The management chapter describes a controller that manages endpoints and switches. Servers expose a Redfish data model and switches expose a YANG data model. Vendors claim compliance by self-attestation. They must name the features and profiles they support and run the UEC checklist tests. Emmons said UEC cannot comment directly on interoperable products and that the matter is up to its member companies. “I believe all of the features in UEC 1.1 require some degree of hardware support,” Emmons said. “However, because these features have been in progress for quite some time, I expect companies to announce support for them on existing hardware via software and firmware updates.”