Below you will find a collection of frequently asked questions and answers. If you have additional questions or would like to meet with a Consortium representative, please Contact Us.
An open systems interconnect designed to provide memory semantic access to data and devices via direct-attached, switched, or fabric topologies.
Yes, Gen-Z is developed by the Gen-Z Consortium, an open industry consortium incorporated and formally formed on August 18, 2016. The Gen-Z Consortium continues its work to develop the Gen-Z specifications. Refer to our Members List to see the list of current consortium members.
Gen-Z was developed to enhance existing solution architectures, and enable new solution architectures, while delivering new levels of performance (high-bandwidth, low-latency), software efficiency, power optimizations, security, and industry agility.
Customers are demanding new levels of performance, functionality, security and isolation as well as ways to unlock innovation and execution agility. After many months of investigation, the member companies determined that a new, comprehensive data access technology was required; one that could support a wide range of new storage-class memory media, new hybrid computing technologies, new memory-centric solution architectures, and a wide range of applications using a highly-efficient and performance-optimized solution stack.
Gen-Z provides the following technical benefits:
- Memory media independence: Gen-Z abstracts the memory media to enable any type and mix of DRAM and non-volatile memory (NVM) to be directly accessed by applications or through block-semantic communications.
- High-bandwidth, low-latency: Very efficient, memory-semantic protocol that simplifies hardware and software designs, reducing solution cost and complexity. Gen-Z supports a wide range of signaling rates and link widths that enable solutions to scale from tens to several hundred GB/s of bandwidth with sub-100 ns load-to-use memory latency.
- Multipath: Multipath increases aggregate component performance, enables very high signaling rates enables solution resiliency to prevent loss of data access, and enables traffic segregation to allow services and applications to be isolated from one another.
- Scalability: Scales from the simple point-to-point optimized solutions to rack-scale, switch-based topologies without sacrificing performance or solution flexibility.
- Security and isolation: Supports a combination of hardware-enforced isolation techniques and full packet authentication to prevent errant or malicious components from communicating with unauthorized components or accessing unauthorized resources, e.g., memory and storage.
- Advanced workloads and technologies: Supports hybrid and data-centric computing to deliver high-performance and power-optimized solutions.
- Mechanical compatibility: Can be incorporated into any solution using a wide range of existing mechanical form factors and cables. This enables Gen-Z to be integrated into existing platforms and solution stacks. Further, Gen-Z specifies a new scalable connector that provides significant improvements in signal integrity and density as well as supports all signaling rates from day one.
- Software Compatibility: Gen-Z supports unmodified operating systems, unmodified applications and middleware and unmodified I/O device drivers to enable Gen-Z to be integrated into existing solutions tacks and take advantage of many of Gen-Z’s capabilities, and with some enhancements, software can take full advantage.
- High-speed signaling rates: Gen-Z has the immediate capability of 56 GT/s with a path to 112 GT/s and beyond. Higher-signaling rates not only increase bandwidth, but also enable solutions to significantly reduce package pin counts, e.g., a solution using 56 GT/s requires one third the number of pins and a solution using 112 GT/s requires one seventh the number of pins to deliver the equivalent bandwidth of one using 16 GT/s.
- High-efficiency protocol: Gen-Z is ~80% efficient when transferring 64-byte data payloads and ~93% efficient when transferring 256-byte data payloads in point-to-point topologies. In switch-based topologies, Gen-Z is ~70% efficient when transferring 64-byte data payloads and ~90% efficient when transferring 256-byte data payloads (regardless of topology scale). High protocol efficiency provides greater performance and reduces the power required to deliver a given bandwidth.
Gen-Z provides the following industry and customer benefits:
- Gen-Z abstracts memory media from the memory controller to enable the industry to deploy a wide range of memory media without waiting for the industry to move in lock step. This enables new media types or multiple generations of a given media type to be transparently supported in any solution. It also enables customers to independently replace and upgrade components based on their needs— e.g., processors, memory modules, NVM modules, etc.
- Gen-Z specifies a highly-efficient and flexible protocol capable of supporting a wide range of application needs. Gen-Z simplifies hardware and software designs, reducing complexity, overheads, and end-to-end latency. Simple, efficient solution stacks ease deployment and security, speed innovation, and reduce development and support costs.
- Gen-Z supports a robust hardware-enforced isolation and security framework to help protect customer solutions from cyber threats. Components and resources can be isolated to prevent unauthorized access. Data plane and control plane communications can be fully authenticated to prevent tampering and anti-replay attacks. Further, data payloads can be encrypted using industry or customer-driven algorithms and policies (encryption is performed within the source and destination components).
- Gen-Z interoperability enables the architecture to simultaneously and efficiently transport standard and customized communications between components. This enables customers and vendors to rapidly innovate and deploy new capabilities and services without waiting for the industry to move in lock-step.
Gen-Z promotes innovation in multiple ways:
- Gen-Z breaks the processor-memory interlock and enables new types of memory media to be transparently deployed at an accelerated rate.
- Gen-Z supports a wide variety of component types including processors, memory modules, FPGAs, GPUs / GPGPUs, DSPs, I/O, accelerators, NICs, custom ASICs, and many more.
- Gen-Z supports a wide range of physical layer signaling rates and types (electrical and optical). This enables hardware to optimize performance while minimizing package costs, to scale to any bandwidth and distance within an enclosure or across data center racks, and to provision multiple paths to provide aggregate performance and resiliency.
- Gen-Z is processor agnostic. Solutions can be flexibly composed of any mix of processor types and capability. Further, Gen-Z specifies a common atomic protocol to ensure interoperability between any processor and any component type.
- Gen-Z supports traditional processor-centric and new memory-centric solution architectures.
- Gen-Z can be inserted into existing processor-centric solution architectures. This immediately enables any solution to reap Gen-Z’s benefits.
- Gen-Z supports new memory-centric architectures that enable any-to-any communication among all component types. Memory-centric architectures minimize data movement, reduce power consumption, reduce latency, and increase data access parallelism. Memory-centric architectures take advantage of Gen-Z multipath capabilities to increase aggregate performance and resiliency and enable new services.
All finalized Gen-Z specifications will be made publicly available to all through the Gen-Z Consortium website.
- Draft specifications will be made available to all member companies.
- Select draft specifications and technical proposals are periodically made available to the public to elicit direct feedback and input into the specifications and technologies under development. See our specifications page for the latest specifications.
- The Core Specification 1.0, covering the architecture and protocol, was published in 2018. Additional specifications covering firmware, mechanical form factors, physical layers, etc. are also available.
Gen-Z specifies a packet protocol. The protocol packets are transported across serial links that connect components. The Gen-Z protocol uses memory-semantic communications to move data between the components.
Memory-semantic communications are used to move data between memories located on different components with minimal overhead. For example, Gen-Z-attached memory can be mapped using a processor memory management unit (MMU) such that any processor load, store, or atomic operation is transparently translated into Gen-Z read, write, or atomic operation and transported to the destination memory component. Similarly, Gen-Z supports buffer put and get operations to move up to 232 bytes of data between buffers without any processor involvement.
Memory-semantic communications are extremely efficient and simple. Efficiency and simplicity are critical to delivering optimal performance and power consumption, which directly impact customer capital and operational costs.
Each component can support up to 264 bytes of addressable memory. Gen-Z can scale from as few as two components to up to 4096 components per subnet. In theory, the architecture supports up to 276 bytes of memory per subnet and up to 216 subnets.
To access data, the component and associated address ranges are advertised to the communicating components. Once advertised and mapped / configured into each communicating component’s address space, the components use memory-semantic communications to directly access each other’s data. Components use low-latency read and write operations to directly access up to 256 bytes of data request,and use a variety of advanced operations to move up to 232 bytes of data with minimal application or processor involvement.
Gen-Z supports point-to-point, daisy-chain, mesh, and switch-based topologies.
Gen-Z delivers maximum performance without sacrificing flexibility. This enables solutions scalability and flexibility in choice of:
- Interconnect topology and routing algorithms
- Switch design and implementation. Gen-Z supports a wide range of switch radix, buffer capacities, virtual channels, etc. from small to rack-scale solutions.
- Latency will depend upon switch radix. Small radix switches should deliver 10-30 ns latency and large radix switches should deliver 40-60 ns latency
Gen-Z components and existing solution stacks can be transparently supported by unmodified operating systems and application middleware. For example:
- A Gen-Z memory component that supports DRAM or byte-addressable storage class memory media is mapped into an unmodified operating system the same way that a DRAM DIMM component is mapped. Once mapped, the Gen-Z memory component is accessed by applications just as any other DRAM memory component is accessed.
- A Gen-Z I/O component is mapped into an unmodified operating system as a conventional PCI or PCI Express© device. Once mapped, the Gen-Z I/O component can be associated with any existing, unmodified I/O stack and accessed just as a PCI or PCI Express device is accessed. This enables a wide range of I/O device types to be supported through the Gen-Z interconnect.
- A Gen-Z block storage component is mapped into an unmodified operating system and unmodified storage stack either as a PCI NVM device (e.g., NVM Express) or through an I/O host bus adapter (e.g., SAS). Once mapped, the unmodified storage stack is used to access the Gen-Z block storage component as it would any other block storage component.
- A software-emulated NIC (eNIC) is mapped into an unmodified operating system and unmodified IP network stack to enable traditional network applications to transparently operate across a Gen-Z topology. One implementation option is to augment existing virtualized NIC (vNIC) implementations to operate across the topology as though the communication were local to the virtual machine or operating system.
- To support high-speed, low-latency communications, a Gen-Z provider library is mapped underneath the OpenFabrics’ OFI Libfabric infrastructure. This enables a broad set of advanced messaging applications to operate across Gen-Z, e.g., MPI, SHMEM, Sockets, and many more.
All of the above enable Gen-Z components and existing solutions to be constructed using unmodified operating systems and application middleware. As components and solution stacks evolve to take advantage of advanced Gen-Z capabilities and topologies, software changes will be required. For example, rack-scale composable solutions will require new software to manage and coordinate shared memory pools.
Gen-Z specifies a physical layer abstraction that enables it to support multiple physical layers without impacting the higher-level functional blocks. This enables faster deployment of new physical layers and signaling rates. It also enables co-packaged solutions, discrete electrical solutions, and discrete optical solutions.
Gen-Z supports the PCI Express Physical layer. This enables processors, accelerators and I/O components that currently support PCI Express to quickly and readily integrate Gen-Z functionality. Signaling rates from 2.5 GT/s to 32 GT/s can be supported.
Gen-Z builds upon the IEEE 802.3 electrical layer specification. signaling rates from 25 GT/s to 112 GT/s PAM 4 can be supported
To reduce power consumption, complexity, and cost, Gen-Z optimizes the 802.3 electrical layer to support different loss budgets and topologies, e.g., multiple loss budgets for chip-to-chip within an enclosure and fabric-distance loss budgets to connect multiple enclosures. In low-loss solutions, up to 80% power and cost reductions can be achieved.
Gen-Z architecture supports from 1-256 lanes per link. The number of provisioned lanes will vary based on solution requirements and the choice to avoid complex skew compensation technologies. In general, at 56 GT/s and higher signaling rates, the links will be narrower. Though this might limit the maximum bandwidth per link, Gen-Z supports multiple links per component, and thus can scale bandwidth to hundreds of GB/s and beyond.
Gen-Z supports symmetric links where the number of transmit and receive lanes are equal. Gen-Z supports asymmetric links where the number of transmit and receive lanes are not equal. Asymmetric links provide numerous advantages:
- For read-intensive applications, more receive lanes can be provisioned to increase read bandwidth. For example, many applications have a 3:1 read-write ratio, i.e., they issue three read requests for each one write request. Shifting half of the transmit lanes to be receive lanes can increase read bandwidth by
50%. - Asymmetric links enable the same physical infrastructure to deliver greater read or write bandwidth, i.e., solutions do not need to use larger connectors, cables, etc. to deliver greater bandwidth. This enables platforms to be adapted to a wider variety of application needs.
- Asymmetric links increase solution resiliency. For example, if a transmit lane fails, only transmit bandwidth is impacted. Receive bandwidth is not impacted since the number of active receive lanes does not need to be reduced to match the number of transmit lanes.
Gen-Z can scale from as few as two components to enclosure to rack scale and beyond. The architecture supports up to 4096 components per subnet and up to 64K subnets. Initial products will be focused on simple, enclosure and rack-scale solutions.
Yes. Gen-Z is designed to work with a wide range of existing, high-volume mechanical form factors and cables including u.2, PCIe CEM, etc. Gen-Z will work with the new DDIMM (Differential DIMM) mechanical form factor for high-speed memory.
Yes. Gen-Z developed a scalable connector specification.
- This scalable connector supports multiple component types and use cases, including: memory (DRAM / NVM), FPGAs, GPUs / GPGPUs, DSPs, I/Os, NICs, SSDs, etc. With sufficient volume, current estimates indicate connector cost should be similar to existing low-cost, high-volume connectors.
- The scalable connector is a high-density connector (0.6 mm pitch) that minimizes board space consumption. Connector size will vary with the number of provisioned lanes, e.g., a NVM module that delivers 10+ GB/s of read and 10+ GB/s of write bandwidth would require 4 transmit and 4 receive lanes operating at 25 GT/s. In contrast, a high-speed memory or I/O module that delivers 100+ GB/s of read and 100+ GB/s of write bandwidth would require 16 transmit and 16 write lanes operating at 56 GT/s. Though these examples illustrate symmetric links, the modules could support asymmetric links to enable greater read or write bandwidth without requiring a larger connector or moving to a higher signaling rate.
- There are three connector sizes:
- 1C provides 80W of power, management, and 8 differential pairs. Each differential pair can be used as a transmit lane or as a receive lane.
- 2C is an extension of the 1C, and provides a total of 16 differential pairs.
- 4C is an extension of the 2C, and provides a total of 32 differential pairs.
- To increase mechanical interoperability, the scalable connector is modular, i.e., any form factor can be inserted into any slot independent of the connector size, e.g., a form factor that supports 32 differential pairs can be inserted into a 1C, 2C, or 4C connector.
- The scalable connector supports vertical and right-angle insertion.
- The scalable connector enables solutions to scale from 2.5 G1T/s NRZ to 112 GT/s PAM 4 signaling.
- The scalable connector supports cable solutions (which will become increasingly important as signaling rates increase and modular solutions become more pervasive).
One of the primary goals of the Gen-Z Consortium is to create an open ecosystem and standards body where all companies can come together to quickly develop new technology. For example, in just a few months, the Gen-Z Mechanical Work Group developed the Gen-Z Scalable Connector’s requirements, translated these into connector concepts, and completed a high-quality draft 0.9 specification. The Consortium also recently published its Core Specification 1.0, which is available for public download.
From its inception, the Gen-Z Consortium wanted to maximize adoption of the Gen-Z Scalable Connector across market segments and use cases. The Gen-Z Consortium published the draft 0.9 specification to the industry in the hopes that multiple industry standards bodies and organizations would reference this connector, thus ensuring that all would benefit. However, due to concerns raised by some companies that are not members of the Gen-Z Consortium, the Board of Directors concluded that in the best interests of the industry, it would support submission of portions of the draft 0.9 specification to the SNIA / SFF organization as a new SFF connector specification, and reference this new SFF connector specification in all applicable Gen-Z Consortium specifications.
Yes. Gen-Z Consortium members have determined that a new mechanical form factor is required to address evolving solution needs. The proposed attributes are:
- The mechanical form factor will be modular to enable solutions to scale in the x, y, and z axis. This will increase solution flexibility and agility while ensuring interoperability across multiple use cases.
- The mechanical form factor can support multiple Gen-Z Scalable Connectors. Each connector provides incremental power, bandwidth and connectivity. For example, a mechanical form factor could support two 4C connectors that provide a total of 160W of power, 64 differential pairs, up to 8 links, etc. Support for multiple connectors simplifies solution development and eliminates mechanical complexity not possible when using alternative technologies.
- Gen-Z architecture enables solutions to scale to 1024 W. Similar to other technologies, a portion of the power will be delivered through the scalable connector(s) and remaining power will be delivered through separate high-capacity power cable attached to the power supply.
Yes. Gen-Z can be used in discrete and co-packaged solutions. In the near-term, the industry will generally focus on discrete solutions using existing mechanical form factors and evolve to use the new mechanical form factor.
Yes. Gen-Z supports cache coherency in point-to-point, meshed, and switch-based topologies. Cache coherency can be used between processors with accelerators, accelerators with accelerators, or accelerators with memory and storage.
Yes. To improve performance and to reduce complexity, Gen-Z cache coherency enables components to selectively determine which communications are coherent and which are non-coherent. For example, communication between a processor and an accelerator could be entirely coherent while communications among a set of accelerators can be selectively coherent.
Gen-Z supports PCI and PCIe technology as follows:
- Gen-Z supports the PCIe physical layer. This enables PCIe solutions to integrate Gen-Z with minimal disruption.
- Gen-Z supports PCI and PCIe configuration space. This enables an unmodified operating system to transparently use Gen-Z I/O components.
- Gen-Z components and switches may support PCIe Compatible Ordering (PCO). This enables an unmodified device driver to be transparently used with a Gen-Z I/O component. Though PCO solutions are constrained by the PCIe architecture, e.g., a single interface and a single path between an I/O component and a host system, they can still take advantage of a subset of the Gen-Z capabilities, e.g., low-latency switching. In contrast, non-PCO solutions can take full advantage of Gen-Z’s capabilities.
Yes. I/O components can take full advantage of Gen-Z capabilities:
- Low-latency switching
- Memory-speed (multiple GB/s) processor-to-device communications
- security and fine-grain hardware-enforced isolation (any-to-any communication without compromise)
- Supports many more atomics than the three supported by PCIe
- Simplified single and multi-host I/O virtualization and sharing capabilities
- Multipath – transparent aggregation, resiliency, and robust topologies
- Very high-speed electrical and optical physical layers
- New Gen-Z Scalable Connector and modular form factors
- CPU-based data movers to enable new software paradigms
- Scale-up and scale-out connectivity and performance, e.g., enables up to 8192 devices per SoC
- Simplified hot-plug management and scaling, e.g., eliminates OS rebalancing complexity that inhibits scalability
- And many more.
Yes. Gen-Z supports multiple processor architecture Atomics including x86 / ARM / Power.
- Supports multiple data sizes – 8, 16, 32, 64, and 128-bit sizes
- Supports integer and floating-point data
- Supports arithmetic and logical operations
- Supports vector atomics, e.g., up to 32 32-bit data values
Yes. Gen-Z supports multiple collective operations including broadcast, barrier, scatter, gather, reduce, map reduce, etc. Further, these collective operations can be implemented in collective accelerators incorporated into a switch topology. Collective accelerators can improve performance, reduce the number of packets exchanged between components, and reduce the probability of congestion.
Yes. The Gen-Z Core specification describes multiple techniques to optimize load-to-use latency that can be incorporated into any design. The specifcation also describes techniques to enable media controller differentiation, solution-specific customizations, optimize power management, and much more.
Gen-Z was created to simplify and unify data access at nearly any scale. At a very high level, Gen-Z provides functionality that supports more topologies, more scalability, and more use cases than what is in the market today.
- Gen-Z supports multiple topologies – co-packaged, point-to-point, mesh, switch-based, and transparent router topologies within a single enclosure or at rack scale.
- Gen-Z supports very robust, flexible, and extensible data access protocol that can access any type and number of components including: processors, memory, I/O, accelerators, storage, FPGA, DSP, etc.
- Gen-Z supports two of the highest volume physical layers – the PCI Express physical layer (up to 16 GT/s today, and 32 GT/s in 2020) and an optimized version of the 802.3 electrical (up to 56 GT/s today, and 112 GT/s in 2020). Gen-Z supports electrical and optical physical media that can be used to support co-packaged, single enclosure, or rack scale solutions.
- Gen-Z uses a highly-optimized, memory semantic protocol that inherently supports multiple use cases including: memory (DRAM and Storage Class Memory (SCM)), I/O – Gen-Z native and logical PCI device support that scales beyond the PCI Express architectural constraints, block and hybrid SCM + block storage, (non-) coherent and hybrid accelerators, Ethernet-compatible networking, low-lateency messaging, coherency, etc.
- Gen-Z supports up to 64-way memory interleaving across a set of point-to-point or switch attached memory components. Memory interleave can be explicitly managed by a memory controller or can be transparently implimented to provide additional capabilities, e.g., multi-host support, highly-resilient memory, transparent management and mechanical servicing, tiered memory, etc.
- Gen-Z supports hardware-enforced isolation and built-in security.
- Gen-Z supports nearly any routing algorithm.
- Gen-Z supports congestion management.
Yes. The Gen-Z consortium strongly believes in developing an open ecosystem where members, the broader industry, and customers can work together to deliver robust, high-quality specifications that meet solution needs. The Gen-Z Consortium will periodically publicly post draft specifications and technical concepts to elicit input from the broader industry and directly from customers.
Yes. Member companies have created a variety of open source projects that are available to the broader industry. These are available on GitHub.
The Gen-Z Consortium is comprised of leading technology companies dedicated to creating and commercializing the Gen-Z technology. The founding consortium’s members include: AMD, ARM, Broadcom, Cray, Dell EMC, Hewlett Packard Enterprise, Huawei, IDT, Micron, Samsung, SK hynix, and Xilinx. Refer to our Members List to see the full consortium membership.
The Gen-Z Consortium mission is deliver a suite of specifications to enable Gen-Z to be integrated into any solution. The Gen-Z Consortium will establish an open ecosystem where customers and the industry can directly engage with one another in creating these specifications.
The Gen-Z Consortium is formed as a not-for-profit corporation. To join, a company executes a Membership Agreement. Twelve member companies hold Board of Director seats and provide overall governance. Board seats are voted on annually. All general member companies may participate in any Work Group to develop the Gen-Z specifications and other materials. For more information on becoming a member of the Gen-Z Consortium please get in touch with our administrative team through our Contact Form.
The Gen-Z Consortium owns the copyrights and all of its final published specifications.
The Gen-Z Consortium made its public announcement on Tuesday October 11th, 2016.
The present annual general membership fee is $10,000. Refer to our Members Page for additional membership information.
No, the technical specifications are available via the consortium web-site free of charge. The Gen-Z Consortium does not charge an end user or implementer fee.
A new industry body was created to enable customers and the industry to directly work together in an open ecosystem. This requires a new organizational structure and engagement model.
No. Solutions will continue to incorporate existing interconnects where needed.
Please see member companies for their specific product plans. The expectation is initial products could appear in the 2019-2020 time frame. Multiple member companies have demonstrated solution concepts that highlight the benefits from using Gen-Z technology.
Each technology has been created to meet specific solution needs in terms of functionality, performance, scale, etc. The Gen-Z Consortium views these new technologies as complementary, open industry standards that will enable customer choice and drive innovation.
