Every open standard needs a robust ecosystem if it is to be widely adopted—and that includes testing and verification capabilities. Just as vendors have rallied around the rapidly evolving Compute Express Link (CXL) specification to announce a variety of products, so have players offering tools to help make sure these products perform reliably and play nice with each other.

Making CXL testing and verification legwork easier is the fact that the interconnect runs on the Peripheral Component Interconnect Express (PCIe) bus standard, which is both ubiquitous and well-understood. PCIe also provides the underlying foundation for the rather mature Non-Volatile Memory Express (NVMe) specification.

While there is a lot of validation involved with CXL for an SSD maker like Phison Electronics, there is nothing that’s substantially divergent from PCIe validation, Phison CTO Sebastien Jean said in an exclusive interview with EE Times.

About 80% of the firmware revolves around NAND management, he said, and then there’s about 20% that is a chunk of code that’s protocol-specific. “From our perspective, it really is just another interface.”

The process of embracing CXL is figuring out what an SSD has to do so it can service memory read and write requests, Jean said.

Once you get past the protocol, it’s all about NAND management, which must be married to CXL in the back end. Servicing PCIe commands involves following a strict order in which commands are processed—typically when they’re received, he said. “But if you’re going through sufficiently complex topology, there are cases where certain commands are allowed to pass others, and those rules are very strict.”

CXL testing traits not unique

CXL is the same in that rules of the protocol must be applied, and it is a relatively simple protocol compared with other storage protocols that added a lot of needless complexity, Jean said. “The CXL protocol is very simple. It is very fast.”

PCIe is also very elegant in its simplicity, and its ubiquity means there’s a large pool of technology and expertise, including tools for testing and verification. “PCIe is the de facto interconnect bus,” Jean said, adding that CXL is essentially 90% PCIe with a few additional things, and not nearly as unapproachable and mysterious as it’s made out to be. “It’s not adding substantial complexity above things that we are already doing.”

Even with CXL’s versatility, which includes having three primary use-case categories, it’s not as complex as other storage protocols. (Source: CXL Consortium)

But while CXL testing and validation isn’t inherently complex, it’s still an enterprise-grade endeavor.

“You have to test it much more robustly,” Jean said. “Massive interoperability testing has to happen.”

Mark Orthodoxou, a marketing VP for Rambus’s interconnects business unit, said NVMe is also a good comparison when thinking about CXL, especially given that it also rides over PCIe. But CXL also harkens back to the early days of developing serial attached SCSI (SAS) and serial advanced technology attachment (SATA), he said in an exclusive interview with EE Times. “It feels even more greenfield in that sense than NVMe.”

Orthodoxou said the concept of tiering storage had already been introduced when NVMe emerged in 2011 to target flash and its specific requirements, whereas the concept of tiering in memory is even more nascent. Rambus is drawing on a long history of working with SAS, SATA and NVMe and sharing its expertise with the CXL Consortium’s Compliance Working Group.

In the same way that the PCIe Special Interest Group (SIG) has a list that tracks integrators and compliance to the specification, Orthodoxou said something similar for CXL is likely coming down the pipe, as well as testing workshops that attract vendors in the ecosystem, including CPU, memory, and controller makers as well as those making CXL analyzers and test equipment. In terms of timelines, he expects that most qualifications for CXL 2.0 will be completed by the end of the year.

But the timeline begs the question: Is testing and verification struggling to keep up, given that CXL 3.0 was introduced only last year?

Weighing interoperability concerns

Orthodoxou said the good news is that the reason the standards are moving so fast is because there is a real opportunity in the data center for the memory tiering that CXL provides to solve real problems. “You have a lot of voices introducing a lot of different needs and use cases,” he said. But, he added, there is some discussion within the Consortium that the standard is moving faster than members can make the silicon and the systems. “Maybe we need to pump the brakes a little bit on the standards development.”

And while CXL testing may have fewer growing pains because of its grounding in PCIe, Orthodoxou said the pace of evolution can translate into issues that Rambus is seeing in its lab testing. “It’s open to interpretation at a certain level,” he said. “There’s this kind of feature creep that happens from implementation to implementation because engineering change notices [ECNs] get added to standard versions. Then new standard versions introduce features that look like they’re really useful for the prior version.”

One challenge that’s come up is hybrid implementations: A device that’s CXL 2.0–compliant that goes into production may have 3.0 capabilities, Orthodoxou said, which reflects high-speed pace of the specification. Hybrid implementations and feature creep can hamper interoperability, which emphasizes the need for massive interoperability testing at the enterprise level, especially because the value of proposition of CXL is in the data center, where interoperability is table stakes.

CXL interoperability is the driving force behind Astera Labs’ expansion of its Cloud-Scale InterOp Lab to provide robust interoperability testing for its customers, Ahmad Danesh, the company’s associate VP of product management, told EE Times in an interview.

“When there’s new technology, interoperability is just so critical to being able to launch something at the scale,” he said.

Astera Labs had expanded its InterOp Lab capabilities to include CXL, enabling its customers to test everything from the physical layer all the way up to the application layer, including PCIe electricals, the DDR physical layer and the DDR controller. (Source: Astera Labs)

The company has its own CXL memory connectivity platform, dubbed Leo, and as it ramps up production, there’s a great deal of testing that needs to be done. In the data center, Danesh said, memory is a core component for server reliability, which is non-negotiable in high-availability systems.

When it comes to implementing CXL in a data center environment, the most fundamental change is the move away from a traditional path from the CPU to the DDR subsystem, which changes the paradigm of testing responsibilities, Danesh said. Low latency, reliability and security are all a part of Leo’s value proposition, “but the key part is that seamless interoperability.” Validating interoperability accelerates times to market for Astera’s customers, he said.

As a serial interface, CXL is easy to understand, and like many vendors, Astera has been working with PCIe and DDR links for some time. Putting them all together is what’s new to customers, Danesh said. “Ultimately, somebody needs to do the interop work.”

The InterOp Lab reduces the risk associated with interoperability, he said, allowing them to deploy CXL with confidence.

Astera customers can use the automated lab to test everything from the physical layer all the way up to the application layer, including PCIe electricals, the DDR physical layer and the DDR controller—scenarios include injecting errors and making sure they’re being corrected properly, Danesh said. Along with testing for compliance with the CXL specification, Astera does full system-level testing as well with industry-standard tools. He said components lead to different permutations.

Astera is just one of many vendors offering testing and validation options of CXL. Avery Design Systems, known for its functional verification offerings for various interfaces and specifications, has added CXL to its roster with a new validation suite to ensure system interoperability, validation and performance benchmarking of systems. The suite covers all versions of the CXL standard from 1.1 to 3.0, as well as both pre-silicon virtual and post-silicon system platforms.

Avery Design Systems, known for its functional verification offerings, has introduced a CXL validation suite covering all versions of the CXL standard from 1.1 to 3.0. (Source: Avery Design Systems)

Testing feedback loops back

No matter how smoothly testing and verification goes, it is all potential feedback for the CXL Consortium itself to incorporate into the specification, which has an incremental development process before it’s finalized, said Debendra Das Sharma, the Consortium’s technical task force co-chair and co-general manager of memory and I/O technologies at Intel, in an exclusive interview via email with EE Times. “The feedback comes in as an ongoing process through reviews as well as pre-silicon and post-silicon validation as members implement the specification through its evolution.”

He said any feedback coming in for clarification, such as a table to explain different permutations, or inconsistency gets addressed. “Any new feature request is considered based on the merit and the maturity of the specification.”

Any feedback that comes in after the final specification is published is also tracked, Das Sharma said, while a new feature request may show up as an ECN or for the next final specification, after discussion within the Consortium.

“ECNs are optional for the prior final specification but may become mandatory for the next final specification,” he added. That means an ECN that shows up after CXL 3.0 would be optional for devices designing to the CXL 3.0 specification but may become mandatory for CXL 4.0 devices later.

Despite the rapid advances the CXL specification has made within a relatively short period, Das Sharma said the Consortium’s Compliance chapter moves in tandem with the rest of the specification. “We have a fairly detailed Compliance chapter that we think through upfront.”

This is possible thanks to the experience of Consortium members who have a lot of experience driving products across a wide range of usages over the decades, he said. “We bring that to bear in the specification, so compliance-testing mechanisms keep up with the fast evolution and expansion of the specification, which is in response to the huge demand for CXL in the ecosystem to solve a wide range of problems.”

By EETimes