What is NVMe, and how is it changing enterprise storage

NVMe (non-volatile memory express) is shaking up the enterprise storage industry.

A communications protocol developed specifically for all-flash storage, NVMe enables faster performance and greater density compared to legacy protocols. It’s geared for enterprise workloads that require top performance, such as real-time data analytics, online trading platforms and other latency-sensitive workloads.

NVMe vs. serial-attached SCSI (SAS)

NVMe is aimed at reducing the software overhead between applications and storage in all-flash systems.

Flash-based storage systems typically use SAS (serial-attached SCSI) links from their controller processors to communicate to flash. SAS relies on the legacy protocol SCSI, which was originally developed for use with slower hard-disk drives. SCSI can be used for communicating to flash, but it’s not ideal. “How it tries to read and write – SCSI just assumes that the flash drive it’s talking to is a hard-disk drive. It’s not very efficient when it comes to getting the full performance value out of flash,” says Eric Burgener, research vice president for storage at IDC.

NVMe is designed to take better advantage of the mere microsecond latency provided by all-flash arrays and eliminate the SCSI bottleneck. “NVMe was built specifically for flash. It doesn’t even support hard disks. And it’s a much more efficient protocol because it only operates in the best way to read and write to flash,” Burgener says.

How it works

Specifically, NVMe uses parallel, low-latency data paths to speed performance within the back-end of flash arrays: NVMe supports 64,000 queues that can each hold up to 64,000 commands.

“SCSI is a serial protocol, so when you’re trying to talk to disk devices that are attached to a SCSI controller, you can only talk to them one at a time,” Burgener says. “NVMe lets you have up to 64,000 simultaneous outstanding requests, so you can get a lot more devices on the back end. It’s just a much more efficient way to access storage capacity on the back end with this massive parallelism.”

Better performance for enterprises

There are extremely latency-sensitive workloads that will benefit immediately from NVMe’s performance boost, such as real-time big-data analytics or high-performance database environments. Real-time analysis and response are becoming a necessity for businesses, and IDC predicts that between 60% and 70% of Fortune 2000 companies will have at least one mission-critical workload that leverages real-time big data analytics by 2020.

“When an enterprise has a workload that, if it runs faster, drives a bottom-line business benefit such as more commissions, higher revenues, better customer service – those are the workloads that might drive a customer to say, ‘I’ve got to have NVMe today,’” Bergener says.

NVMe products

The first storage platforms that leveraged NVMe instead of SCSI technology began to ship in 2016 and 2017 from startups including Apeiron Data Systems, E8 Storage, Excelero, Micron and Pavilion Data Systems. These systems are mainly targeted for specialized workloads with extremely latency-sensitive workloads, such as trading analysis.

Established storage players, meanwhile, haven’t ignored this faster interconnect protocol and have been re-tooling their systems to support NVMe. There has been a flurry of NVMe-centric announcements in the first half of 2018, including:

• Dell EMC rearchitected its flagship VMAX enterprise product line, which is now called PowerMax, to include NVMe support. Future support for NVMe over Fabric interconnects is promised. “It’s the first time we’ve designed a system from the ground up for NVMe,” said Sam Grocott, senior vice president of marketing for the infrastructure solutions group, in a briefing held in advance of the Dell Technologies World 2018.
• HPE announced the next generation of its Nimble Storage platform; its new HPE Nimble Storage All Flash arrays as well as Nimble Adaptive Flash arrays for hybrid implementations have both been engineered to support NVMe.
• IBM announced in February that its FlashSystem 900 supports Infiniband NVMe over Fabric and shared plans to support a new NVMe over Fabric networking capability in future releases of IBM Spectrum Virtualize and Spectrum Accelerate.
• NetApp unveiled the AFF A800, a new enterprise all-flash array and its first to offer end-to-end NVMe. The AFF A800 combines NVMe solid-state drives with NVMe over Fabric. It can achieve sub-200 microsecond latency from application host through to data storage and throughput of 300 GB/s through a combination of NVMe SSDs and NVMe/FC connectivity, NetApp says.
• Pure Storage (which has been selling its NVMe-based FlashArray//X70 since last year) announced that it is extending NVMe technology across its entire FlashArray X family. Five models – the X10, 20, 50, 70, and 90 – can be configured with NVMe direct-flash architecture.

One key difference between the startups and the larger vendors is that the larger vendors are offering NVMe support in conjunction with their existing enterprise-class data services.

“What’s different is that they took their flagship storage platforms – that had a very complete set of software functionality, such as snapshots, encryption, inline data reduction, replication, all that kind of stuff – and they basically put NVMe technology into those systems,” Bergener says. “So they can go after a different type of workload than the startup guys, who by and large lacked the software functionality. What they were providing was just pure performance.”

In particular, the well-established enterprise storage providers can target denser, mixed enterprise workload consolidation, Bergener says. In a mixed workload environment, traditional database applications and modern web-scale applications could share the same infrastructure. The startup platforms, by comparison, are typically deployed for a single, dedicated workload with high performance needs.

NVMe and flash storage

As NVMe technology matures, costs decline, and storage performance requirements rise, enterprises will move toward flash-based storage systems that implement NVMe end-to-end and can support mixed workloads. “There’s a TCO argument around a system for mixed workloads that’s built out of NVMe instead of SCSI. It still has flash, but it’s flash that you talk to over NVMe versus flash that you talk to over SCSI. You can build a more powerful system in a smaller footprint that uses less energy with the NVMe-based version,” Bergener says.

“At IDC, we think NVMe absolutely will replace SCSI for primary storage arrays,” Bergener says. “But it’s going to take until 2021 before more than 50% of the revenue that’s being generated by primary storage arrays – these are all-flash arrays – comes from NVMe-based systems. We’re just at the start of that.”

For enterprise storage managers that are weighing SCSI versus NVMe technology for their next enterprise storage platform purchase, it’s important to make sure the platform they choose can be upgraded to newer storage technologies (NVMe over Fabric and storage class memory, for example) without requiring disruptive, forklift upgrades, IDC recommends.

There are still going to be a lot of SCSI-based all-flash storage systems sold over the next several years. But “customers that are looking for increased infrastructure density, so they can get more workloads on a smaller platform, or if they need the absolute latency and throughput advantages they can get out of NVMe – those are the guys that would need NVMe right now,” Bergener says.