Cisco: AI networking adds to data center sustainability challenges

Data centers are notorious energy hogs, and the increase in heavy-duty networking and compute power required to handle AI workloads is only going to exacerbate the sustainability issue.

Advancements in AI infrastructure can be at odds with energy-consumption goals, notes Denise Lee, vice president for Cisco’s engineering sustainability office. But there are efforts underway to address the growth of AI and ML in data centers while keeping energy efficiency, cooling and performance in mind.

Lee called out three trends that are driving greater attention to power use and sustainability across the US and globally. For starters, non-governmental organizations, or NGOs, are raising awareness of carbon emissions, greenhouse gases, and the impact on the atmosphere. In addition, government initiatives, such as the European Green Deal, are setting sustainability-driven policies related to retrofitting buildings for energy efficiency, for example, and limiting new data center construction, Lee said.

And in the private sector, businesses are making commitments to net-zero and carbon-neutral development and using the ratings provided by the Climate Disclosure Project (CDP), Lee said. The CDP is a global organization that collects self-reported data from organizations on climate change, water security, and deforestation risks and opportunities. That data is used by investors, corporations, and regulators to address sustainability issues.

“All of these things start to become really interesting factors to make these business decisions,” Lee said, such as assessing the environmental impact of data centers for AI, deciding where to build new data centers, and determining the amount of renewable energy versus nonrenewable energy that’s available from the local grid.

Going forward, organizations will need to address a number of key issues surrounding data center performance requirements, power consumption, cooling, space, and the impact on network infrastructure. In a recent blog post, Lee laid out some specific issues that data center owners and operators will have to consider if they want to keep sustainability in mind as they reimagine their environments for AI:

• Network implications: “Ethernet is currently the dominant underpinning for AI for the majority of use cases that require cost economics, scale and ease of support. According to the Dell’Oro Group, by 2027, as much as 20% of all data center switch ports will be allocated to AI servers. This highlights the growing significance of AI workloads in data center networking,” Lee wrote. “Furthermore, the challenge of integrating small form factor GPUs into data center infrastructure is a noteworthy concern from both a power and cooling perspective. It may require substantial modifications, such as the adoption of liquid cooling solutions and adjustments to power capacity.”
• Performance challenges: “The use of Graphics Processing Units (GPUs) is essential for AI/ML training and inference, but it can pose challenges for data center IT infrastructure from power and cooling perspectives. As AI workloads require increasingly powerful GPUs, data centers often struggle to keep up with the demand for high-performance computing resources. Data center managers and developers, therefore, benefit from strategic deployment of GPUs to optimize their use and energy efficiency,” Lee wrote.
• Power constraints: “AI/ML infrastructure is constrained primarily by compute and memory limits. The network plays a crucial role in connecting multiple processing elements, often sharding compute functions across various nodes. This places significant demands on power capacity and efficiency. Meeting stringent latency and throughput requirements while minimizing energy consumption is a complex task requiring innovative solutions.”
• Adopter strategies: “Early adopters of next-gen AI technologies have recognized that accommodating high-density AI workloads often necessitates the use of multisite or micro data centers,” Lee wrote. “These smaller-scale data centers are designed to handle the intensive computational demands of AI applications. However, this approach places additional pressure on the network infrastructure, which must be high-performing and resilient to support the distributed nature of these data center deployments.”

Cooling is another important consideration in data centers that are handling AI workloads. “Traditional air-cooling methods can be inadequate in AI/ML data center deployments, and they can also be environmentally burdensome. Liquid cooling solutions offer a more efficient alternative, but they require careful integration into data center infrastructure,” Lee wrote.

“The industry has to work on liquid cooling. We just simply cannot cool the chips that are coming,” she said.

Indeed, as server rack densities increase and temperatures rise, more data centers are finding ways to add liquid cooling to their facilities. The global data center liquid cooling market was estimated at $2 billion in 2022 and is expected to grow at a compound annual growth rate of 15% between 2023 and 2032, according to Global Market Insights.

Cisco pursues energy networking

Another interesting technology that could help sustainability efforts is a relatively new class of power system known as Class 4 Fault Managed Power (FMP). The National Electrical Code (NEC) in late 2022 adopted Class 4 FMP, which can handle substantial power levels, up to 2,000 watts, over longer distances compared to older circuit classes, the NEC stated.

Class 4 FMP systems are described as “sustainable” as they allow the use of smaller cable gauges, no conduit, and intelligent control over power delivery; these attributes can lead to reduced material usage and embodied carbon per project, contributing to more sustainable electrical infrastructure, according to NEC. They’re designed to provide power distribution in various applications, such as power over Ethernet, Internet of Things (IoT) devices, smart building systems, monitoring and control of electronics and appliances, security systems, and electronic components in large areas like buildings, stadiums, or campuses.

“Class 4 is ‘touch safe’ direct current so it’s easy to work with,” Lee noted. “We are fast working on this technology with others in the market for the education and the awareness and the adoption of it over time.”

Cisco is also working “energy networking,” which involves embedding energy management capabilities and APIs across its networking portfolio to turn the network into a control plane to measure, monitor, and manage energy. The idea is to transform data into real-time energy visibility and insights that customers can then use to optimize energy consumption, minimize emissions, lower costs, and improve reporting capabilities, Lee said.   

“Energy networking is the concept that we can network power the same way we’ve been networking data. And what if you could even do it together on the same line? It opens up a lot of new use cases for customers to manage power,” Lee said.