Data centers, the increasing use of AI and reindustrialization in some parts of the country are driving a big increase in energy demand that concerns officials at all levels of government, including state legislators concerned about the ability of their state utilities to meet that growing demand. We sat down with two experts on data centers to find out what’s coming in the next several years.
Data centers, the increasing use of AI and reindustrialization in some parts of the country are driving a big increase in energy demand that concerns officials at all levels of government. State legislators in particular, are concerned about the ability of their state utilities to meet that growing demand and the effect on the price residential and business customers pay for energy.
We sat down with two experts on data centers to find out what’s coming in the next several years. Our guests are David Porter, vice president of electrification and sustainable energy strategy at the Electric Power Research Institute or EPRI, and Mukul Anand, global director at Johnson Controls dealing with data center cooling and energy management.
Porter explained the range of energy demand scenarios and the very significant demand that will be put on both the generation and transmission of power in the next several years. Anand discussed the work he does to help data centers explore alternative energy sources and find innovative ways to dissipate the extraordinary amount of heat generated by the racks of servers in the centers. He also expects a tremendous increase in power demand and suggests more planning may be needed to be ready.
Resources
Ed: Hello and welcome to “Our American States,” a podcast from the National Conference of State Legislatures. I’m your host, Ed Smith.
DP: Data centers are going to be a big part of it.Today, in the U.S., data centers use about 4% of the total electric energy that is generated in the U.S.
Ed: That was David Porter, vice president of electrification and sustainable energy strategy at the Electic Power Research Institute or EPRI. He is one of my guests on this podcast along with Mukul Anand, global director at Johnson Controls dealing with data center cooling and energy management.
Data centers and the increasing use of AI along with reindustrialization in some parts of the country are driving a big increase in energy demand. That concerns officials at all levels of government. State legislators, in particular, are concerned about the ability of their state’s utilities to meet that growing demand and the effect on the price residential and business customers pay for energy.
David explained the range of energy demand scenarios and the very significant demand that will be put on both the generation and transmission of power. Mukul discussed the work he does to help data centers to explore external energy sources and find innovative ways to dissipate the extraordinary amount of heat generated by the racks of servers in the centers. He also expects a tremendous increase in power demand and suggests more planning may be needed to be ready. Here is our discussion starting with David.
Ed: David, welcome to the podcast.
DP: Thank you.
Ed: So to get started, why don’t you tell us a little bit more about EPRI in your work.
DP: Well we are fortunate enough, Ed, to be in our 53rd year of operation. We started as the Electric Power Research Institute. We still use that name in EPRI, but it’s broader than that today and we do a lot of work in alternate fuels, natural gas, far more than just the pieces of the electric industry, but our primary focus is still on electrical energy all the way from the generation through the transmission, distribution and ultimately the end use of electric energy.
Ed: So, our focus today is data centers and the amount of energy they use and what the future holds for them. I wonder if you can start by giving our listeners just kind of a primer on energy demand generally in the U.S., kind of a picture of where we are right now.
DP: The landscape for energy demand in the U.S. has changed significantly in the last couple of years. We’ve been in a situation for the last couple of decades where the demand for electricity has stayed relatively flat and that’s been due to a couple of things. But primarily it’s been due to energy efficiency in virtually everything that we do.For example, everything from new cars coming on the highway be they electric or gas, they are more efficient than the previous generation. Through programs that have been around a long time such as Energy Star computers, appliances for the home, HVAC systems. All become more efficient as new equipment comes out. So as old equipment has changed, new comes on. It is even more efficient than its predecessor so while there is an increase in the use of energy, the overall demand for energy has stayed relatively flat. We are in a situation now where we are seeing growing again broadly across the industry for a couple of reasons. One of the main reasons is from large point loads like data centers and the reshoring of industry back to the U.S. Government incentives like the CHIPS Act that are encouraging manufacturing to start up here again in the U.S. And then another key thing is the electrification of transportation which is going to be significant low growth for electric utilities as well as we continue the transition to a clean net zero future it’s going to require a greater use of electricity in the end use in the replacement of fossil fuels along with the replacement of fossil fuels in terms of supplying the electrons to the system. So, we are going to see a steady growth for electricity in the coming decades.
Ed: Well, I think that’s a great explanation of the basic dynamics of energy demand and where we are. We are talking to state policymakers of course and for them, getting accurate forecasts would seem critical in terms of trying to develop any kind of state level policy. So where are we headed on this and what do the projections tell us?
TM: 04:58
DP: Having an accurate load forecast for the energy industry is really important and as you know because the utility planning cycle is not a short one. Planning all the way to getting an asset up and running in the field takes a long time.Different things can be done at different speeds, but for example, one of the key things for low growth for the electric industry is being able to increase capacity on the transmission system across the US. That’s a complicated process. Today on average, from the time a utility decides we need more transmission capacity, they start the design to the point where that’s up and running in the field.The average across U.S., that’s a 10-year project. It’s like that does not match up well with the growth that is coming to the grid.There are opportunities to improve not only the forecasting for the load that is coming, the timing of when it is coming, but also find some new approaches for regulatory approvals particularly around transmission lines, new technologies to get more out of the assets that are in the field today through grid enhancing technologies to better meet the needs and the speed with which the customers want that service.
Ed: So, let’s talk a little bit about the data centers and the role they play here. My understanding is data centers and particularly AI are using or are going to use a lot more energy. But, as you mentioned, there’s other factors behind this increasing demand including what some people would say is kind of a reindustrialization in a lot of places in the country. Fill us in on that. How big a factor are data centers in the increased load demand?
DP: Data centers are going to be a big part of it.Today, in the U.S., data centers use about 4% of the total electric energy that is generated in the U.S. By a variety of different projections and scenarios that we laid out in a recent report which we can connect you with that you can make available to your listeners, Ed, we project that that growth between now and 2030 will get to the point where data centers could consume as much as 9% of all the electric energy in the U.S. by 2030.
Ed: Let’s talk a little bit about how you make these projections. What’s the difference between sort of the high end and the low end?
DP: Yeah, we took a look through some different scenarios. And on the low end, we projected growth to be something on the order of about 3½% per year between now and 2030 all the way to the high end which looking at estimates from other folks looked to be on the high end about 15% annual growth. And that’s what would translate into that 9% total energy consumption by 2030. On the lower end, it would only get to about 4½% somewhere in between the 4½ and the 9%.We think by our estimates, Ed, it is probably going to be more like 6 to 7% rather than 9 by 2030.
Ed: If I’m in a part of the country or a part of my state that doesn’t have a data center, I might think well it’s not going to affect us, but my understanding is these data centers because we are talking about so much energy, they have kind of an effect on the whole system. Is that right? I mean is the whole grid affected by the amount of power that these places are going to along with some of these other consumers.
DP: That is a bit of a yes and no answer, Ed. And a lot of it depends on where the data center is located exactly to your point. Because as you look around the U..S depending on a location at a data center or a large point load or even someone like Amazon who has electrifying their fleet to move packages that last mile, depending on where they are located, there may be a wealth of capacity available on the grid. But there might not be too, so it’s very regionally aligned.In certain parts of the country and I’ll use northern Virginia as an example which has become a hotbed for data centers, capacity is quite a bit more constrained there than it is if you go to North Dakota for example. But when the data centers are looking to build a new facility, there are several things they take into account. Most importantly is what they call availability zones which is the access to the communications infrastructure. There has to be really strong communications infrastructure for all the reasons that we understand what’s going on in the data center and where the computational results need to get to.
Second to that is the price of the land, the price of the electricity and what type of interruptible events are there on a regular basis. What’s the skilled labor pool that’s there to work in those data centers and keep them up and running. All those factors play into it. In terms of impact on the grid, right now most of the data centers are pure off-takers of power from the grid. Some of the work that we are doing is aiming to make them a flexible resource for the grid and be able to provide power back to the grid or load flexibility to the grid in times of need in particular for some of those areas where they might be some demand constraints a few hours of the year.
TM: 10:28
Ed: Well David, I think you’ve given us a pretty good primer on how all this works and I appreciate that and the audience does.And, I wonder just to close out, thinking about this audience of state legislators and legislative staff, you mentioned the new report on data, I’ll put that in our show notes. I wonder if there is anything else you’d direct them to ah people they might talk to so they could better understand this issue.
DP: Certainly, our paper and there’s a lot of information that is out there today from a variety of sources. There’s a lot of different looks depending on who is putting the information out there, but I think the message I would give for them to think about is take a good look at a broad array of the information and to get that really balanced view. The data center topic can be one that gets to be a bit of a lightening rod for some folks depending on where they are. There’s certainly a challenge for the big hyperscalers, the Meta, the Googles, the Microsoft, the Amazon web services because it is important to their business. Their businesses are continuing to grow just with the services they provide today. Artificial intelligence is really an add-on to their existing services and it is much more energy intensive. Just to give you an example, a standard Google query uses about a tenth of the energy that a chat GPT AI request takes. So, it takes about 10 times the amount of energy for a chat GPT query as it does a Google query. That’s still small increments of energy at about 3-watt hours for a chat GPT request.But when you multiply that times millions of users, it can turn into a lot of energy fairly quickly.
Ed: Well, any time we start talking about 10X, you do start to get a little concerned those things are going to go up very, very quickly.
DP: Yeah, the other thing I would add too Ed, for particularly for folks that are in the regulatory and policy area, a lot of the projections that are out there particularly on the high end are thinking unbridled growth and there will be a lot of building of data centers and continued growth. But in some of the conversations I have had with some of the key technologists at companies like Google, they are quick to remind us of the history of the efficiency that comes not only from chips, but from their ability to improve computational architecture so that over time that inherent energy efficiency that we were talking about continues to gain inside the data centers with the chips, with the architecture so that it’s not a continued mountain climb for demand and things do level out over time.
Ed: Well, we will have another chapter on this one I think for sure. David, thank you so much for filling us in and sharing your expertise with us. Take care.
DP: Thanks, Ed. Thanks for having us.
Ed: I’ll be right back after this short break with Mukul Anand from Johnson Controls.
TM: 13:44
Ed: Mukul, welcome to the podcast.
MA: Thank you for having me, Ed.
Ed: Well to start, I wonder if you could briefly tell us about your role at Johnson Controls and your work with data centers.
MA: Absolutely. Thank you for having me. It’s a pleasure to be here. It’s a pleasure to have an opportunity to talk about a topic that I’m super passionate about. Over the past 26 years, I’ve held several engineering and general management roles within Johnson Controls. Currently, I’m the global director for business development and strategy for our applied HVAC equipment team within the data center vertical. And I focus on current and future data center cooling architectures and the HVAC equipment needs associated with these designs.
Ed: Why don’t you start and explain to us the approaches data centers can use to minimize their energy use and I know this is sort of a core area that you concentrate on. So, fill us in keeping in mind that most of us are not very sophisticated about this.
MA: Sure. Yeah, so this is a great topic and I will cover it under three separate sections.The first one is the IT side so on the IT side, we conform the compute more efficiently using efficient IT equipment.Using efficient software algorithms and running the processes at a time when the infrastructure, which is power and cooling, consumes less power for the cooling of the data center. So that is the main one on the IT side.
Now, looking at that on the infrastructure side, one of the core concepts is to operate the data center at its designed or optimized points so in a lot of instances what we see is that a data center is designed for the hottest afternoon in 20 or 50 years, but it’s operated at a much more nominal condition, which is prevalent within the city where the data center is located. So, care must be taken to ensure that the power and cooling infrastructure is actually optimized for the real-world operating conditions.
And then finally, my focus is on the HVAC or the data center cooling side. We take a lot of steps to ensure that the power consumed by these data centers are as small as possible for the compute loads desired. What do we do there? We make sure that the cooling architecture is actually optimized for the real-world conditions in which these data centers will operate. We also make sure that the operating logic incorporates chilled water reset. This is a condition where for a few hours in the year, you can change the chilled water set point and the cooler temperature to make sure that on those days there is good uptime, but on the rest of the year the data center efficiency is the best that it can be.
And then finally, I would like to point out that the needs of the industry have gone beyond just beyond an energy efficiency.We also focus on carbon neutrality and overall sustainably off these data centers. What does that mean? It means that as an example, we use HVAC equipment that are vigorously texted at extreme conditions so the 2-end redundancy is not required, we can build data center with N+2 or N+20 percent redundancy. It also makes sense for us to design, build and operate equipment that are relatively quiet so that we are good appropriate citizens in the community is where these data centers live and operate. Also ensuring that we use refrigerates that are ultra-low GWP to make sure that these are sustainable solutions throughout their lifetime.
And then finally, I would like to point out that most of the HVAC architectures today do not consume any water and we attempt to provide good to great efficiency in data centers with sedile water usage effectiveness. So, there are a not of things we can do to be good corporate citizens and to leave this world a better place for our kids.
Ed: So let me follow up on one thing. Just again, we have an audience of policymakers and folks that may not be that educated on this, but I guess I think a lot of people think of data centers as on the consuming energy side and they don’t think of it that much on the cooling side and as you were pointing to me in an earlier conversation, this cooling side is a tremendous challenge and one that that’s one of the things you go in and help them figure out how to get rid of all that heat. Is there any example of a specific project maybe you worked on where you able to really reduce substantially the amount of heat in an innovative way?
MA: Yes. I will talk about two cases that come to mind. The first one is a data center application where we had a fixed number of generators supporting data center that was designed as a number of modules.In this application, we offered a particular chiller called our YVAM chillers which use … centrifugal compressors to reduce the design power consumption. The design power consumption being the power consumed on the hottest day of the year at design loading conditions. And we were able to reduce this by about 14% allowing a smaller sized generator to be used to run the data center modules and therefore an efficient, lighter and more green.
Another example that comes to mind is where we had a reduction in the weight of the equipment. Some of our applied HVAC equipment including the wireframe chillers, they can provide economization or in simple words free cooling in the absence of additional free cooling coils. Now these pieces of equipment are significantly lighter and when you pair it up with the fact that this equipment are sitting on top of the roof 60 to 70 feet up in the air with the wind loading on them, the structural steel that is used by the building can be significantly reduced if you are using lighter pieces of equipment. And therefore, we were able to bring down the carbon content not only of the HVAC equipment, but also of the building itself making it easier for our customer to reach the goals of carbon neutrality. So, these are just some of the examples that come to mind right away.
TM: 23:08
Ed: I talked earlier in this pod cast with David Porter from EPRI. He sketched out what we are looking at in terms of energy demand in the coming years and I’m wondering when you look at the coming energy demand and the energy supply, do you think that the energy management approaches that are being employed now are sufficient or is this a scenario where we need to develop more.
MA: I think we absolutely need to develop this more.We have over the past couple of years seen the growth in the industry driven by the need for AI based workloads.And we need to triangulate this information so when we think about the processor manufacturers, Enridia both in February of 2024 and May of 2024 have reported data center related cumulative annual grow trade of more than 400%. It means that every year, year over year, it becomes 5 times the shipment.
In addition to that, we have seen that our president has passed the CHIPS Act and there is an intense focus on America to increase its domestic semiconductor production. These two effects together will lead to a large increase in the need for infrastructure which is power and cooling. And as such, when we triangulate it based on the ability for us to build high density racks and multistory data centers using less labor as well as novel cooling technologies, I think the need for power will grow at a pace that we haven’t seen in the past few years. We need to get ahead of it. We need to plan for this growth and we need to make sure that our power generation both legacy and renewable are robust and forward looking.
Ed: Well, I think that is exactly the sort of information policymakers need is to understand what is coming down the pike.Looking at the wider system, I’m just wondering what sort of role regulators play as they are looking at data centers and operators. This of course is a concern of policymakers to ensure stable rates, reliable energy for all their residential and other business customers, but can you talk a little bit about that relationship.
MA: Absolutely. I believe that our regulators play a very critical role in developing and supporting a long-term plan. How do we begin to engage in this effort. Let’s triangulate on the load right. So, let’s think about what the chip manufacturers and the server manufacturers are saying about their backorders and about their shipment levels right. Let’s review on the other hand with the utility providers and what their customers are asking them for which is individual data center campuses asking for 200 to 300 megawatts of power in the next 12 months that they cannot provide.In addition to that, we also can triangulate using feedback from newer industries like autonomous driving, AIMLA loads, global communications as well as the difference industry. Right. So, when you look at these three industries and triangulate the need for compute and therefore the need for power in the future, we can come up with a very solid demand curve and then create a plan that is forward looking to take it off that demand both in terms of generation and in terms of transmitting it over the grid.
When we think about this sort of a plan, obviously all sorts of renewables should be a part of it, but the growth and the robustness of infrastructure that is required leads us to believe that nuclear power especially the small modular reactors will be an integral part of this plan where our policymakers can really streamline the process. So, this is a great question in my opinion.
Ed: And this really involves them working with the utilities of course. When you look at the utilities themselves as opposed to the regulators, are they on track to be able to meet the kinds of demands that you see coming or do you think that’s something where we really need to get policymakers and regulators and all of those people all pulling in the same direction.
MA: I think the regulators are in the same boat as the rest of us. I believe that they understand that a tsunami of demand is coming at us. I believe that they are doing everything in their capacity to increase production and make the grid more robust. However, a national level infrastructure development plan is probably needed to ensure that we can grow at a very rapid pace in a reliable fashion such that the processor manufacturing companies can keep growing such that the server manufacturers can trust the plan and keep growing so that the data center owners and operators can keep investing in new data centers and the equipment providers can keep planning for larger factories to enable the growth of this industry. If all of us work together towards a robust plan, then the whole engagement is very efficient and none of the industries will waste resources or overbuild so to say.
So overall, I believe that whether it is the utilities or any other industry, our policymakers can make this overall plan and execution a very smooth one.
Ed: What do you think state lawmakers should know maybe that you haven’t mentioned yet. Anything that they should know or anything they should really understand about what’s going on in their states.
MA: Certainly. I think that our policymakers should have in-depth discussions with different segments of this industry. We all understand our segmentary really well and we all trust the grown that is coming our way. However, the growth that may reach us could be significantly higher than what we see today which is what we are seeing the accelerating grill trade of some players in this vertical. What does that mean? It means that the data center construction let’s say by the general contractors when the unemployment fade in the country is 4% then you know there are only 4 out of 100 people available to be hired. But when the technology goes where the same number of people can build a 300kw rack as opposed to a 30kw rack then you don’t need more people. But the needs of the processors, the electrical power and the cooling capacity just went up tenfold without increasing the number of people needed in the construction vertical. So, there are some elements of this growth that can hit different industries differentially. We need to understand that and we need to respect that plan for that. But to be able to be able to plan for that, we need to do deep dives into the growth of several different industries and understand what is obstructing their growth today and once that obstruction is removed what is the natural tendency to grow. What is the natural growth rate of that vertical. Once we understand and respect that for the several verticals I just mentioned, we will come up with a true plan, an unrestricted plan that we should create our 10-year plan to support.
Ed: Well, Mukul, I think that is great advice and I really appreciate you taking the time to walk us through this. It’s a very complicated topic. I know there is a lot more for people to learn, but I appreciate you getting us started on it. Take care.
MA: Thank you, Ed. Thanks for having me.
Ed: I’ve been talking David Porter from EPRI along with Mukul Anand from Johnson Controls about energy demand, how the increase in data centers will affect demand and steps that can be taken to address the issue. Thanks for listening.
TM: 32:40
You can check out all the podcasts from the National Conference of State Legislatures by searching for NCSL podcasts wherever you get your podcasts. This podcast “Our American States” dives into some of the most challenging public policy issues facing legislators. On “Across the Aisle” host Kelley Griffin tells stories of bipartisanship. Also check out our special series “Building Democracy” on the history of legislatures.