Mark Bramfitt - Utilities & IT

The Real "Cloud" Story: Three Utility-like Providers

I came across this story in Quartz this morning, and found it a fascinating review of what is happening with cloud computing - starting with perhaps the best definition of what it is in the fourth paragraph:

Generically, the cloud is just a vast mass of computers connected to the internet, on which people or companies can rent processing power or data storage as they need it. It’s used for everything from hosting websites to storing archives to running massive data-crunching operations.

Besides the discussion of how Amazon has jumped the whole market, with Google in second and Microsoft a distant third, what struck me is the conclusion that compute and storage services will soon become a commodity, and yes, utility service.

That means that the three players who are building utility-scale data centers, denominated incidentally by their electric load in megawatts, are essentially building the equivalent of power plants. They're also building the data transmission and distribution system - though I think Google is clearly in the forefront on that score.

The first rule of a utility service is to drive the commodity cost as low as possible, and to monopolize the distribution market. So build the data centers cheaply, in places with cheap power, and start thinking about tying customers to your system.

As the article points out, tying up customers looks like a software/system battle now, but perhaps one of the three players (though I don't think Microsoft will make the play) will seek to buy into the distribution network - the lines that tie directly to consumers. Who wants to buy Comcast? Or maybe build their own high-speed system (Google)?

The upshot is that the leading three will be deeply entwined with utilities for service to their "power plants", and by virtue of their size will explore self generation and demand direct access to the wholesale power markets. And a huge portion of the existing loads from enterprise data centers right down to server closets will shift to the cloud, obviating the need to pursue energy efficiency as a capacity mitigation strategy.

 

Apple to Buy a Small Hydropower System

In what has to be a first, Datacenter Dynamics reports that Apple is securing ownership of a small hydropower system near their new data center in Prineville Oregon.

I wonder what the actual terms of the deal are - is Apple actually buying the facility (what sounds like a short canal and a three MW generation plant), or have they simply contracted to purchase the power. It does seem clear that an arrangement has been made for wheeling the supply, with Bonneville Power and a local cooperative in the mix.

I'm afraid the article makes the common mistake of dealing in electric demand to determine if Apple is closing in on its goal of sourcing 100% renewable power for the Prineville facility. 20 MW of solar PV and another 3 to 5 MW from the hydro system does not represent about two-thirds of the projected facility load of 37 MW (if that is even where Apple is going on the demand side).

With ten or so hours of solar production, and a clear indication that the hydro plant is seasonally off-line, we're likely looking at maybe a third of the power needed for the data center coming from these sources. You always have to talk energy use, not demand, when characterizing a power supply portfolio under these circumstances.

That said, all due considerations to Apple for identifying a power supply option that I haven't seen another operator find. There are data centers (notably in Eastern Washington) that are powered exclusively by large hydro, provided by the local utility, but this is the first small-scale system deal I've seen. 

I expect Greenpeace will still cast a critical eye on Apple's efforts, and the industry as a whole. Their likely criticism of this deal is that it doesn't represent new renewable generation but rather secures power from an existing resource. Nevertheless, Apple is clearly moving in the direction of responsible power sourcing.

A Blue-Chip Case Study Gets Even Better

One of my favorite examples of the positive impact that IT can have is to cite UPS, which uses an algorithim to optimize the routes of delivery trucks to minimize fuel use and maximize driver productivity.

So first, I was pleased to come across some reporting on what UPS does, titled "Why UPS Trucks Don't Turn Left".

But even better, I met two guys at an event in Atlanta the week before last, who have been responsible for the operations at UPS's data center. 

They told the story of examining the operations of the center years ago to find cost and energy savings, primarily attacking airflow opportunities - well before the current focus on efficiency took hold.

Their strategy was quite simple: they compared the total airflow needed by equipment (using nameplate ratings and specs) with the total airflow provided by the cooling equipment. Turns out that like many enterprise data centers, they were over-provisioned by a factor of greater than three.

They commenced efforts to plug leaks, instituted blanking panel installation policies, and started to put cooling units into standby, saving hundreds of thousands of dollars in annual energy costs in the process.

So my favorite story remains so: UPS uses efficient IT to deliver real on-the-ground environmental and productivity benefits. That's a blue chip case study.

Latest Greenpeace Report on DC Industry Gets Big Time Traction

Gary Cook and his colleagues at Greenpeace have reported for several years running on the environmental credentials of major data center industry players, focusing mainly on internet-based companies that many consumers would recognize.

The point of the reports is to grade these companies on a slate of metrics from how much "green" power they use (nuclear doesn't count!) to how forthcoming they are about their practices.

The latest report has gotten more attention than ever, perhaps because it seems some players have taken the grading to heart and moved way up in the rankings. Apple, Google, and Facebook are the new winners, while Amazon (and AWS) and collocation operators are now solidly in the criticism cross hairs.

What does attention look like?  Reports in mainstream press outlets like Salon and Wired, coverage in environmental media like Climateprogress, and of course industry press like Datacenter Dynamics.  There's even reporting of Grennpeace's awareness campaign featuring a blimp flying over Silicon Valley Tech campuses spreading the word.

I don't think you can overstate the impact that the report is now having, notwithstanding Amazon's stiff-arming. Some of the new leaders on the grade chart used to have pretty bad marks indeed.

And even if the laggards do nothing to improve their status, there is a clear preference now in the data center industry to source renewable power. Google has done it, Facebook is doing it, and Apple is almost wholly sourcing it.

Apple even used it's buying power to convince regulators and Duke Energy to debut a new program so that they could purchase new renewables (an addition to whatever Duke was already sourcing) for their data center in Maiden, NC.

So if you are a utility seeking to attract development from these big players, I think having some kind of renewable purchasing scheme or direct access provision is now crucial, in addition to your current and future rates and ability to serve very large loads responsively.

A Cloud War in SF This Week

And I'm not talking about the five days of rain we are grateful to get this week...

The battle for hearts and minds and customers of cloud services is intensifying with news yesterday and today of price cuts and a big market entry.

First up is Google, announcing big price cuts for cloud computing capacity as well as efforts to entice developers to design services using their platform at a conference here in San Francisco.

Google has also intriguingly opined that cloud service costs should decline (or services increase) according to Moore's Law. I find that hard to get a grip around because I'm not sure that the underlying IT can keep up with Moore's Law (though many have predicted its demise before), and more importantly I don't know how much more energy and cost efficiency can be squeezed out of the data center infrastructure supporting the cloud.

Amazon's AWS is expected to make announcements in a similar vein later this week at a conference in...San Francisco. We'll see if pricing hits the headlines.

And notice that Microsoft is mentioned in the press as having to step it up on price competitiveness as well.

Lastly, here comes Cisco, with an announcement that they're going to get in the cloud services (another report here) game with $1 billion in data center investments over the next couple of years. I would think it hard to go from a low level of development to that rate in such a short time frame - the best developers who have big build programs take over a year to get a project done.

For utilities, we may be seeing the beginnings of a really big move by the IT industry towards a utility computing model: central data centers looking like power plants, with compute and storage capabilities priced just like utility service.

And for those looking to attract utility-scale data center development, Cisco represents a new opportunity. Unlike the other players who are increasingly expanding existing facilities rather than seeking new sites, Cisco will likely be open to new locations. Economic development teams, start your engines.

 

"First" DCIM ROI Calculator from Emerson Network Power

An announcement from Emerson Network Power hit my email box a week or two ago saying they were offering the first (and "only") data center infrastructure management financial evaluation tool, and I naturally was curious to see what they had come up with.

If the tool had solid calculations predicting energy efficiency gains, it might be suitable to support utility energy efficiency incentive program applications.

Sorry to say that I haven't yet gotten a good look at the tool, or been able to ask questions about the underpinnings of the program. (You have to contact Emerson to get an evaluation using the tool, and though I spoke with company representatives at Teladata"s Technology Convergence Conference last week, I neglected to ask them for a test drive.)

I'm guessing that utilities will still have to rely on actual before and after measurements of conditions in a data center where DCIM tools are implemented given the complex nature of the energy savings opportunities.

After all, DCIM technology can generate efficiencies in a half dozen or more ways, in combination with other measures.  For example, asset management features can lead an IT manager to distribute IT equipment more homogeneously on an electric load basis, allowing for lower airflow requirements and possibly less stringent environmental conditions.

DCIM can support the implementation of a host of airflow management solutions, from a simple re balance through tile management, to direct control of variable speed fans in CRACs or CRAHs.

Given that energy savings are likely to be a significant portion of making a case for DCIM projects, I'm sure Eaton's tool must make some assumptions. I know it must be part of the package as a screen shot of an opening window of the ROI calculator asks for the local energy price!

I'll be on the lookout for calculation tools that evaluate DCIM to see if any could seamlessly mesh with utility program requirements, but I expect that evaluation tools will make broad assumptions that would still require at-the-site metering and monitoring to qualify.

Novel Use for Fuel Cells: Backup Generation

Fuel cells and solar PV get all of the attention in the data center industry as alternatives to traditional electric utility service, though there are rumblings for combined heat and power (cogeneration) installs in the background.

Part and parcel of any discussion of primary power sources (including utility service) is backup generation. Sometimes that discussion includes relying on utility service as the backup, but I imagine that the UPS and diesel generator combo has ninety-nine percent plus market share.

Now a company called CommScope is proposing the use of fuel cells as backup, rather than primary power sources for data centers, per this article in Datacenter Dynamics.

I agree with the criticism of diesel generators up to a point - they of course produce emissions of pollutants and greenhouse gases when operated and tested, but with proper maintenance they are soundly reliable. Most backup generation failures that I've heard of (many at hospitals as well as at data centers) occur due to problems in switchgear, not the gen sets themselves.

But there is another reason I think that CommScope may be grasping at straws. They are positing a sixteen hour run time, using hydrogen fuel. From my days promoting alternate fuel vehicles, hydrogen storage was always an achilles heel. Costly. Leaky. Very high pressure.

If fuel cells were to make a go of it as backup generation, I'd talk to Bloom or ClearEdge, and use natural gas service as the fuel source. 

Now utility natural gas service is not 100% reliable either, but it does have the advantage of being able to serve backup gen essentially continuously if needed.  I tried to make the case that diesel gen sets should be converted to natural gas (easy to do; 15% diesel still used as a pilot fuel for ignition) for this reason, and because emissions would be dramatically lower, but have never seen that idea take hold.

This is not to say that there aren't data center operators out there willing to try new approaches. Google put the UPS on the server in the form of a battery, and Microsoft has looked at fuel cells at the rack level.

I'd like to see a head-to-head TCO comparison of diesel gen sets and fuel cells as backups, though regardless of the outcome, an operator is looking at the most expensive power he will ever use!

 

A Good Review of Immersion Cooling

Passing along a link to this piece in the New York Times regarding liquid immersion cooling, which to date is in use in high performance cooling applications but may eventually move down into the utility-scale data center market.

Though immersion cooling causes shills down the spine of IT managers who didn't have the experience of running water-cooled mainframes from years ago (and there are plenty still out there, the mainframes, I mean!), serious gamers have been dunking their PCs in mineral oil baths for many years.

Although the technology offers clear efficiency advantages (look ma, no fans, anywhere!), I expect it will be used mostly in the super computing arena.

The utility-scale compute environments that I've seen have pretty high energy densities compared to enterprise environments, but they're not pushing 20 kW per rack, which leads you down the path of close-coupled cooling solutions. I'm guessing the energy savings from the 100% outside air cooling schemes used by the big firms will lead them to keep densities low enough to be supported by air-based cooling schemes. 

 

EULs for Data Center Power Equipment

A bit of utility energy efficiency program managers insider baseball, so skip if not of interest!

I collected a bit of useful data yesterday at the Technology Convergence Conference hosted by Teladata in Santa Clara yesterday in a presentation by Eaton, a big provider of power delivery and conditioning equipment for data centers.

Eaton posted a short list of the expected useful life for four classes of equipment. That's obviously useful for data center facility managers who have to anticipate equipment replacements, and even for accountants who have to set depreciation schedules. (Well maybe not for the accountants as I suspect the schedules are set by IRS policy, not reality!)

Here's the list:

UPS equipment: 12 years

Transmission switchgear: 12 - 15 years

CRAC units: 10 years

Circuit breakers: 12 years

It's a pity we don't have estimates for power distribution units and distribution transformers, and I wonder if there isn't quite a lot of variation for cooling units (how long to CRAHs last?), but there's something here for utility program managers as well as data center operators.

For the DC managers, I wonder if they really plan for equipment replacement based on these time frames. In particular, I've seen switchgear and circuit breakers that have lived several lives based on these numbers!

For utility program managers, the UPS and CRAC units figures could be a good basis for establishing Effective Useful Life (EUL) metrics for your energy efficiency programs. 

Let me explain that in layman's terms for non-program folks: If you approach a utility for an energy efficiency rebate for a UPS system that is say nine years old, they are likely to base the payment on the energy savings of a premium efficiency unit you might purchase, compared to a "standard" efficiency unit on the market today - not the difference between the unit you buy and the existing unit.

The premise is that at nine years old, you have to buy a new unit anyway, so the efficiency gain of buying a modern unit is a given. The utility will only reward you for going to a premium efficiency unit.

Now, if you replace a unit that is not close to the end of its EUL, the utility can provide more of an incentive because you are retiring the equipment early.

I'll look for more references to EULs for data center equipment out in the industry and pass them along as I uncover them.

 

A One-Way View: What the Data Center Industry Thinks of Utilities

I'm not sure that I'll comment as extensively as I'm tempted to do by this article from Yevgenly Sverdlick in Datacenter Dynamics - my central complaint with the article is that it doesn't present a utility perspective on any of the issues covered.

And I will say that with twenty-five years in the utility industry I am still defensive when issues like grid reliability come up. For example, I would change the opening sentence to "electrical utility grids are notoriously reliable.."

The issues of service reliability, the carbon content of power, and the opportunities for self-generation that could lead to grid independence for major data centers are all worthy of discussion, and the thoughts from the quoted industry leaders all have their merits.

But I would counter that the transmission-service power that multi-megawatt data centers get from utilities is very very reliable and a great value.

Yes, you have to make your own investments if you want a higher degree of reliability, but utilities have never been asked or required to provide that level of service. They would have to make the same kinds of investments that data centers do to provide that reliability, and the customers who want that reliability would have to pay for it. Does that sound fair? It is the basis for utility rate regulation after all.

The comments on renewables and a cleaner generation portfolio are apropos, but fail to acknowledge that getting to a cleaner, greener mix will mean higher costs. Sure, wind is pretty cheap, but it has to be offset with costs for generation or storage to make up for intermittent generation. 

And fuel cells? The mumbled truth I keep hearing is that they provide power at nine cents a kWh after tax and incentive subsidies, and with the historically low gas prices we see today. And that may not include the standby charges that the utility will assess to back you up.

Large data centers in California, paying perhaps the second highest rates in the country, see average rates of under nine cents per kWh. You bring your backup systems and factor in those costs. But please remember that the utility pays taxes, and franchise fees, and provides the money for self generation incentive programs. No tax incentives in this equation.

I said I wasn't going to comment extensively (and I could go on), and now I've made myself a liar! It is a shame though how far apart the worlds of power utilities and the data center industry remain, after a solid decade of staggering data center development in terms of both scale and scope.

I recommend this article as key reading for utility managers seeking to understand the data center market - it certainly covers that side of the story well.