In 2017 alone, more external-based storage will ship than has cumulatively shipped prior to 2013.

That got us to thinking about what companies need to be doing to prepare to have a data center that can scale and handle all that data. To get some answers, we turned to Jonathan Caserta, a Physical Infrastructure Solutions Architect with Carousel Industries.

Data Center Upgrades Start with a Frank Assessment

Simply put, Caserta says companies will need to upgrade their data centers to keep up with the forthcoming onslaught, which he likens to a “digital black hole.”  If they simply keep adding equipment willy nilly, they will undoubtedly run out of energy to power and cool their data centers, if not physical space.

A thorough data center and storage assessment is the first step toward creating a plan to more intelligently grow your data center capacity. Such an assessment involves a detailed audit of everything in the data center, including not only IT equipment but power and cooling capacity as well as how racks are oriented.  

“More often than not, racks are not oriented in a hot aisle/cold aisle configuration, which is kind of shocking,” he says, noting the configuration has long been an industry best practice.

Creating A Data Center Roadmap

The next step is to map out where the business is going and what the repercussions will be for the data center. Maybe you’ll be implementing lots of virtualization technology, installing new SAN clusters and implementing high-powered, high-density computing. You’ll need to come up with a plan for dealing with all this technology, while at the same time ensuring energy efficiency.

“If I had to pick the most important thing going on within data centers, it’d be moving older data centers to modern technology to make them more efficient,” Caserta says. “That’s the most critical thing we can do as an industry.”

Dig Deeper: Download the Free Whitepaper- 6 Keys to Energy Savings in Your Data Center

++++++++++++++++++++++++++++++++++++++++++

That means paying attention to data center best practices and methodically implementing them. They include not only the hot aisle/cold aisle configuration but technologies such as in-row cooling, economizer mode cooling and containment systems. Computational fluid dynamics (CFD) technology is now often used to help companies decide what the effects of any changes will be.

“Having a roadmap is the key to success,” he says.

Making the Move: How to Upgrade your Data Center

Actually implementing your data center upgrade plan can be quite the juggling act. A data center retrofit Caserta recently completed for a university client is a case in point. Part of the upgrade involved completely replacing the raised floor of the existing data center. Obviously, that meant everything in the data center had to be moved out.

Fortunately, the school had a second data center site that could handle all the equipment. But it still required painstaking planning to map out times when each piece of equipment could be taken offline and moved, with most of the work being done after hours or on weekends.

If you don’t have the luxury of a secondary data center that can see you through an upgrade, the job becomes a bit more complex – but not impossible. Experts like Caserta and his team at Carousel can create a plan that will work for you and your business. Just contact us to learn more.

After 10 years as a field technician, about 2 years ago Derek started working inside Carousel’s offices, doing everything from building cubicles to running cabling and building out the demo room at headquarters. Along with the rest of the Carousel support group, he helps maintain facilities such as HVAC and power, and work on internal projects.

One Warm Server Room

One such project cropped up back in 2010, when it was getting mighty warm in the Carousel server room. “We were using the building HVAC system to cool the room, the rooftop unit that’s really made for providing comfort air for the offices,” Herard says. “In the server room it was on full blast all the time and it was never below 82 degrees.”

Even worse, the temperature would fluctuate dramatically, by about 10 degrees every minute. When the room warmed up, the HVAC would crank up, going full bore until the temperature hit what the thermostat was set at. But it would stay there only a short time before the heat coming from the servers cranked it right back up again.

That constant on/off cycling is of course not good for the health of the HVAC system – nor for the power bill.

In-Row Cooling: A Cure for Overheated Data Centers

Around that time Carousel had just learned about in-row cooling systems from its partner APC by Schneider Electric. As the name implies, such systems involve installing cooling units in the same rows of racks that house the IT equipment it is intended to cool.

“The cooling units are the same size as our data racks, taking up one 2-foot wide space in the rack,” Herard says. And they’re the same height as the server cabinets, about 7 feet tall.

The idea behind in-row cooling is to dramatically reduce the distance the cool air has to travel to reach the IT equipment it is intended to cool, thus making it far more efficient vs. having the air travel long distances through air ducts.

++++++++++++++++++++++++++++++++++++++++
Best Practices:  Download the free whitepaper, “6 Keys to Saving Energy in Your Data Center”
++++++++++++++++++++++++++++++++++++++++

Figure 1: Temperature Fluctuations with In-Row Cooling

And it works. “Now the temperature is sustained wherever we want it at all times,” Herard says. He notes the building HVAC system is no longer used to cool the server room at all. What’s more, the APC NetBotzremote monitoring system enables Herard to stay on top of what the temperature is and get an alert should it fall outside the prescribed range.

Figure 1 is a screen grab from the NetBotz system, showing the temperature in the enclosure at Rhode Island headquarters consistently hovering right around 75 degrees. Compare that to Figure 2, which shows a smaller server room at a Carousel site in Conn. that doesn’t yet have in-row cooling: it clearly shows the rapid 10-degree swings that used to plague the Rhode Island site.

Carousel actually has two in-row cooling systems, one for each of its two rows of racks. One of the systems, the APC ACRP101 DX, monitors temperatures and can control humidity as well. The other, the APC ACRD501 DX, has the same cooling capacity but no humidity control, since the other unit is sufficient to control humidity in the room.

Figure 2: Temperature Fluctuations No In-Row Cooling

Hot Aisle Containment Adds Further Data Center Efficiency

Not quite a year later, Carousel added a hot aisle containment system to make the setup even more efficient. It’s a standard data center best practice to configure rows such that the front of servers face each other, as do the rear. In a data center with multiple rows of racks, that creates one aisle  - the “cool aisle” – where cool air can be pumped into two rows of servers at once. Similarly, the exhaust coming from two adjacent rows comes out the rear, creating a “hot aisle.”

A hot aisle containment system is basically a large metal hood that covers the hot aisle and captures the air so it doesn’t circulate back into the cool aisle. Rather, it gets pumped directly back into the cooling system. While it seems counterintuitive, it actually takes less energy to cool that warm air than it does to cool fresh, outside air.

From a cooling perspective, the hot aisle containment system basically cut the size of the server room in half, Herard says. “We’re only cooling the air that needs to be cooled for the servers, the intake air,” he notes.

And it shows in the energy bill, which Herard says has been cut in half since installation of the hot-aisle containment system.

Carousel has also replicated this approach in their on-demand labs data center.  “The amount of equipment in the lab is constantly fluctuating up and down depending on the types of projects we are working on,” say Herard.  “Utilizing the hot aisle containment and in-row cooling best practice in our labs has resulted in the same types of savings we’ve seen in our production data center.”

If cutting your data center energy bill sounds like a good idea, contact Carousel - we can help you identify ways to make it happen. And don’t forget to download the free whitepaper, 6 Keys to Saving Energy in Your Data Center.”

The effect will be a renewed focus on low emissions from data centers and alternative sources of energy. One of the reports strategic planning assumptions reads:

By 2016, 15% of large, new data center builds will provision for low-emission outcomes.

Based on our experience here at Carousel, it seems to us that number might be a little low.  With the high level of interest in tracking energy metrics like Power Usage Effectiveness (PUE) the best practices being shared by industry groups like the Green Grid, and an almost universal interest in Virtualization technology that makes data centers run more efficiently, organizations large and small are looking to reduce energy burn in their data centers.

+++++++++++++++++++++++++++++++++

Dig Deeper:  Download the free whitepaper – 6 Keys to Energy Savings in Your Data Center

+++++++++++++++++++++++++++++++++

Keeping Climate Promises

At a climate conference in Durban in 2011, governments committed to coming up with agreements on climate change no later than 2015, with changes going into effect by 2020. Gartner says that will have a ripple effect on enterprises:

So, by 2016, we would anticipate carbon pricing, taxation, and cap and trade to be back under discussion. By 2018, this discussion should be getting much more heated and finding its way back onto the political agenda to the point where enterprises making long-term investments in energy-intensive assets, such as data centers, will want to factor carbon-pricing risks into their decision processes.

Large Enterprises Are Investing in Energy Production Technology

Companies will look for ways to produce at least some of their own electricity, to reduce costs, carbon footprint and reliance on utilities because, as Gartner says, “the utilities’ abilities to sustain low-cost and rock-solid reliability are being increasingly challenged.”

Gartner cites four examples of large enterprises that are already implementing so called “supply side” energy initiatives:

• Apple’s investment in a 4.8 megawatt (MW) biogas fuel cell and 20 MW solar farm at its data center in Maiden, North Carolina
• eBay’s 6 MW fuel cell installation in Utah
• McGraw-Hill’s 14.1 MW solar farm in New Jersey
• National Australia Bank’s 2 MW trigeneration plant

Power-Purchase Agreements Take Hold

On-site power generation technologies are maturing to the point where service providers are now offering power purchase agreements, which can be beneficial for enterprise customers. Under the agreements, a service provider covers the capital cost of installing and maintaining the power generation technology, whether solar, biogas or trigeneration plants, which involves the simultaneous generation of electricity and heating and cooling from the same source. The service provider then sells back some of the power to the enterprise at an agreed-upon price and sells any remainder on the open market.

As the Gartner report says:

This is an attractive option for data center operations, since it removes much of the risk and provides a guaranteed supply at a known cost.

Gartner also advises companies looking to build new data centers to consider “international locations that offer long-term mitigation of energy and carbon-related risks and the associated rising costs.”

Sounds like good advice. If you’re looking for ways to reduce your own carbon footprint and cut energy costs, contact Carousel - we have experts who can help.

While they don’t do a great job of defining the difference between the many different types of data centers in the world, their findings are surprising and bit upsetting.  As they describe:

Most data centers, by design, consume vast amounts of energy in an incongruously wasteful manner, interviews and documents show. Online companies typically run their facilities at maximum capacity around the clock, whatever the demand. As a result, data centers can waste 90 percent or more of the electricity they pull off the grid, The Times found.

They continue:

Worldwide, the digital warehouses use about 30 billion watts of electricity, roughly equivalent to the output of 30 nuclear power plants, according to estimates industry experts compiled for The Times. Data centers in the United States account for one-quarter to one-third of that load, the estimates show.

And of course a statistic about backup diesel generators (which many of these businesses keep idling all the time, just in case) that even I found shocking:

Terry Darton, a former manager at Virginia’s environmental agency, said permits had been issued to enough generators for data centers in his 14-county corner of Virginia to nearly match the output of a nuclear power plant.

Wow! A nuclear plant worth of electricity, just in one portion of one state, just for backup.

This Doesn’t Have To Be You

These numbers are intimidating, but in many ways the industry is not as disinterested in energy efficiency as this article portrays, and for obvious reasons. Often consuming more than 45% of a typical enterprise’s IT budget, power and cooling expenses have been found to represent the second largest common budgetary line item associated with data center operations, so companies have a fiscal as well as an environmental reason to address this issue. That is why technology and industry leaders are coming together within organizations like The Green Grid to unite global industry efforts to improve data center resource efficiency, create a common set of metrics (like PUE – Power Usage Effectiveness), share best practices around planning, layout, cooling and virtualization and develop technical resources and educational tools to lower energy burn.  Google and Facebook in particular are very committed to energy efficiency and also to sharing best practices.

 ++++++++++++++++++++++++++++++++++++++++++++++

Dig Deeper:  Download the Free Whitepaper – 6 Keys To Energy Savings in Your Data Center

++++++++++++++++++++++++++++++++++++++++++++++

And these best practices are not limited to massive cloud providers. As we’ve reviewed at Carousel Connect many times in the past, there are a number of steps that businesses can take to lower their energy burn and improve data center resource efficiency. These include:

• Performing an energy audit to establish a well-defined energy usage benchmark
• Planning effectively prior to execution, taking into consideration:

• Existing energy usage status
• Servers and applications that are no longer necessary (it’s amazing how many of these there are)
• Planned strategic IT initiatives
• A well thought through design including layout, hardware, software and other infrastructure
• Modeling and improvement of design plans via 3D space modeling (computational fluid dynamics)

• Server Virtualization and migration to the cloud where reasonable. Learn how Carousel went from 100 servers to 5.
• Data Center Hardware upgrades and common sense energy saving improvements
• Cooling system upgrades and improvements
• Deploying an effective hot aisle / cold aisle containment system
• Monitoring energy usage upon project completion for access to effective metrics and insight into areas for continued improvement

Don’t let your data center continue to be an energy hog. contact Carousel to discuss how to dramatically cut your data center energy burn and improve performance. For a deeper dive, download the free whitepaper “6 Keys to Energy Savings in Your Data Center“.

To get a handle on what SDNs are all about and the role they can play, we talked with Sameer Mohile, a senior product manager with Extreme Networks. Extreme is a network infrastructure vendor (and Carousel Industries partner) that in July announced some new initiatives relative to its own SDN strategy.

Explaining Software Defined Networking

SDNs bring to network infrastructure the kind of programmability that is common with software. Traditionally, every switch and router in a network has its own control and data plane, which means all routing decisions are made on the switch itself.

The idea behind SDNs is to abstract that control plane from the switches and routers and run it on a central controller. That leaves the switches with just the data plane that actually ships packets in and out; the decisions on where the packets will go is based on instructions coming from that central controller, Mohile says.

That setup gives you more control over data flows as well as multi-vendor support. Any switch that supports the OpenFlow protocol that is used to create SDNs will be able to work with an OpenFlow-compatible controller. “You can have a true multi-vendor deployment and interoperability,” he says.

The Role of SDNs in a Virtualized Environment

SDNs also play a key role in a highly virtualized environment, where it’s common for virtual machines to be added and moved regularly. Such an environment can be difficult to support with traditional network infrastructure, which relies on fairly static connections between switches and servers.

With SDNs, however, those connections can be made more dynamically, and with the proper level of security. It’s similar to the multi-tenant concept of server virtualization, where a single server can support multiple virtual machines, Mohile says. SDNs enable a single switch to have separate data planes, each with its own service level that is appropriate for each data flow. “You are giving resources to different tenants at different times,” he says. “Every tenant doesn’t get the same resources each time.”

SDNs Are All About the Applications

For its own SDN initiative, Extreme decided to focus on four areas, he said. First, it opted to support the OpenFlow protocol and associated APIs, as the company deemed it the most open, fully baked protocol for SDNs. Secondly, Extreme supports OpenStack, which is a set of services that essentially provides the orchestration on top of a controller that enables the controller to interact with various switches.

Extreme also decided to strictly follow the OpenFlow 1.0 specification. “We don’t want to restrict customers to using just one or two vendors, so we have to strictly follow the 1.0 spec,” Mohile says. That means Extreme won’t be adding any “special sauces” on top that might disrupt interoperability. “Any controller that’s compliant with the OpenFlow 1.0 spec will work with our switches.” Initially, Extreme has marketing relationships for controllers from NEC and Big Switch Networks.

But the real value-add with SDNs, he says, is the applications that run in the environment. It’s similar to the smartphone environment. While the phones themselves are nice, it’s the applications that run on them that make the phones truly useful. Just like Apple and Google, Extreme is taking an open approach to application development by launching xKIT, a new application and knowledge base portal for sharing applications. “Anyone can contribute applications and work with our infrastructure,” Mohile says.

A security application, for example, could help companies address the bring your own device trend. An OpenFlow application could identify a device when it connects to the network and immediately apply the appropriate security policy, and ensure the policy remains in place as the user moves about the network. “The central controller knows the device has moved and applies security policies on the fly,” he says.

Extreme will support OpenFlow and OpenStack across its entire line of switches, and Carousel will be on hand to help customers make the most of this exciting technology. To learn more, contact Carousel.

To learn more about how to ensure success with VDI, we talked with Phil Magnuszewski, Director of Virtualization and Cloud Solutions for Carousel, who quickly pointed to three potential pitfalls.

Perform Due Diligence on VDI Use Cases

“Performing analysis to determine which use cases will be a good fit for VDI is probably number one,” he says. “Do your due diligence up front.”

Users who sit on the corporate LAN and use only a small number of applications are good candidates, because you want a workload that is fairly consistent and static. The finance and customer service groups are good examples, as they use the same handful of applications day in and day out.

Similarly, lab environments such as in educational institutions are a “home run” for VDI, Magnuszewski says. Rather than put expensive client machines at each student’s desk, they can instead all run off of powerful servers in a data center.

Pick That Low-hanging VDI Fruit First

When you’re just diving in to VDI, maybe looking for an application to pilot, go for an easy use case first. “Pick something that you know is a good fit and should be successful, then grow to other uses cases from there,” he says. “Don’t try to roll it out to the entire enterprise out of the gate. Pick areas where you’re confident it’ll be successful and build on that.”

Prepare the Network for Added VDI Traffic

A crucial element to the success of a VDI deployment is to ensure the underlying network can handle the additional traffic. Every single user request now has to travel over your corporate network to a faraway server. It all adds up. And nothing will torpedo a VDI deployment faster than users suffering with poor response time.

“Take a close look at what your network looks like today and what it’ll have to look like 12 to 18 months from now,” Magnuszewski says. “Think about the high bandwidth applications such as video that people are adding now. You’re going to have to seriously look at redesigning your network.”

One historical VDI issue that Carousel has successfully resolved is cost. As we covered previously, Carousel has partnered with Atlantis Computing for its ILIO software, which addresses one of the biggest cost drivers in VDI deployments: storage. The ILIO software optimizes and deduplicates traffic from virtual desktops to back end storage devices, decreasing requirements in most cases by 90% or more.

With cost off the table as an issue, and bearing in mind the potential pitfalls to avoid, it’s time to put VDI technology to work for you. By all means, feel free to contact Carousel if you need a hand.

Data Center Containment: Assessment

As with with most technology projects, Davis says it is critical to start with a clear eyed assessment of where you are starting from, the goals and objectives and the assets and liabilities that must be taken into consideration prior to execution. “There are many types of containment systems – vertical containmnet, horizontal containment, hot aisle, cool aisle, rear door, etc.” Davis explains.  “When Carousel sits down with customers to discuss updating or upgrading their containment systems, the first thing we do is ask questions and discuss options.  It is amazing how often the result of the initial discussion results in an approach that bears little resemblance to what the team was initially considering.”

Issues that should be taken into consideration are things like:  The existing data center layout, how cabling is run, how racks are built, energy density per rack, the type of floor in the data center, the type of CRAC system, how their CRAC system works, where the vents and returns are, how much energy is being used by the CRAC unit(s), how many people work there, whether the are leasing or own the space, how the fire suppression system works and is layed out.

Out of this assessment phase comes potential scenarios, as well as issues that need to be taken into consideration.  For example, in a recent conversation with customer, it was determined that a hot aisle containment approach would result in some necessary upgrades.  “They had such a high energy density per rack, that effectively containing the hot air in the hot aisle meant that they needed to upgrade they type of cables they were using to withstand the heat over time.”

Data Center Containment: Planning and Modeling

Once the assessment is complete, it is time to develop potential redesign scenarios.  Although it does not work out in every instance, Davis says that a hot aisle containment system is preferrable.  “CRAC systems work better when hot air goes into them.  Air goes into the front of a server at 75 degrees and comes out the back at 90.  If you can contain that 90 degree air and get it out of the building, up to the condenser on the roof and release that energy into the atmosphere, your costs will come down.

Dig Deeper: Carousel partner APC/Schneider Electric concurs that hot aisle containment is the way to go when given the choice.

Now that potential scenarios are defined, prior to investing in the project to change layout and improve racks and containment systems, Davis suggests taking the opportunity to model the system with Computational Fluid Dynamics (CFD).   CFD provides a very accurate look at how the different layouts and scenarios will perform.  “Modeling the new layout with CFD can quickly pay for itself.  In many cases, relatively inexpensive changes identified in the CFD model can dramatically improve air flow and eliminate the need for costly increases in hardware or cooling capacity that were being considered.”

Dig Deeper: To learn more about Computational Fluid Dynamics, take a listen to the podcast with Jamie Davis.

Data Center Containment: Execution

After defining the scenario and layout that is going to meet the company goals and provide the biggest bang for the buck, it is time to execute.  This is (literally) nuts and bolts time.  Will the design require new racks?  Some racks are air tight, while others may need to be updated to be air tight with gaskets, sweeps, and blanking panels.  Will you need a vertical containment system that will keep hot air from escaping over over the tops of the racks and infecting the cool aisles?  Will you need to make modifications to your fire suppression layout to compensate for the new hot aisle / cold aisle configuration?

“Plan for some rewiring, and for putting in blanking panels and wiring opening pillows to keep air moving into locations you don’t want it to go,” says Davis.  “And when it comes to racks, remember that cheaper does not necessarily mean better.  The right rack can make a tremendous difference in keeping your energy costs down, and it they don’t fit in your budget in the short term, considering racks that can be upgraded over time with gaskets and add ons will allow you to easily adapt to future requirements.”

For more information deploying an effective containment system in your data center, contact Carousel’s Physical Infrastructure experts today.

Jamie Davis as been working in and around data centers for some 25 years, so when he gets excited about any given data center technology, you can bet it’s with good reason. And he is absolutely excited about how computational fluid dynamics (CFD) technology is being used to create 3-dimensional models that help data center engineers make solid decisions on data center layout and cooling infrastructure.  Listen to this information packed Podcast to get all the details.

“Gartner says by 2013 all data center design is going to be done with 3D modeling,” says Davis, who is head of the Physical Infrastructure Division at Carousel Industries. Davis sees a world where all companies have a 3D design for their data centers and consult it before making any changes, from putting in a new blade server to updating cooling infrastructure.

What CFD Is and Why Your Data Center Needs It

CFD isn’t exactly new technology but it’s only fairly recently been applied to data center design. It’s used to create 3D models that show how air is flowing in and around the data center. By changing variables in the model, designers can immediately see what the effect will be on air flow.

Armed with CFD modeling tools, Davis and his team can help solve even the thorniest data center cooling problems. “I can’t tell you how many data centers I go into and they have these big wall fans on the hot aisle and they’re blowing air all over the place,” he says. “If you did a CFD model of that scenario, you’d find you’re creating more problems than you’re solving.”

Too often, if customers have hot spots in their data centers, they’re told they need to add cooling capacity. Davis likens that approach to putting out a fire. Simply dumping more and more water doesn’t necessarily put it out quicker; it’s where you apply that water that matters. “It’s the same with cooling. If you apply it appropriately, you don’t need as much,” he says.

Creating an Effective CFD Model

Creating a good CFD model means taking stock of every piece of equipment you’ve got, and noting its exact location, rack by rack. It’s painstaking work but perhaps not as bad as it sounds because tens of thousands of devices have already been modeled, can customers can access those libraries to populate their own model. Customers also need a detailed data center map, showing the location of all racks and other equipment. And they need details on each air conditioning unit they have, including what type it is, capacity and so forth.

“The more data you collect, the better the model will run,” Davis says. And the more of the data customers gather on their own, the lower the cost of the CFD analysis will be – it’s that much less time a 3rd party technician will be on site doing data collection.

CFD Can Deliver Impressive ROI for Data Center Projects

While Davis says Gartner puts the price tag of a CFD modeling job at about $25,000, in his experience it’s not usually that high; more like $6 to $12 per square foot of data center space.

But the job can quickly pay for itself, he notes. In many cases, relatively inexpensive changes can dramatically improve air flow, such as installing blanking panels so cool air doesn’t escape from empty spaces in racks or blocking so that cool air doesn’t escape from the data center altogether.

That was the case with a recent customer for which Carousel performed a CFD analysis. “They weren’t aware the cooling was escaping out underneath the conference room,” Davis says. “By putting in some blocking and bringing the walls all the way down to the data center, we managed to change how the cooling flow pattern worked. They did not have to invest anything in [additional] cooling.”

A good CFD analysis can also help companies gauge how much bang they’ll get for various levels of cooling upgrades. By seeing exactly what $200,000 will buy vs. $500,000, companies can make more informed decisions on whether they really need the more expensive system. Or, they can opt to divide the data center up, giving more critical resources a more advanced, redundant level of cooling than other areas.

Most customers do have a third party firm conduct the CFD analysis, he notes, because the software is expensive and complex to use. But he advises customers to choose a provider that’s vendor agnostic. “Some third party vendors are aligned with a manufacturer and it doesn’t help the customer because they’re constantly trying to put that manufacturer’s solution into the model,” he said.

To learn more about CFD and how it can help you make better data center design decisions, contact Carousel.

For last week’s blog post on data center design, we talked with Jonathan Caserta, Physical Infrastructure Solutions Architect with Carousel Industries.  If you’ve ever had the pleasure of speaking with Jonathan, you know he’s a fountain of information on data center topics. True to form, in the course of our conversation he offered up not only more than we needed on the data center design topic, but a number of other tips that can help customers operate a more effective, efficient data center.

Centralized UPS – How it Saves You Money

Replace UPS Batteries:  Set A Schedule

Keep Clutter Out from Under Raised Floors

Use Economization Mode to Save Energy – and Money