Elastic Sky Labs

DRS Advanced Settings

2016-05-09T15:01:00.001-04:00

Recently, I performed a study on DRS, which focused on workload placement of virtual machines to load balance a cluster.

It demonstrated that by using DRS advanced options, we are able to balance memory usage across a cluster. In my next blog post, I examine memory consumption (usage) vs active memory (demand); and discuss the importance of understanding the difference and the impact on workload placement.

Key Terms

Active (Demand) - Amount of memory that is actively used, as estimated by the VMkernel based on recently touched memory pages.

Virtual machine: Amount of guest “physical” memory actively used.
Host: Sum of all active metrics for all powered-on virtual machines plus vSphere services (such as COS, vpxa) on the host.

Granted - The amount of machine memory or “physical” memory that is mapped for a virtual machine or a host.

Virtual machine: Guest “physical” memory that is mapped to machine memory. Includes shared memory amount. Does not include overhead.
Host: Sum of all granted metrics for all powered-on virtual machines, plus machine memory for vSphere services on the host.

Consumed - Amount of memory consumed by a virtual machine, host, or cluster.

Virtual machine: Amount of guest physical memory consumed by the virtual machine for guest memory. Consumed memory does not include overhead memory. It includes shared memory and memory that might be reserved, but not actually used. Use this metric for charge-back purposes.
vm consumed memory = memory granted - memory saved due to memory sharing
Host: Amount of machine memory used on the host. Consumed memory includes Includes memory used by the Service Console, the VMkernel, vSphere services, plus the total consumed metrics for all running virtual machines.
host consumed memory = total host memory - free host memory

Usage - Memory usage as percentage of total configured or available memory.

Virtual machine: Percentage of configured virtual machine “physical” memory.
active ÷ virtual machine configured size
Host: Percentage of available machine memory.
consumed ÷ machine-memory-size

Idle – Consumed memory that is not actively used.

DRS Migration Thresholds – The threshold set to automatically migrate virtual machines from one host to another to optimize resource usage.

Level 1 – Apply only priority 1 migrations, vCenter will only apply recommendations that must be taken to satisfy cluster constraints like affinity rules and host maintenance.
Level 2 – Apply priority 1 and 2 migrations, vCenter will apply recommendations that promise a significant improvement to the cluster’s load balance.
Level 3 (default) – Apply priority 1, 2, and 3 migrations, vCenter will apply recommendations that promise at least good improvement to the cluster’s load balance.
Level 4 - Apply priority 1, 2, 3, and 4 migrations, vCenter will apply recommendations that promise even a moderate improvement to the cluster’s load balance.
Level 5 - vCenter will apply recommendations that promise even a slight improvement to the cluster’s load balance.

DRS Calculation

Before moving virtual machines, DRS calculates the benefits of migrating the guest based on:

Resource Usage

CPU usage
CPU read time
Active memory usage
Entitlements based on limits, shares, and reservations

Cost

CPU impact of vMotion at source
Memory consumption of "Shadow VM" at destination host
Memory reclamation impact at the destination host

Benefit

Positive benefits to the virtual machines at the source host
Positive performance gain for the migrated virtual machines at the destination host
Overall load at source and destination

A great resource on DRS is Duncan Eppings and Frank Dennemans book VMware vSphere 5.1 Cluster Deepdive.

VMware Test Lab Environment

My VMware test lab environment consists of 10 virtual machines on 5 nested ESXi hosts:

Before I setup DRS, I moved several virtual machines around to make an unbalanced cluster from a memory usage standpoint.

In the below diagram, we can see that vmt-esx02 is the most heavily loaded host and vmt-esx03 is the least utilized.

DRS Migration Level 3 (Default) Test

By default, DRS uses active memory + 25% idle memory when calculating memory load balance. This is important to note, because in some of our tests we are going to change an advanced DRS setting to move to a memory usage model.

In my first test, I set DRS to fully automated with migration level 3 (moderate) using the default DRS setting; vCenter will apply recommendations that promise at least good improvement to the cluster’s load balance.

There were no recommendations and no virtual machine moves.

DRS Migration Level 5 Test

In my next test, I set DRS to fully automated with migration level 5 (aggressive) using the default DRS setting. DRS will apply recommendations that promise even slight improvement to the cluster’s load balancing.

There were two recommendations, DRS recommended moving VMT-W2K1210 and VMT-W2K1207 to other hosts.

2-hours after the migrations, the memory consumption levels were more evenly distributed on the VMware test lab hosts.

3-hours after the initial moves, it moved VMT-W2K1206 to another host.

The memory isn’t quite as balanced, but still the load balance is much better than the original virtual machine placement and the unbalanced memory utilization.

12-hours after the migrations, there were no additional DRS moves, the workloads stayed fairly balanced, vmt-esx05 and vmt-esx03 have higher memory utilization.

DRS Migration Level 3 (Consumption) Test

Like mentioned earlier, DRS uses active memory + 25% idle memory when looking at balancing workload in a cluster. There is an advanced DRS option that can be set to use 100% of the idle memory, this changes the DRS load balancing to use consumed memory for load balancing recommendations. I used this advanced setting for two test scenarios.

When I set DRS to fully automated with migration level 3 (moderate) using memory consumption, it recommended that we make two virtual machine moves. The recommendation was the same as the DRS Migration Level 5 (Default) Test.

2-hours after the migrations, the memory consumption levels were more evenly distributed on the VMware test lab hosts.

12-hours after the migrations, the memory usage appeared to be slightly more balanced.

DRS Migration Level 5 (Consumption) Test

For my last test, I set DRS to fully automated with migration level 5 (aggressive) using memory consumption, it recommended that we make three virtual machine moves as illustrated below.

2-hours after the migrations, the memory consumption levels were very evenly distributed on the VMware test lab hosts.

12-hours after the migrations, there were no additional moves, and the memory consumption stayed very balanced.

DRS was able to spread the workload usage and make a balanced cluster with 2 to 3 vMotions, this is a small lab environment, but it was very efficient in minimizing the amount of virtual machine moves when maintaining a cluster with balanced memory usage.

With the advanced DRS option, you can configure DRS to be more aggressive with load balancing by setting 50% idle memory, 75% idle memory, or going with the consumption model like we did in this post with 100% idle memory. For CPU, you can set the advanced DRS setting of AggressiveCPUActive = 1 to react to bursty CPU workloads.

vRealize Operations 6.2 Overhead Calculation

2016-03-09T09:46:00.001-05:00

If you opened up the vRealize Operations Manager Analysis page in the latest version, you will notice it looks considerably different than it has in the past. Previously, you would expect the overhead and demand to show a smaller line than usage. In vRealize Operations Manager 6.2, they have changed the calculations for overhead, which I will walk you through.

The new overhead calculation is Total Capacity - Configured Capacity, the description of total cpacity in vCenter shows Total amount of memory reservation used by and available for powered-on virtual machines and vSphere services on the host. The new overhead model is basically taking the amount of memory that is left over after accounting for all the memory being consumed by virtual machines, vSphere services on the host, and VMkernel core functionality, such as device drivers and other internal uses and subtracting that from the configured capacity.

In the vSphere Web Client; select a host, click on Monitor, and then select the Performance tab. We are going to want to change our view to Memory and then select Chart Options.

In the Chart Options, I have unselected all the counters except Total Capacity.

In the image below, we can see that the total capacity available for powered-on virtual machines and vSphere services is on average 7480 MB.

If I subtract this from the configured capacity of 15930 MB, I have a remaining amount of 8450 MB, which is the overhead that is being used by ESXi and the virtual machines.

If I move back to vRealize Operations Manager, we notice that the overhead is 8.63 GB or roughly 8427 MB.

If you want to dive a little deeper into overhead, you can go back to the vSphere Web Client or vRealize Operations Manager All Metrics, then look at reservedCapacity which is the total amount of memory reservation used by powered-on virtual machines and vSphere services on the host including overhead amount; and sysUsage which is the amount of machine memory used by VMkernel for core fuctionaly, such as device drivers and other internal uses, it does not incude memory used by virtual machines or vSphere services.

vRealize Operations Manager Maintenance Schedules

2016-02-03T19:56:00.002-05:00

I had a customer ask me about Maintenance Schedules today, so I thought I would write up a quick post about them.

Maintenance Schedules stop vRealize Operations Manager from collecting data on a specific object. This can be helpful when resources are offline or in a state of maintenance that could make the metrics misleading. For example, if you are performing monthly patches to Windows Servers, you may not want to capture the restart of the virtual machines every month because you place an abnormal load on system demand that isn't the regular state; which could generate incorrect anomalies and alerts that affect the data for setting dynamic thresholds.

Then again, you may want to ensure the peak state is accounted for in metrics collection for capacity planning purposes. Either way, using Maintentance Schedules can serve a purpose when trying to determine steady state workloads.

To set a Maintenance Schedule, we are going to click Administration and the go to Maintenance Schedules on the Navigation panel.

On Maintenance Schedules, we are going to click the green + icon to add a new schedule. I am going to create two schedules, the first is going to be a Weekly schedule that runs from 21:30 to 22:00 every Sunday for Production Maintenance, then I am going to click the Save button.

My next schedule is going to be Development Maintenance, it will run from 18:00 to 18:30 every day.

At this point I have created two Maintenance Schedules, I now need to attach them to my policies in vRealize Operations Manager. I am going to click on Policies, select the Policy Library Tab, and then edit my Development policy by clicking the pencil icon.

Since I want to assign this to my virtual machine objects, I go to the vCenter Adapter - Virtual Machine policy and unlock the Time Range setting. If I drop-down the Maintenance Schedule, I see both the schedules I created are listed. I select the Development Maintenance schedule and save the policy.

Maintenance Schedules are easy to setup and can ensure you are capturing the data that you need.

Bada Bing Bada Boom... vROps 6.2 Upgrade

2016-02-02T10:09:00.000-05:00

One of the beautiful things about vRealize Operations Manager is how easy it is to upgrade. Version 6.2 of vRealize Operations Manager was released on January 28th, so I decided to upgrade it in my home lab.

To upgrade to the latest version, you need to download the vRealize_Operations_Manager-VA-OS-6.2.0.3445569.pak file from MyVMware. Next we are going to Launch the vRealize Operations Manager admin console, which will be https://vropsserver/admin.

After we log into the admin console, we are going to click Software Updates under the Administration panel.

The first step is to point our browser to the upgrade PAK file we downloaded. I did not check Install the PAK file even if it is already installed (because it isn't) and I didn't select Reset out-of-the-box content, overwriting to a new version provided by this update. Note, user modifications to out-of-the-box Alerts, Symptoms, Recommentations and Policies will be overwritten.

After you click Upload, you will see the PAK file progress bar.

When the upload is completed, it provides you a warning letting you know that after the installation is completed it will restart the vRealize Operations Manager cluster. Click Next to proceed.

Carefully read every line of the EULA, then check I accept the terms of this agreement.

The installation will proceed, after it is completed it will reboot the vRealize Operations Manager cluster.

At this point, the vRealize Operation Manager appliance has been reconfigured and restarted. Logging back into the admin console, we notice the version number is now 6.2.0.3445569.

It took me about 25 minutes to complete the upgrade.

So What's new in vRealize Operations Manager 6.2?

Enhanced Distributed Resource Scheduler (DRS) Integration
vRealize Operations now offers enhanced integration with the vCenter Distributed Resource Scheduler (DRS) when making and executing workload placement recommendations. The vRealize Operations Manager analytics determine cross-cluster placement opportunities, while vCenter Distributed Resource Scheduler determines the best destination within clusters. The enhanced integration uses all DRS rules, constraints, and enterprise-class capabilities.

New Workload Utilization Dashboard
The Workload Utilization Dashboard enables you to see the object workload utilization for Cluster, DataCenter, and Custom DataCenter containers. The new dashboard incorporates an updated Utilization widget, capable of operating in either a capacity or workload utilization mode.

Ability to Import Single Sign-On Users
As an Administrator, you can now add and authorize new users for vRealize Operations Manager by importing them from a Single Sign-On source.

Telemetry Enablement on Upgrade
This release includes a one-time dialog after you upgrade that allows you to participate in the VMware Customer Experience Improvement Program. This program collects anonymous product configuration and usage data to enhance future versions of vRealize Operations.

Portable Licensing
The portable licensing feature adds the ability for customers to license use of the product in vSphere as well as non-vSphere environments.

For more information on the features, go to the vRealize Operations product page.

vRealize Operations Manager Super Metrics

2016-01-12T08:44:00.000-05:00

Super metrics are the equivalent of a custom metric, it can be used to help simplify tracking by watching a few key metrics instead of many metrics. By bringing forward related data, it helps to troubleshoot specific information, which speeds time to resolution. A super metric is a mathematical formula that contains one or more metrics.

After you create a super metrics, you assign it to one or more object types. For instance, you could assign a super metrics to cluster object (resource kinds) that calculates the average memory demand on all virtual machines in the cluster.

Super metrics can be found on the Content page. vRealize Operations Manager keeps a consistent format, click the + sign to add a new super metrics, the pencil to edit a super metrics, the two triangles to clone a super metrics, the x to delete a super metrics, and clicking the gear icon allows you to import and export super metrics.

When creating a super metric, there are several different functions to choose from for your formula. They are either looping functions or single functions. Looping functions work on more than one value and single functions on a single value.

Also, there are three different categories of super metrics. They included rollup, generic resource, and specific resource or pass-through.

Rollup: The ability to apply a looping function (sum, avg, etc) to a group of resources that all have the same metrics.
Generic Resource: The ability to combine metrics on a type of resource using standard mathematical operators.
Specific resource or pass-through: The ability to combine specific resource metris and apply them to any resource.

The looping functions are:

avg: Average of collected values
combine: Combines all of the values of the metrics of the included objects in a single metric timeline
count: Number of values collected
max: Maximum value of the collected values
min: Minimum value of the collected values
sum: Total of the collected values

These are the core functions we are going to work with in this post. Other functions, such as atan (Arctangent of x) are inverse trigonometric functions that I have no intention of covering!

The last function that needs to be discussed is depth, depth supports multilayer rollups. The default depth is 1, vRealize Operations Manager searches for objects one level below the object type where you assign the specific metrics. If you have a positive number you go down the relationship chain and if you have a negative number you move up the relationship chain.

Let's build out a super metric to show the average CPU utilization for all the virtual machines on a host. First, we are going to select the looping function of avg from the Functions drop-down selector.

Next we are going to select the left parenthesis ( from the Operators drop-down, we are going to put our formula between the open and closed operators.

On the Object Types, we are going to drop down the Adapter Type drop-down list and select vCenter Adapter.

From the object types that appear, we are going to scroll down until we find Virtual Machine and select it. In the lower left panel, on the Metrics list, we are going to expand the CPU category, then scroll down and double-click the Usage (%) metric.

In the above picture, we can see that the formula shows in the top panel as avg(${adaptertype=VMWARE, objecttype=VirtualMachine, metric=cpu|usage_average, depth=1}). You need to remember after double-clicking Usage% metric, that we need to add the right parenthesis ) from the Operators drop-down to close the formula.

Give your super metric a name, you can see in my example I have used the name Average Virtual Machine CPU Usage %.

Next we are going to verify the super metric formula is working. Click the Visualize Super Metric button. On Object Types, click the Adapter Type drop-down menu and select vCenter Adapter. Next scroll down and select Host System, then select one of the child object hosts listed in the objects panel. If information is displayed in the visualization window, you know the formula is correct.

After you verified that your super metric works, we need to add it to an object. On the Objects Types tab we are going to click the + button to add a new object type.

Expand the vCenter Adapter object and then add Host Systems.

Before we can see our super metric in All Metrics on the Troubleshooting tab, we need to add it to the default policy. We are going to edit the vSphere Solution's Default Policy and go to Collect Metrics and Properties.

Drop-down the Attribute Type filter and uncheck Metric and Property, this will bring up only the super metric you created.

Change the State of the super metrics from Inherited to Local and save your changes.

Our super metrics should now be available in All Metrics under Troubleshooting, because we set the Object Type as Host System, we will only see this super metrics when a host has been selected. In the picture below, I have expand the Super Metrics tree and Average Virtual Machine CPU Usage % is available for labesx07.home.virtlab.com.

After double-clicking the metrics, a chart for the average virtual machine CPU usage % on labesx07 displays on the right; below that chart is CPU usage of the two virtual machines on the host. The SCOM virtual machine is running between 4.14% and 4.76% and the VMware vCenter Server Production virtual machine is running between 8.43% and 9.58%. The virtual machine average for both is 6.45% at that specific point in time.

This is merely a sample of what you can do with super metrics. Super metrics are really powerful, they are certainly worth exploring to find out how it can enhance your monitoring capabilities.

EMC ScaleIO Overview

2015-12-28T16:12:00.000-05:00

EMC ScaleIO is a flexible software-only solution that leverages host-based internal storage media to make a scalable virtual storage pool.

In that respect, there are three primary components that make up EMC ScaleIO:

ScaleIO Data Client (SDC)
ScaleIO Data Server (SDS)
Metadata Manager (MDM)

The ScaleIO Data Client (SDC) is a block device driver that exposes ScaleIO storage volumes to applications. The SDC runs locally on any application server that requires access to the block storage volumes. The blocks that the SDC exposes can be blocks from any device in the ScaleIO storage pool. This enables the local application to issue an I/O request and the SDC fulfills it regardless of where the particular blocks reside.

The ScaleIO Data Server (SDS) possesses local storage that contributes to the ScaleIO storage pools. An instance of the SDS runs on every server that contributes some or all of its local storage space. The role of the SDS is to perform I/O operations as requested by an SDC on the local or another server within the cluster.

The Metadata Manager (MDM) holds the cluster-wide mapping information and is responsible for decisions regarding migration, rebuilds, and all system-related functions. It manages the ScaleIO system. The MDM is installed on at least three servers and functions as a quorum; a primary MDM server, a secondary MDM server, and a tie-breaker. The ScaleIO monitoring dashboard communicates with the MDM to retrieve system information for display in the ScaleIO GUI. The MDM is not on the ScaleIO data path, reads and writes never traverse the MDM.

In a VMware environment, you use the ScaleIO vSphere plug-in deployment wizard to install the MDM and SDS components on a dedicated ScaleIO virtual machine (SVM); whereas the SDC is installed directly on the ESX host. This is something new as of version 1.31, the SDC is installed inside the hypervisor using a vSphere Installation Bundle (VIB).

In an ESX environment, you need to decide how devices are going to be added to the SVM. You can add the devices using RDM mapping, a device is created on the SVM that points to the physical disk on the ESX; or you can create it with a VMDK, and the VMDK is added to the SVM. ScaleIO requires thick provisioning for VMDK, so this process can take a long time. VCE highly recommends going with RDM mappings in their installation guide.

To access EMC ScaleIO you can use the vSphere Web Client plug-in or the Scale IO GUI, the plug-in communicates with the MDM node and the vSphere server, enabling you to view components as well as to perform configuration and provision tasks right from within the vCenter environment.

There are five elements that make up ScaleIO’s virtual storage pool:

Protection Domains
Storage Pools
Fault Sets
Volumes
Chunks

EMC ScaleIO uses the concept of protection domains, which is a unique set of ScaleIO Data Servers (SDSs) grouped together for reliability and tenancy separation. In the diagram below, we see there are four protection domains which include a Dev/Test Environment, VDI Desktops and SQL DB, Oracle and SAP HANA, and a NoSQL DB environment. Protection domains are logical groupings, such as business specific tenants, application groups, or geographic separation.

In the vSphere Web Client plug-in, you add a new protection domain by creating a name and selecting the size of the RAM Read Cache per SDS. The RAM Read Cache is the memory that is reserved for caching storage devices in the storage pool. ScaleIO does not have auto-tiering between different storage media (SDD and HDD), instead it uses the RAM Read Cache to help accelerate read requests.

By default, all volumes and storage pools are configured to use caching, and all SDSs have caching enabled using a cache size of 128 MB. You can configure individual SDSs in the storage pool with different RAM values, or disable caching completely. The maximum amount of RAM cache is 128 GB.

As of version 1.31, RAM Read Cache can be enabled for an entire SDS, a storage pool, or a specific volume.

A storage pool is a sub-set of physical storage devices in a protection domain. Each storage device can only belong to one storage pool. Storage pools allow the creation of different storage tiers in the ScaleIO system. For illustration, the SSD drives can be pooled together in Volume 2 to create Tier 1 storage services and your spinning media could be your Tier 2 storage pool.

A fault set is a logical entity that ensures that SDS data is backed up on other hosts that belong to alternate fault sets, thus preventing double-point-of-failure scenarios if rack power outages occur. This is similar to Virtual SANs fault domains.

A volume consists of multiple blocks spread evenly on a storage pool’s devices. A single volume is divided into chunks. These chunks will be stripped on physical disks throughout the storage pool, in a balanced and random manner.

Each volume block has two copies located on two different SDSs, this is called ScaleIO mesh mirroring. This allows the system to maintain data availably following a single-point failure. The data will still be available following multiple failures, as long as each failure took place in a different storage pool.

In the diagram below, we can see that there are two copies of each chunk on different SDS nodes. SDS 1 contains chunk F from Volume 1, and Volume 2 contains chunk E and B.

To illustrate how mesh mirroring protects in the event of a failure, when SDS 1 goes down it begins a forward rebuild with the chunks on the failed device. The degraded chunks will be rebuilt on a new SDS node in a balanced way to help accelerate performance. Chunk F from Volume 1 is recreated on SDS 5, chunk B is recreated on SDS 100, and chunk E is recreated on SDS 4.

To create a storage pool in the vSphere Web Client, you simply give the storage pool a name and select the appropriate protection domain.

Next we select the hosts that will participate in the protection domain and provide storage capacity as SDS resources.

The below screen shows the devices whose free space can be added to the selected SDS. To add a device’s space, select the Add Device box and choose the storage pool it will be assigned to.

After everything is configured, you can drill down into the summary of the protection domain and all its related objects.

In addition to the vSphere plug-in, you can use the Scale IO GUI. You access the ScaleIO GUI by pointing to the IP address or host name of the MDM node.

The ScaleIO GUI is very intuitive, the main dashboard provides an overview of the ScaleIO environment, including the total amount of capacity, I/O workload, mapped volumes, protection domains and storage pools, SDCs, SDSs and the number of devices, and the MDM cluster status under management.

The main dashboard also indicates if ScaleIO is currently rebuilding RAID 1 data. A rebuild is usually a result of a recovery due to failure of a server or a storage device.

The backend view provides detailed information about objects in the system, and lets you perform various configuration operations.

The Command menu button on the backend veiw displays a list of commands which you can perform on rows selected in the table. The contents of the Command menu differ, depending on the object selected in the table. Many of the commands can also be accessed from the context-sensitive menu when table rows are right-clicked.

The different commands include:

Overview
Capacity Usage
Capacity Health
Rebuild and Rebalance
Application I/O
Overall I/O
I/O Bandwidth
State Summary
Configuration
Device Details
RAM Read Cache
Rebuild and Rebalance (Detailed)
Planned Rebuilds (Advanced)
Planned Rebalancing (Advanced)
Rebuild I/O Priority (Advanced)
Rebalance I/O Priority (Advanced)
Network Throttling (Advanced)
RAM Read Cache (Internal)

The Alerts view provides a list of the alert messages currently active in the system, in table format. You can filter the table rows according to alert severity, and according to object types in the system.

The ScaleIO system includes advanced I/O priorities and bandwidth settings, which can be used to fine-tune system performance. The number of concurrent rebuild and rebalance jobs can be configured, and bandwidth for rebalance jobs can be adjusted. I/O prioritization is configured per storage pool.

Network throttling affects network limits, and is used to control the flow of traffic over the network. It is configured per protection domain. The SDS nodes transfer data between themselves. This data consists of user-data being replicated as part of the RAID protection, and data copied for internal rebalancing and recovery from failures. You can modify the balance between these types of data loads by limiting the data copy bandwidth. This change affects all SDSs in the specified protection domain.

Security Controls Document

2015-12-24T10:21:00.001-05:00

The VMware Security Hardening Guides contain recommended processes for deploying and operating VMware products in a secure manner given a specified risk profile. You may not need, or may not be able to follow each step of the security hardening guides because of the balance of operational efficiency, cost, risk tolerance, and security requirements. The security hardening practices are recommended by VMware, but equally important is having a security controls document that incorporates VMware best practice recommendations and your specific security policies. It can be an invaluable tool during an audit.

Security has a wide scope that touches every aspect of the datacenter; an important part of security is recognizing the tolerance of risk. To do that, you need to understand the value of the assets you are trying to protect and the cost of protecting that asset. What is the likelihood of the asset being damaged or compromised? And what does it cost the company if that asset is compromised? A risk analysis provides a cost/benefit understanding of the cost to safeguard an item compared with the expected cost of loss. The security policy should be proportionate to the value of the asset, which may range from innocuous data processing up through a mission critical business process dealing with highly sensitive information. Each of these examples represents a different risk profile, which translates to different security requirements and thus different recommendations in the hardening guide. Securing systems is not an inexpensive endeavor. Even in terms of operations expenses, locking down systems can make internal operation teams far less inefficient on updating systems because of strict security controls. In many cases, a security policy will not be implemented unless the cost of the loss exceeds the security policy itself.

In certain instances, a customer may not require or desire the level of controls outlined in the VMware Security Hardening Guides; but in other cases customers may have stronger controls already implemented due to regulatory compliance. In the end, you are the one that's best suited to make the decisions on the security posture of your IT assets.

Security Controls Document

Controls and audit compliance includes the process and mechanisms companies utilize to comply with laws such as Sarbanes Oxley or standards such as COBIT. In general, the larger the company, the more these regulations and practices are going to affect the organization. The trick is effectively implementing security hardening best practices across your organization in the most efficient way possible.

A security controls document provides the specific controls and options that are implemented in your environment. They are the standards that determine how hardware and software products are to be used in your organization. You can have them divided up into different indexes or appendices in the document based on the extent and scope of the security controls document.

Operation controls may include:

Resource protection
Hardware controls
Software controls
Privileged-entity controls
Media controls
Physical access controls

For our instance, we are going to focus on virtualization; in the sample document we will cover VMware vSphere and VMware View. The document contains tables that describe the security settings that are specific to each category. The sample document is just that, a sample, and not an extensive list of all the controls that you may want to include in your security controls document.

To understand security controls, let’s assess the common risks that we find in the modern datacenter.

Physical damage
Human error
Equipment malfunction
Inside and outside attacks
Misuse of data
Loss of data
Application failure

As we can see, these are some of the everyday threats we face when managing a data center. What is a threat? Simply put, it is any event that can cause damage to a system and create loss of confidentiality, integrity or availability. Risk comes in many forms, not all just computer related; not surprisingly the number one risk is human error. Security controls are enacted to mitigate the risk of human error, attacks, and misuse of data and to limit exposure in those events.

But, we also need to determine the overall value of an asset. A hardened security posture for a system that provides little company value and doesn’t impact the rest of the environment may not be cost effective.

Some of the costs we need to take into consideration include:

Cost to acquire or develop the asset
Cost to maintain and protect the asset
Value of the asset to the business
Value of intellectual property
Productivity that is affected if the asset is unavailable
Liability issues if compromised

Understanding the value of your assets and the support required for the asset to meet your business expectations is the starting point for understanding the security mechanisms that will compose your security controls document.

Security Controls Cycle

Below is a table that illustrates a standard security controls lifecycle.

Task	Start Time
Security Controls Baseline Created · Create baseline security controls document · Review with stakeholders · Publish the document
Security Controls Implementation · Document current values · Project plans · Security controls remediated	When the Security Controls document has been published
Semi-Annual Review · Determine if update is required · Document draft · Review with stakeholders · Final copy published	6 months after the published date or after the last annual review end
Semi-Annual Review Implementation · Document current values · Project plans · Security controls remediated	When the semi-annual review ends
Quarterly Health Check · Ensure that the controls that were in place are maintained	Every 3 months

Like in my previous post, we can ensure that the controls that were in place are maintained by using a tool like vRealize Operations Manager. We can use the Compliance feature to check for drift in our controls policy.

Risk Profile

A risk profile is a way to group similar systems together for the purpose of defining applicable security controls that are appropriate for specific groups. For example, a database server that stores Protected Health Information (PHI) for a healthcare organization will require security controls that are far more rigid than another database server that stores less valuable data. Just as it is a poor practice to unduly relax security controls on a high-value resource, it’s also a poor practice to impose overly rigorous controls on low value resources.

Recent versions of the VMware Hardening Guides provide recommendations for three different risk profiles:

Risk Profile 1: Guidelines that only be implemented in the highest security environments, e.g. top-secret government or military, extremely sensitive data, etc.
Risk Profile 2: Guidelines that should be implemented for more sensitive environments, e.g. those handling more sensitive data, those subject to stricter compliance rules, etc.
Risk Profile 3: Guidelines that should be implemented in all environments

For convenience, we can think of these as high risk, medium risk, and standard risk. Customers will need to evaluate which of their resources belong in which category. Some customers may not even use all categories, while others may find the need to define additional categories. In general, these three will provide most customers with a good start.

Security Baseline

The Security Baseline is the set of specific security controls for a resource that falls within a specific risk profile. This baseline will vary from resource to resource, for example a vCenter Server will have a different set of specific controls than an ESXi server even if both systems share the same risk profile. Think of risk profile as a general ranking of the criticality of a resource and the security baseline as the specific controls (or “settings”) required to meet that profile. Additionally, business requirements may dictate that specific systems have a higher level of protection due to regulatory compliance or because it is a mission critical system. For example, the security baseline for the password policy may be a minimum password length of 8 characters with upper case letters, lower case letters, and numbers. However, you may have applications that hold financial information or client medical records that require a higher level of security, such as a minimum password length of 12 characters with upper case letters, lower case letters, numbers, and symbols. The security baseline is the minimum security settings required. Any unique requirements below the baseline should be documented as a security exception.

Document Retention

Typically, you only need to retain the current version of your security controls document. However, there is a window when you may need to retain the latest and previous versions. If you have finished a review, and there are control processes that need to be modified, you should retain the previous version of the security controls until the implementation is completed. If you don’t do this, you may have a problem if an audit/review occur before the implementation is complete.

Security Appendices

If the security controls document covers several technology sections, you can break them up into individual appendices with the appropriate Executive Sponsor.

Appendix Title	Executive Sponsor
A. vSphere	Joe Smith
B. Windows Servers	Jane Poole
C. Network Infrastructure	Mike Doe
D. Oracle	Fred Cooper
E. Apache Webserver	Nancy Welsh

Here is a sample security controls document I put together that you can download:

Sample Security Controls Document

vRealize Operations 6.1 Compliance

2015-11-30T14:18:00.001-05:00

vRealize Operations 6.1 is packaged with it's own compliance engine, the compliance engine helps you monitor the security best practices of your vSphere virtual machines and hosts, which is essential in keeping your IT auditors happy! vRealize Operations utilizes the vSphere 5.5 Security Hardening Guide, hopefully we will have it updated to the latest version in an upcoming release, but it is still invaluable for providing insight into the security posture of your virtual infrastructure.

To start, we need to enable the alert definition in our base policy. The ESXi Host is violating vSphere 5.5 Hardening Guide and Virtual Machine is violating vSphere 5.5 Hardening Guide alert definitions aren't enabled by default. I am going to enable both alert definitions in my vSphere Solution's Default Policy, all child policies I have created underneath this base policy will inherit the setting. If you aren't familiar with vRealize Operations policies, I wrote a blog post covering the creation and modification of policies called vRealize Operations 6.1 Policies.

When editing the base policy, you are going to jump to step 6 Alert/Symptom Definitions and filter on hardening. This will display both of the alert definitions, you will notice that the State is local disabled.

To enable the alert definition, we are going to use the drop-down list under State and select local enabled (green check). You will notice that there are 20 symptom definitions for ESXi hosts and 49 symptom definitions for virtual machines. After we have modified both policies to enable the alert definition, we are going to save our base policy.

Let's take a look at the alert symptoms, we are going to click on the Content icon and select Alert Definitions.

Next we are going to filter on hardening, this will bring us to our ESXi host and Virtual Machine alert definitions.

If we edit the ESXi Host is violating vSphere 5.5 Hardening Guide alert definition it shows us the list of the 20 symptoms. We can negate these symptoms by clicking on the cross-though icon or delete the symptom by clicking the x icon. This provides modest flexibility to tailor the alert definition based on a custom security controls document for your organization. In my next post, we will dive into creating a security controls document to establish your security guidelines.

When a host or virtual machine is not complaint, it will show up as an alert under the Risk badge. In the image below, we have 6 ESXi host objects and 3 virtual machine objects that aren't compliant with the vSphere 5.5 Hardening Guide Rules. It also provides us with a link to download the vSphere 5.5 Hardening Guide spreadsheet.

If we look at the Compliance tab under Analysis for one of my ESXi hosts, I can see that it is 100% compliant with the vSphere 5.5 Hardening Guide.

Highlighting the compliance standard allows you to view the violated rules or all the rules. This detailed overview allows you to scroll through all the symptoms and provides visual indicators of the status. If you have modified the default alert definition for the ESXi host, the removed items will not show in the All Rules view.

There is no option to automatically remediate non-compliant systems with the Python Actions adapter, essentially it is a read-only view of definition rules. While the vRealize Operations compliance engine isn't as powerful as vRealize Configuration Manager, it isn't nearly as complex to deploy and manage; it provides a solid understanding of the vSphere systems that aren't compliant with your specific security guidelines.

As mentioned previously, in my next blog post I will walk through the steps of creating a security controls document.

vRealize Operations 6.1 Rebalance

2015-11-12T16:43:00.001-05:00

If you read my post from last week about vRealize Operations 6.1 Policies, one of the policy options I discussed was workload automation. Workload automation in vRealize Operations Manager 6.1 allows you to balance your infrastructure environment, the Action can be done at the datacenter or cluster level to balance workloads across your clusters and hosts.

The Workload Automation policy allows you to balance out the workloads conservatively or aggressively, depending on your infrastructure requirements; and you can decide if you want more or less consolidation for host efficiency.

I thought I would step through the process of performing a workload rebalance at the datacenter level. And because it is fairly straight forward, this should be a short post!

I am going to highlight my PaloAlto datacenter, which has my 6.0 cluster.

I am going to click Actions and then select Rebalance Container.

vRealize Operations is going to calculate the workload plan taking into consideration any changes you have made at the policy level.

After the calculation is completed, it gives you a list of virtual machines that could be balanced to other hosts. You will notice that my first virtual machine will move from the source host to a new destination host, but the datastore destination is staying the same. It also provides you with the reason for the rebalance.

vRealize Operations 6.1 Policies

2015-11-06T21:34:00.004-05:00

Policies are ways we can apply different metrics, filters, buffers, and thresholds to a specific object in the vRealize Operations inventory. This gives us a much finer layer of control of how we monitor our data. Mission critical applications, such as customer facing applications and financial systems are vital to the smooth operation of a company's business. These applications are core to the company's mission, and system downtime translates to financial losses to the organization. While other applications, like general-purpose printing, software media libraries, and infrastructure monitoring tools don't require the same service level capabilities as mission critical applications.

This is where vRealize Operations Policies come into play! It gives us the ability to provide policy controls for specific service levels for various application workloads.

To start we are going to click on the Administration Link and then Policies.

When you open up the Active Policies, it shows all the policies that have associated objects assigned. Initially, there will only be the vSphere Solution's Default policy, which shows the priority rating of D. As you add additional policies, it will show the priority ranking. In the image below, we recognize the Gold SLA is priority 1, Silver SLA is priority 2, Bronze SLA is priority 3, Production is priority 4, and Development is Priority 5. If an object is a member of multiple policies, it uses these rankings to break the tie. As we will see shortly, my Gold SLA is nested in the Production policy, because of the priority ranking 1 all objects assigned to the Gold SLA policy will use this policy.

You can change the ranking of the policies simply by dragging them into the respective priority slots.

You will want to use policies with custom groups, custom groups is outside the scope of today's post, but if we look at my Environment custom groups I have organized my development, production, production bronze, production silver, and production gold virtual machines and applied these custom policies to the groups. This is the sweet spot for policies, like we discussed above we cannot treat all IT datacenter systems the same, some systems are more critical to the operation than others.

To create a new policy, we are going to click on Policy Library tab and the click the green + sign. For my example, I am going to build my policies off the vSphere Solution's Default Policy.

Let's walk through the build process for my Bronze SLA policy. First, I am going to give my new policy a name and a description, then select to Start With vSphere Solution's Default Policy.

In most circumstances, you will not make changes to the Select Base Policies, but if you wanted to modify the settings from the configuration wizard during the original installation, you can select to apply a new policy from the Overide settings from additional policies: option. In my case, since this is a bronze level SLA, I change it to VMware Test and Dev Policy (w/o Allocation).

For Analysis Settings, I am going to scroll down to the vCenter Adapter, my Workload policy has already been modified due to the override settings I specified on the base policy. I notice that my orange Workload Score Threshold is OFF, I want to change this so that I get notified at 95%.

I drag my yellow Workload Score Threshold to 90%, then double click my orange threshold and drag it to 95%.

Clicking the lock icon next to a given policy enables you to edit the value, which overrides the default setting from the base policy. You select lock to inherit settings from the parent policy.

In Time Range, I am going to unlock the policy and modify the Track Usage to only the work week by unchecking Saturday and Sunday.

You can set the workload automation options for your policy, so that vRealize Operations Manager can balance the workload in your environment. Rebalancing is an Action option available at the datacenter and cluster level to balance workloads across your hosts and clusters.

For Balance Workloads, select Aggressive balancing when you have stable populations. It minimizes contention but moves workloads more, which can cause disruption. Consider Conservative balancing when you have dynamic populations. It exposes potential contention, but moves workloads less.

For Consolidation Workloads, select more consolidation when you have virtual machines with steady demand. It puts workloads into as few hosts as possible to reduce licensing and power costs. However, this might cause less responsive capacity. Choose less consolidation when you have virtual machines with irregular demand. It uses all available hosts, which leaves more room for demand spikes. However, this increase licensing and power costs.

For each object type, you can override the attributes that vRealize Operations tracks. Select Enabled next to each item you want to track, Disabled for items you do not want to track, or elect Inherited to use the settings from the base policy. Like Select Base Policies, this is a setting you will rarely need to change, but it does give you the ability to become very granular when creating your policies

vRealize Operations uses alert definitions and symptoms to identify problems. Like Collect Metrics and Properties, you can override which alert definitions and symptoms will be used. Select Enabled next to each item you want to track, Disabled for items you do not want to track, or elect Inherited to use the settings from the base policy.

Profiles show you how many virtual machines will fit with the capacity remaining. By default you get four profiles, which include average, large, medium, and small. You can add new profiles on the Capacity Remaining tab under Analysis.

Like the previous settings, you can choose Enabled next to each item you want to track, Disabled for items you do not want to track, or elect Inherited to use the settings from the base policy.

To finish up my policy, I am going to select my Production Bronze Virtual Machines group and click Save.

For my gold and silver virtual machine policies, I am going to step through the same settings; however I am not going to use VMware Test and Dev Policy (w/o Allocation) in the Select Base Policies. Also, on the Analysis Settings, I am going to lower the Badge Score Thresholds for Workload and Anomaly, so that I get notified earlier in the event of a potential resource constraint.

To do this, I am going to add the vCenter Adapter – Host System and Virtual Machine, then switch on the individual analysis types to override the settings. I adjust the bars for yellow, orange, and red to be more conservative in my analysis.

After my policies are created, I can highlight it in the Active Policies window and select the Complete Settings Including Inherited tab to view all the details for the policy.

You can create new child policies underneath policies you previously produced, in the image below you will notice I created three nested policies under the Production policy, they are Bronze SLA, Gold SLA, and Silver SLA. The Silver SLA policy will inherit settings from the Production and vSphere Solution's Default Policy, and will override any settings that I modify.

As you will notice, there is a lot more to policies in vRealize Operations 6.1 then there was with 6.0. It is a powerful tool that can help set the right monitoring policy to meet application business requirements.

What is Driving Server SAN Adoption?

2015-10-22T17:30:00.000-04:00

Business Challenge

Why are IT leaders so interested in server SAN solutions? Let's look at some of the challenges facing IT departments. Computer Economics reported that there was a nominal increase in IT budgets from 2.4% in 2014 to 3.0% in 2015, more companies are investing in information technology; but that doesn’t show the complete picture of IT investments.

IT Budget Change from Prior Year

When we look at the IT spend as a percentage of revenue, Computer Economics reports that it decreased in 2015 to 2.3%. Even though there was an uptick in IT budgets, the amount of IT spend as a percentage of revenue declined showing IT budgets are not keeping pace with company growth.

IT Spending as Percentage of Revenue

To illustrate the point further, there has been a steady decline of the total IT spend per user. In 2012 the average IT spend per user was $10,514.00 compared to todays $6,847.00. Companies are investing more in IT, but not at the same rapidity as corporate growth.

Total IT Spend per User

This places increased strain on the IT department to innovate because most of the IT budget goes to running the environment. According to the Gartner IT Key Data Metrics, only 14% of the IT budget is invested in transformation, while 67% of the IT spend is consumed by running and maintaining the existing environment.

Most IT Budgets Still Focus on Run

Business unit heads are expecting IT leadership to provide technology innovation to help drive corporate revenue and profits, however with only 14% of IT spend is funneled to transformation. CIOs and IT senior management are looking for new opportunities to focus on strategic innovation and getting out of the business of tactical maintenance and upgrades.

IT Growth and Staffing

Even though virtualization has brought a tremendous amount of cost savings, infrastructure flexibility, and increased application availability, it has also brought a substantial amount of growth due to ease of deployment.

Below is the growth of one of VMware’s customers in healthcare. They have increased from 400 virtual machines in December 2008 to 3,930 virtual machines in March of 2015. That is an 882% increase in virtual machine growth over the past 6 years. Like most organizations, staffing over that timeframe has remained flat, the number of virtual machines each engineer is supporting has grown from 51 virtual machines per engineer in 2009 to 623 virtual machines per engineer today. That is a staggering increase!

Customer Growth and Staffing

So why haven’t organization increased headcount? Even though IT leaders understand the strain being placed on internal IT, they don’t have the capital to invest on staff to support the infrastructure sprawl. According to Gartner IT Key Data Metrics, 40% of IT spend goes to personnel salaries and benefits. It is twice the amount of hardware and software combined.

IT Spending on Hardware, Software, Personnel and Outsourcing

Even though businesses have increased their application portfolio supporting business initiatives, there is no way to justify the headcount at even half the pace of infrastructure growth. This not only stifles innovation, it makes it so most corporations are in constant lifecycle management of their independent infrastructure elements (compute, storage, networking).

This is driving companies to look for new solutions.

Driving Toward Simple Solutions

With the enormous disconnect between the business expectation to become nimbler to foster corporate profitability and the trend to “Do More with Less”, an opportunity has taken root for new forms of infrastructure solutions to displace traditional approaches. IT leaders are looking for ways to reduce time, reduce risk, and reduce costs by getting out of the business of integrating systems. For over a decade, labor-intensive maintenance and upgrade procedures with disjointed management tools have drained IT staff productivity, which ends up driving up the cost of IT operations. Converged and hyper-converged solutions bring together server, storage, networking, and management resources into a pre-integrated system designed and optimized for the software-defined datacenter.

According to Gartner by 2018, 40% of midsize enterprises will replace all data center servers and storage with integrated systems.

In a recent poll, 62% of CIOs said their primary driver for adopting converged infrastructure was buying a pre-certified configuration and having all the components supported by a single vendor. They are looking for frictionless solutions, like mentioned above they don’t want to be in the business of integrating systems any longer. There is no strategic value in do it yourself (DIY) traditional deployments.

CIO Influence in Decision on Buying Converged Infrastructure

In fact, an IDC study on The Business Value of Converged Systems showed some startling statics on the benefits of moving to pre-integrated solutions.

36% reduced IT infrastructure and IT infrastructure staff costs
41% less IT time spent keeping the lights on
338% more time spent on business enablement
2.1% productivity increase
55% faster application development life cycle
4.4 times faster time to market for services and products
2.4% higher revenue

Because of the current business challenges, the integrated server SAN market is about to blossom. IDC is predicting 68% compound annual growth rate by 2018. Traditional storage arrays are predicted to decline by 26.4%.

Integrated Server SAN Growth Rate

If you are interested in looking at sever SAN solutions, there are many different offerings ranging from software-define storage solutions like VMware Virtual SAN to complete converged solutions for software-defined datacenter. I highly recommend talking to your Value Added Reseller (VAR) or VMware Pre-Sales team to understand the product offerings.

vRealize Operations 6.1 vSphere VMs Memory dashboard

2015-10-21T13:34:00.000-04:00

In today's post, I wanted to explore the vSphere VMs Memory dashboard in vRealize Operations 6.1 to find potential memory contention issues. But before we dive into vRealize Operations, I wanted to review memory management in vSphere.

Let's start by looking at Utilization on the Monitor tab of the vSphere Web Client to understand the memory consumption of a guest virtual machine. In my example below, we are looking at my vRealize Operation Manager 6.1 appliance.

On the Virtual Machine Memory widget, VM Consumed is the amount of physical memory consumed by the virtual machine for guest memory. Unlike host consumed memory, virtual machine consumed memory does not include overhead memory.

The bottom bar is VM Overhead Consumed, which is the amount of machine memory used by the VMkernel to run the virtual machine.

Some of the factors that affect the memory overhead consumption are the number of virtual CPUs, the amount of virtual memory, number of virtual devices, and the capabilities of the physical processor. The table below provides a sample of the memory overhead values for a virtual machine.

The next widget is Guest Memory, which displays several memory counters. Because of the VMkernel's memory-management techniques for allowing overcommitment, guest physical memory mapping does not have a 1 to 1 correspondence. Managing the memory in the hypervisor enables sharing of similar data (such as redundant copies of the same guest OS in memory pages), memory compression, memory swapping, and memory balloon technique, for efficient host memory reclamation.

Here is a description of the memory counters displayed:

Active Guest Memory: The amount of guest physical memory actively being used (Demand in vRealize Operations)
Private: The amount of non-shared physical memory stored on the host's memory chip
Shared: The amount of guest physical memory shared with other virtual machines through Transparent Page Sharing (TPS)
Compressed: The amount of virtual memory that has been compressed by the VMkernel
Swapped: Current amount of guest physical memory swapped out to the virtual machines swap file by the VMkernel
Ballooned: Memory reclaimed from the guest by the ballooning driver
Unaccessed: The amount of memory that has been untouched by the virtual machine

Transparent Page Sharing (TPS) is when pages of memory that are identical take up the same physical space on the physical memory card. Transparent Page Sharing can take place on a single virtual machine or multiple virtual machines on an ESXi host. By default, inter-virtual machine TPS is no longer enabled, but you can adjust the setting in advanced properties.

The memory balloon driver collaborates with the host to reclaim pages that are considered least valuable by the guest operating system. The driver uses a ballooning technique that provides predictable performance that closely matches the behavior of a native system under similar memory constraints. This technique increases or decreases the memory pressure on the guest operating system, causing the guest to use its own memory management algorithms. When physical memory is under contention, the guest operating system determines which memory pages to reclaim physical memory and if necessary swaps them to its own virtual disk.

Host swapping can have a heavy performance impact and is used as a last resort for reclaiming pages. The host drive swaps lower priority pages on to the physical disk where it can still be accessed. However, memory backed by a physical disk has significantly higher latency than backed by physical memory. Swapping out memory to physical disk is not necessarily bad, but swapping in memory from disk can cause significant performance issues with applications.

Once memory is being swapped, pages that can be identified with a minimum 50% compression ratio may be compressed and kept in physical memory. The host zips lower priority pages, so that two or more pages can occupy a single page in physical memory. Compressed memory has small overhead on reading memory; however writes to compressed memory can slow down processing.

vRealize Operations 6.1 provides a dashboard called vSphere VMs Memory. The dashboard includes six widgets, four of them are heatmaps and two top-n dashboards. They include:

VMs Heatmap Sized by Memory Demand (%) and Colored by Memory Swapped (KB)
VMs Heatmap Sized by Memory Demand (%) and Colored by Memory Balloon (KB)
VMs Heatmap Sized by Memory Demand (%) and Colored by Memory Swap In Rate (KBps)
VMs Heatmap Sized by Memory Demand (%) and Colored by Memory Compressed (KBps)
Top 25 VMs by Memory Demand (%) (24h)
Top 25 VMs by Mem Swapped-in (KBps) (24h)

Looking at the dashboard, I notice that my vRealize Operations Manager Appliance is colored red for memory swapped (KB), I can hover over the box for more detailed information including the Sparkline. If we look at the Top 25 VMs by Mem Swapped-In (KBps) (24h) we notice that vRealize Operations Manager Appliance is listed as number three, but it is only recording a swap-in rate of 0.03. Even though my appliance is swapping to disk, it is not swapping in much of the memory from the disk.

On the other hand, my vCloud Connector Server and vCloud Connector Node are experiencing significant swap-in rate of 104.6 and 78.03.

If I switch back to the vSphere Web Client, on Performance of the Monitor tab I recognize I had a few spikes for swap-in rate. I am not overly concerned about performance issues, but it is something I may want to explore further.

In vRealize Operations, the Workload for my vRealize Operations Manager Appliance is 46 and green. Workload shows the demand for physical resources. The Workload badge is calculated by the most utilized resource, in this instance memory is the most heavily used resource with 46.27%.

As we discussed previously, ESXi allocates physical RAM only as necessitated. My vRealize Operations Manager Appliance has 8 GB of memory allocated, but is currently only demanding 3.7 GB of memory (Guest Active Memory). Usage is the combination of the virtual machine private memory and the VM overhead consumed. On my vRealize Operations Manager Appliance, there is 7.48 GB of private memory and 51 MB of VM overhead consumed, which totals 7.53 GB of memory usage.

Again everything looks pretty good for my appliance. But, I want a deeper examination of my virtual machine memory. Under Further Analysis, I am going to select Virtual Machine Memory Diagnose to bring up the respective view. Views provide details on the specific object that has been selected.

The Virtual Machine Memory Diagnose shows the last 24 hours of memory demand, memory consumed, memory ballooning, memory swapping, and memory compressed for my vRealize Operations Manager Appliance. This diagnoses can be extended out past 24 hours by clicking the calendar icon, as shown below.

We can see that my vRealize Operations Manager Appliance is showing memory swapping, compression, and sharing; however, there is no ballooning happening. The demand for the virtual machine is consistently around 30% to 40%.

If I want to take a long term analysis, I can select the Virtual Machine Memory Demand Forecast Trend to ensure that my demand isn't going to cause a problem in the next 30 days.

The last item I want to look at is Capacity Remaining. Because this virtual machine hasn't been on for more than 30 days I am only getting a partial trend, but it does show me that I have enough memory capacity on the virtual machine and it is recommending that I could lower the amount of memory on my vRealize Operations Manager Appliance to 5.06 GB to right-size the resource.

vRealize Operations Manager gives you great perspective on memory performance, capacity, and trending for your virtual machines; and the vSphere VMs Memory dashboard can be the launching point.

vRealize Operations 6.1 Host Memory Consumed Model

2015-09-22T20:13:00.000-04:00

vRealize Operations 6.1 introduces a memory consumed model for hosts and clusters, which models consumption from a guest point of view. Previously, vRealize Operations used a memory demand model based on active memory usage from a vSphere perspective. If you have greedy applications like JVMs, Oracle, or SQL server running; the application reports a much higher memory usage than the actual active memory that ESX perceives. This is because these types of applications claim all available memory at the VM level as usage (hence greedy).

This new memory consumed model through me for a loop at first, when I looked at memory workload for my host, I couldn't figure out why the host demand was so much higher than host usage and virtual machine demand. Then speaking with members from our vRealize Operations development and technical marketing team, they educated me on the change to a memory consumed model and that it was labeled under Memory, as show in the image below.

I was looking at the legend which still indicated demand and to me that was active memory. After that was cleared up, I recognized another issue, when I powered down all the hosts in a cluster there was still host demand and the demand was the same amount as the overhead. Notice in the picture below, both components are reading 1.47 GB.

That is because according to the Memory Counters Reference Guide, consumed memory includes memory used by the service console, the VMkernel, vSphere services, plus the total consumed metrics for all running virtual machines. Unfortunately, we are double counting the overhead. On my lab host, overhead is 19.37% of the allocated resources, there is an additional 19.37% being added because of the consumed memory model; which makes it appear that there is much higher memory consumption than is actually happening on the host. A side affect is that it will trigger alerts early.

This is easily noticeable when navigating to All Metrics and selecting ESX System Usage (KB) and the Consumed (KB). The first image shows the ESX System Usage (KB) at 1,544,011.75.

The second image shows Consumed (KB) at 1,544,011.75. The Consumed (KB) drops to match the static ESX System Usage (KB) after the virtual machines have been powered down. Both are observing the memory used by the service console, the VMkernel, and vSphere services.

From the Performance tab on the vSphere Web Client, consumed memory never goes all the way down, whereas active and granted drop to near zero. The consumed memory in the vSphere Web Client is reading 1,542,468 for the latest reading.

There is a pretty easy fix, update your default policy to use Overcommit CPU and use true memory demand; which was the default policy up until vRealize Operations 6.1.

After you have made the change and return to the host memory workload, it will look like it did in prior version.

I have worked with our R&D team on this issue, they are currently researching it further and we should have an update in an upcoming release.

*Note: If you performed an upgrade, it keeps the default policy for Overcommit CPU and use true memory demand.

vRealize Operations 6.1 Cluster Capacity Distribution Dashboard

2015-09-21T18:57:00.002-04:00

New in vRealize Operations 6.1 is the Workload Placement Engine (WPE); which provides several innovations including initial workload placement through the vRealize Operations REST APIs, workload rebalance, and intelligent operation moves. In a future post, I will write about these capabilities more extensively, but today I wanted to focus on creating a visual dashboard that provides an overview of the capacity distribution.

To start, we are going to launch vRealize Operations and click on the Content icon in the Navigation panel. With Dashboards highlight, we are going to click on the green + icon to create a new dashboard.

For our new dashboard, we are going to provide our dashboard a name, in my example I am going to call the dashboard Cluster Capacity Distribution and we are going to select No for Is default. You will notice in vRealize Operations 6.1, gone are the options for Column count and Layout, in this release layouts are much more dynamic, which we will see shortly.

On the Widget List, we are going to drag over the Object List and the Capacity Utilization widgets. After you have dragged these widgets to the dashboard; you can move the widgets around the dashboard, expand and contract the widgets, and move them next to each other to create two and three column dashboards. You have much more flexibility than you did previously for creating your dashboard design.

We are going to edit the Object List by click on the pencil and input the following information:

Widget Title
Refresh Content: Off
Widget Refresh Interval: 300 seconds
Mode: Children
Auto Select First Row: On
Select Object Types and Cluster Compute Resources
Click Additional Column

Clicking the Additional Column slide out tray allows us to select specific metrics we want to monitor for our cluster environment. For my cluster list, under Object Types I highlight Cluster Compute Resources and then select the following metrics on the Metric Picker screen:

Provisioned CPUs
CPU Workload
CPU Contention
Memory Provisioned
Memory Ballon
Memory Swap-in Rate
Total Disk Space Provisioned
Disk Read Latency
Disk Write Latency

The total amount of metrics you can have on a single Object List widget is ten. After you have selected the metrics, click save and select Widget Interactions.

Widget interactions are the configured relationships between widgets in a dashboard where one widget provides information to a receiving widget. When you are using a widget in the dashboard, you select data on one widget to limit the data that appears in another widget, allowing you to focus on a small subset of data. We are going to select the Cluster List we created to feed our Capacity Utilization widget.

To finish we are going to click Save.

Now we can see the Cluster Capacity Distribution Dashboard in the list.

I am going to navigate back to the Home screen and click on my new dashboard, since I don't need some of the default columns that come with the Object List; I am going to drop down the column menu and remove the adapter type, object type, policy, collection state, and collection status.

This provides more real-estate on the Cluster List widget and I can see all the metrics information on a single screen.

The Capacity Utilization widget provides a holistic view of host systems, resource pool, virtual machine, and datastore utilization. This feature helps you understand the systems that are under utilized, optimal, and over utilized. The resources on the left are under utilized, which provides you an opportunity to right-size the infrastructure components to reclaim capacity that isn't being fully used. The resources on the right are under stress, the infrastructure components could use additional resources because of high demand. In the following picture, we want resources as close to the red line as possible to have optimal performance.

If you hover over any of the objects, vRealize Operations will give you information on the virtual machine and the recommended size. We can see in the example below; my Active Directory server is configured with 4 GB of memory but could be configured with 600 MB.

We can use the Rebalance Container and Set CPU Count and Memory for VM under Actions to optimize our environments; and help us get to datacenter nirvana.

ManageIQ Cloud Providers

2015-09-14T18:48:00.000-04:00

For IT leaders and architects, the ability to match business requirements with technology capabilities has become a fundamental skill for executing strategic initiatives. The move from traditional IT to hosted solution provider is a substantial opportunity for any IT department to add value to the business it serves by offering a choice of infrastructure solutions. ManageIQ is a single pane of glass that helps you manage your on-premises and public cloud resources.

In a previous post, we walked through the process of setting up infrastructure providers for vSphere and Red Hat, providing us the capability of managing a multi-hypervisor datacenter. A provider is a hosting instance with software to manage multiple virtual machines. ManageIQ can manage vSphere, Hyper-V, Red Hat, OpenStack, and Amazon EC2 environments. Today, we are going to focus on the process of setting up Amazon EC2 as our cloud provider in ManageIQ, which will give us the ability to build virtual machines on various platforms depending on the use case and business requirements.

To start, we are going to select Clouds and use the Providers taskbar to add a new cloud provider. Click on the Configuration drop-down and select + Add a New Cloud Provider. Additionally, you use the Providers taskbar to initiate the refresh of any existing cloud provider.

For the Basic Information, we are going to give our new provider a name, select the type of Amazon EC2, and choose a region. You also need to provided your Access Key ID and Secret Access Key from Amazon Web Services.

Access keys consist of an access key ID and secret access key, which are used to sign programmatic requests that you make to AWS. If you don't have access keys, you can create them by using the AWS Management Console and selecting Identity & Access Management.

From the navigation menu, click on users and select your user name.

Scroll down the page to Security Access and click the Create Access Key button.

Your keys will look something like this:

Access key ID: AKIAOSFODNN7EXAMPLE
Secret access key: wjalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Click Download Credentials, and store the keys in a secure location.

ManageIQ uses virtual thumbnails to describe providers. Each thumbnail contains four quadrants by default. This allows users to glance at a provider for a quick view of its number of hosts and authentication status.

Top left: Number of hosts
Bottom left: Management system software
Top right: For future use
Bottom right: Authentication status

As we can see from my provider, I have 1 host on Amazon AWS and valid authentication credentials have been added. Clicking on my Amazon EC2 instance, I get more detailed information on the provider including region, type, availability zones, flavors, and current instances.

Clicking the Availability Zones taskbar shows me the the east zones, since I have selected the US East (Northern Virginia) region. You will notice that I have one instance on us-east-1b.

The flavors tab provides me with the instance types, which include C1 through C4 instances. C4 instances are the latest generation instances, featuring the highest performing processors and the lowest price/compute performance in EC2.

There are also D2 dense storage instance available that provide up to 48 TB of HDD-based local storage and high disk throughput. To get more information on the different instance types, you can go to Amazon's EC2 Instance Page.

Selecting the Instances tab, it gives me details on the instances I have running on Amazon EC2. My current instance is i-1602a1c5 in availability zone us-east-1b and it is a T2.micro type.

To create an AWS instance, first we need to go into our AWS Management Console and select one of current running instances to create a template. Drop-down the Actions menu, click Images, and then select Create Images.

Provide the Image Name and Description, then click Create Image.

After the image has been created, it will show up under AMIs on the Images taskbar.

From the ManageIQ console, click on Clouds and Instances. We are going to select Lifecycle and click on Provision Instances.

Our new image will show in the center panel, we are going to highlight the image and click Continue.

We are going to provide the request information; including e-mail, first name and last name. After that is completed select the Catalog tab.

On the Catalog tab, we select the number of instances and give our instance a name. In my example I am calling my new virtual machine devcloud-VM.

Now we select the Environment tab, this is where we select our Availability zone and Virtual Private Cloud, the Cloud Subnet should fill in by default.

On the Instances tab, we are going to select our Instance Type and then click Submit.

After submitted, it will bring you to the Requests page where you can monitor the status of the deployment. After 3 to 5 minutes, the virtual machine with be deployed in AWS and it will show in the ManageIQ console.

Now we have ManageIQ setup to deploy virtual machines in Amazon Web Services, vSphere, and Red Hat depending on the business requirements for our enterprise applications.

Becoming a hosting provider of IT services enables IT operations to become true strategic partners of the business, rather than being simply a cost center of the organization. You become a value-adding department that actively delivers infrastructure solutions that drive business benefits.

vRealize Operations and SCOM

2015-08-24T10:31:00.002-04:00

One of the beautiful aspects of vRealize Operations is its extensibility. With vRealize Operations; you can broaden the benefits of monitoring, alerting and capacity planning to other infrastructure components in your datacenter with management packs found on the VMware Solution Exchange. We are going to take a look at one of these solution packs today, the SCOM solution pack. The management pack for SCOM is an embedded adapter for vRealize Operations Manager. The management pack collects time-series metrics, object availability information, and object relationships from the System Center Operations Manager database.

The management pack for SCOM supports Microsoft SCOM 2007 SP1, Microsoft SCOM 2007 R2, Microsoft SCOM 2012 SP1, and Microsoft SCOM 2012 R2.

To start, we are going to navigate to the Solution Exchange and scroll down to Enterprise Management Packs. Select the Management Pack for SCOM and then click the Try button to download the management pack.

After the download has completed, click on the Administration link, and then click the green + button to add a new management pack.

Browse to the vmware-MPforSCOM-4.1-2919918.pak file and then click Upload.

Accept the EULA agreement and click Next.

Installation details will appear in the windows, after the installation has completed click Finished. To configure the management pack for SCOM, you must create a management pack instance and credentials. The management pack instance defines the management pack type, describes how to connect to the source database, and specifies the source database type. You can create multiple management pack instances, but you cannot assign different queries to each instance.

Under Solutions, we are going to highlight MP for SCOM and click the gear icon to Configure. For the Adapter Settings, we need to include the following information:

Display Name
Database Host
Database Port (default 1433)
Instance Name
Database Source
Database Name
Timezone
Auto Discovery
Credentials

If you enable Auto Discovery, the adapter instance discovers new objects and creates the objects for you. If you disable auto discovery, you must discover objects manually. You might want to disable auto discovery to limit the number of resources that the adapter instance creates. Since this is my lab environment, I am going to set Auto Discovery to True.

After you have given the adapter time to collect information, you will be able to see SCOM related information in the dashboards that get installed with the management pack. By default, there are 14 SCOM dashboards.

By clicking on Dashboard List and then SCOM, we get the entire list of SCOM dashboards available to us. I am going to navigate to the SCOM MS Exchange dashboard.

The SCOM MS Exchange dashboard provides us with heatmaps and sparkline charts for the Exchange CAS and Mailbox servers. On the third column of the dashboard, we get the relationship widget, which provides us an end-to-end view of the environment.

If we click on Environment and then All Objects, we see all the object information that can be collected by the SCOM management pack into vRealize Operations.

Let's dive a bit deeper into my OperationsManager SQL database. I am going to select Environment, MS SCOM Resources, SQL Servers, and then highlight my OperationsManager database. Clicking on the Troubleshooting tab, I notice that I have a few symptoms, two of which are critical. If I click on Timeline, I can get an understanding of when the events happened.

The management pack for SCOM alert definitions are a combination of symptoms and recommendations that identify problem areas in your environment and generate alerts on which you can act for those areas. The SCOM alerts are based on state metrics. This metrics corresponds to monitoring state in MS SCOM, and reflects the objects overall state on the SCOM server. An alert is generated when one of the main objects monitored has a Warning or an Error state.

Clicking All Metrics gives me the resource map on the top panel and the ability to drill down into specific metrics on the bottom panel. I have selected the DB Avg. Disk ms/Read, which shows me the graph on the right hand side.

I can zoom into the peak value of 20.2 to get a better understanding of the timeframe by dragging my cursor in the graph. You will notice, the spike started around 5:30 AM and lasted until just after 5:40 AM. It provides a great visual perspective of potential performance related issues that you can provide your database administrators and application developers.

Furthermore, I can select the Environment tab and look at an overview of the infrastructure resources that constitute the database to try to find the root of the problem.

If I select Map, it gives me a detail list of the database instances running on the SQL server. If you notice closely, this is a SQL server with 52 databases.

The vRealize Operations management pack for SCOM provides some great insight into your SQL, Exchange, and Windows environment; the capability to gain an accurate and intuitive understanding of your infrastructure resources.

Keep in mind, because this is in-guest level monitoring, it does require Enterprise licensing.

ManageIQ Dashboards

2015-08-16T09:14:00.002-04:00

When you first log into ManageIQ, you are presented with the Cloud Intelligence Dashboard. That has a nice ring to it "Cloud Intelligence", which in reality makes sense because it gives you and overview of your internal and external datacenter resources. I can create dashboard widgets for my VMware resources, Red Hat resources, and Amazon resources. Additionally, ManageIQ can manage Hyper-V resources, I just don't have it running in my lab. The Cloud Intelligence Dashboard is customizable, which is what I am going to walk through today.

After you first log into ManageIQ, you are going to get a dashboard similar to the example above. It provides you information on guest OS, discovered hosts, and virtual infrastructure platforms, just to name a few. But, I wanted to customize my dashboards layout and add a few additional widgets.

To move a widget, you click on the widget and relocate it to where you want on your dashboard.

The buttons at the top of a widget provide you with the ability to zoom, open a full report, minimize the widget, or remove the widget from your dashboard.

If we open up the full report on the Vendor and Guest OS Chart, it gives us a greater level of detail for the infrastructure components in my home lab.

To create custom widgets for our dashboard, we need to go into Reports, I am going to walk you through a couple of widgets I created for my dashboard. First we are going to look at VM by Department, made with the tags we created in the last post.

When creating a new widget, we are going to click on Configuration and then Add a new Widget.

On the create widget screen, we are going to enter the following information:

Title: VM by Department
Description: VM Resources Utilization
Active: Checked
Filter: Performance by Asset Type - Virtual Machine - All Departments with Performance
Column 1: Department
Column 2: VM
Column 3: vCPUs
Column 4 VM:Memory
Run: Hourly every Hour
Visibility: To All Users

After you have saved the widget, you can generate the information by clicking on Configuration and then Generate Widget content now.

After I add the widget to the dashboard, we can see the virtual machine information in my lab environment categorized by department.

Another widget I found interesting was RSS feeds, I created a custom RSS feed widget that displayed VMware News. Below is the field information:

Title: VMware News
Description: VMware News
Active: Checked
Type: External
External RSS Feed: Enter URL Manually - http://feeds.feedburner.com/vmwareblogsfeed
Run: Hourly every Hour
Visibility: To All Users

After I generate the content and add the widget to my dashboard, I get all the latest news from VMware and I click on the news feed to bring up the full article.

For my custom dashboard I have the following widgets, as shown below.

Vendor and Guest OS Chart
Guest OS Information
Host Summary
Top CPU Consumers
Top Memory Consumers
Top Storage Consumers
Weekly Utilization Report
VM by Department
VMware News

As mentioned at the start of the article, the Cloud Intelligence Dashboard provides a view into all three platforms in my lab environment. You will recognize, I have virtual machines in Amazon, Red Hat, and VMware. While it is limited in the amount of widgets available out-of-the-box, the widgets that are available provide valuable insight into your infrastructure resources.

For several small to medium size organizations, they don't have the staff, budget, or infrastructure scale to justify a cloud management suite. An open source solution like ManageIQ provides a valuable alternative for companies in the SMB space.

Infrastructure Measurements

2015-08-08T21:12:00.000-04:00

IT infrastructure measurements, if performed well, can provide clarity to IT and business leadership of the infrastructure resources supporting business applications. This type of transparency helps IT operations build credibility with business unit leaders; unfortunately in most organizations IT operations does not have meaningful measurements and they can't correlate the growing infrastructure expense to business outcomes.

In my previous role as an IT manager at a Fortune 250 company, I was responsible for helping to determine the budgeting of infrastructure resources that were going to be required the following year. In most instances, it was a guess, no better than blind folding me and asking me to hit the bull's eye on a dartboard. I decided to implement a standard, repeatable report that would show me growth trends of my virtual machines, host servers, and FTE to virtual machine ratio. After a few years, I was able to use this data to predict next years growth rate for budgeting and leverage the report to show staffing deficiencies.

Value from IT infrastructure measurements is gained by analyzing the data, reporting it to senior leadership, and taking actions to improve day-to-day operations.

When I joined VMware as a Technical Account Manager, I took these same dashboards and applied it to monthly IT infrastructure measurement reports that I produced for my customer. VMware TAMs have a tool called the VMware TAM Data Manager (TDM) that they use to gather information about a customers environment, and while not all customers allow TAMs to run the appliance, providing the data affords the TAM the opportunity provide a wealth of information.

I am going to walk you through a sample IT infrastructure measurements report that I created on a monthly basis.

To start, I provide the growth rate for virtual machines. We can see that in December 2008 I had 400 virtual machines and over the past seven years I have increased to 4,173 virtual machines. A substantial amount of growth, this chart visually demonstrates to IT and business leadership the number of virtual machines in the environment supporting business applications.

Our next chart shows monthly growth rates, we can see that in April we had a significant amount of virtual machines added to the environment to support a business project and for application migrations due to Windows 2003 Server end of life.

Next, we can look at the quarterly virtual machine growth trend.

IT staffing ratios should be on every IT leaders mind. It is critical making certain you have the right amount of staff to maintain IT infrastructure, while leaving room for strategic projects. In my opinion, one item that always gets lost when managing IT operations, is the fact that even though virtual machines are easier and faster to deploy, Windows and Linux server guests still require the same amount of support.

Windows still needs to be patched, applications deployed, issues resolved, and proactive monitoring completed. As we notice below, we have had a steady FTE ratio of 8 infrastructure engineers supporting the environment, and over that period of time they went from supporting 51 virtual machines each to 522 virtual machines each.

When you have this amount of growth and you don't adequately staff your operations team, you are in constant fire fighting mode, your team never has time to do preventive maintenance and monitoring. This becomes apparent to business users and customers as application performance is routinely slow or down; which makes them distrust ITs ability to meet business needs, never mind innovate.

Of course there are steps you can take to help increase FTE to VM ratio, that would include automation, proactive monitoring, and purchasing converged and hyper-converged platforms to simplify deployments.

I would sum up this information on a single slide.

With all that information in hand, I could then look at yearly growth trend percentages and predict next years growth rate and FTE ratio. My year-over-year growth rate is 19%, the projected server amount for Q1 2016 at the current trend is 4,693, with an FTE ratio of 587 virtual machines being supported per IT infrastructure engineer.

This type of information is invaluable to IT leadership. If you are with the VMware TAM Program, this type of reporting can be provided if you allow the TAM to run the VMware TAM Data Manager (TDM).

I want wrap up the post with a few more items I created with the infrastructure measurements data for the Deputy CIO, in the slide below I combined all the elements above into a single slide showing the 47% virtual machine growth rate from 2013 to 2014, demonstrating the various growth rate periods, and showing that the FTE ratio over that period of time stayed flat. It was a very powerful slide that was presented to the senior business leaders.

I also created the following two slides demonstrating the savings in licensing and power with virtualization.

The value of measurements is derived from routinely benchmarking growth, providing visibility of this information to leadership, and helping to be more prescriptive about infrastructure budgeting.

ManageIQ Categories and Tags

2015-08-07T14:36:00.000-04:00

In my previous post, I demonstrated how to use ManageIQ for cost transparency by using the chargeback feature. In that post, we used tags that were set on our virtual machines to define them by their service class and classify them as internal or external resources. In this follow-up post, we are going to step through the process of managing and creating ManageIQ categories and tags.

To start, we are going to open up ManageIQ and select the Configure tab. Click on the Configuration menu, highlight your Region under Settings, and then select your company Categories.

Categories are intended to group tags together that have certain characteristics. For instance, one category that is provided by default is Service Level. In that category, I have tags that include Bronze, Silver, and Gold.

In the example below, to create a new category, I am going to click on the + sign on the first row of the table.

On the next screen, I am going to give my category a Name, which is the short name in the VMDB (Note: Once this name has been established, it cannot be modified). Additionally, I am going to create a Display Name and Description. For our example, I am creating an entry for Department Vice President; this will allow me to tag the virtual machines in my environment with the Department VP for chargeback reporting.

Show in Console indicates the category is ready for use by objects in ManageIQ
Single Value specifies only one tag can be selected in the specific category (For instance, I can only have one service level for my virtual machines)
Capture C & U Data by Tag is used to group capacity and utilization data by the tag category

After you click Add, you will get a notification banner displaying the Category "Department Vice President" was added and you will be able to see it in the Category table.

Next we are going to define the tags in our new category, click on the company Tags link. Using the Category drop-down menu, I am going to select my new category of Department Vice President and then click the + sign on the top row so I can add new tags.

I have added three Department Vice Presidents tags to my category, I can edit the tags by clicking on the row, delete the tag by clicking the trash can, or add additional tags by clicking the + sign.

Now I am going to add this new tag to a virtual machine, I am going to open up Infrastructure, then Virtual Machines, and select my ManageIQ virtual machine.

Click on the Policy drop-down menu and select Edit Tags.

On the Select a customer tag to assign field, I am going to select Department Vice President and assign the value of Jason Gaudreau.

I am now tagged on the ManageIQ virtual machine as the Department Vice President. I can use this tag as a filter on my chargeback report to find out how many virtual machines are being used by my department, the utilization of the resources, and the weekly or monthly cost.

That is all there is to it! As my 8 year old would say, "Easy peasy lemon squeezy".

ManageIQ Chargeback

2015-07-30T21:37:00.002-04:00

Built into ManageIQ is a chargeback solution; and while it isn't a feature rich business management solution like VMware ITBM or Apptio, for an organization that is looking to provide some basic financial transparency of their infrastructure utilization, it isn't a bad tool.

For me, tiering your infrastructure service capabilities and providing cost transparency for technology capabilities is the foundation to IT hosting. Sure, automation is great for reducing delivery times and providing consistency, which reduces OPEX; but I think marrying the infrastructure technology capability to the business requirement with policy based management provides much more tangible savings. It ensures that mission critical applications have the infrastructure resources for performance, stability, and availability to meet LOB expectations, while less critical applications maximize infrastructure savings with lower tier components.

Over my career, a significant amount of business I have worked with select a one size fits all approach to technology solution to meet all their business needs.

After deploying and configuring ManageIQ, you will want to setup the cost rates that you want to use in chargeback and then create a custom report to run on a reoccurring basis.

To start, from the Cloud Intelligence dashboard we are going to click on the Chargeback tab.

Because we haven't created our custom report, the report option is going to be blank. We are going to click on the Rates tree option. You will notice, under the Compute branch, I have created several different chargeback models, which include Bronze, Silver, and Gold.

The compute rate models are driven by Tags, which I created previously and assigned to my virtual machines. For instance, below are the tag assignments for my ManageIQ virtual machine. I will cover how to create tags and assign them to virtual machines in a future post, but you will notice the Service Level for the virtual machine is Silver.

If I open up my Silver Chargeback profile, there are a few different methods for charging for infrastructure resources. You can charge compute by CPU allocation, used CPU in MHz, or by a fixed daily or monthly charge. Memory is charged by allocated memory or used memory in MB. And last, disk I/O and network I/O are charged by KBps per hour.

So ManageIQ does provide some variety cost allocation methods, but it doesn't provide deep capabilities for unit costing in the tool. That being said, not every company needs to have thorough unit costing and just needs the capabilities to provide insight into the infrastructure costs.

In the model below, I did costs by Used CPU in MHz, Used Disk I/O in KBps, Used Memory in MB, and Used Network I/O in KBps. If you haven't created any cost models before, I have a blog post that goes into some basic ways to create a tiered cost model to support your business called The Cost of Business that I wrote last year.

Here are the charges I used for my Silver tier:

After we have determined our rates, we are going to assign our rates to the tag category. Under assign to field, I select Tagged VMs and Instances and for the tag category I select Service Level; I then match up my service level tags to the compute cost rate profiles.

To finish our cost allocation method, we are going to configure our storage rates and click Save.

To make this all work, we need to create a custom report. You are going to click on the Reports tab and go down to the bottom of the list to choose Custom, then click Configuration and Add a new Report.

On the Columns tab, I am going to enter the Menu Name, Title and then select Base the report on Chargebacks.

Additionally, I am going to select the following fields for my report:

CPU Used
CPU Used Cost
Memory Used
Memory Used Cost
Disk I/O Used
Disk I/O Used Cost
Storage Used
Storage Used Cost
Total Cost

On the formatting tab, I leave all the default settings.

On the Filter, I modify the Chargeback Filters so that I am only showing my internal private cloud resources, and it doesn't include my Amazon EC2 virtual machines.

Show Costs by: Virtlab Tag
Tag Category: Network Location
Tag: Internal
Group by: Date

I preview my report and then click Save.

To make certain this runs on a weekly basis, on the left hand panel I click on Schedules and create a new schedule. I call the report Virtlab Chargeback Report with the description of Virtual Machine Chargeback Report.

Under report selection, I am going to use a Filter that includes the report I created earlier called Virtlab Chargeback Report and set it to run weekly.

After your report has run, it will show you the chargeback costs for all the virtual machines in you environment.

Here is a closer look at the report.

ManageIQ chargeback doesn't provide comprehensive unit cost capabilities, but it is a good starting point of showing cost transparency to the LOB.

vRealize Operations - Stress, Capacity Remaining, and Time Remaining

2015-07-28T13:31:00.002-04:00

Stress measures the amount of workload for a specific object over a period of time. The objects that can be measured include CPU, memory, network I/O, datastore I/O, and vSphere configuration limits. Stress only measures demand, whereas with Capacity Remaining can measure demand and allocation. The default period of time is 30 days, which can be configured in vRealize Operations 6 policy wizard.

Stress is selected by default for Peak Consideration in the Capacity and Time Remaining elements; vRealize Operations uses the stress elements to account for peaks in capacity usage.

The default setting for the stress line is 70%, vRealize Operations Manager calculates the amount of object demand that goes over the stress line during the 30 day time sampling.

Below is the calculation for the measured amount of stress volume:

Total Stress Zone Volume

720 hours (30 days) * (100% total capacity - 70% stress line) = 21,600 Stress Zone volume for the past 30 days

Measure Object Stress Volume

Amount of hours object is over the stress line * (100% total capacity - 70% stress line) = Measured Object Stress Volume for the past 30 days

Measured Object Stress Volume / Total Stress Zone Volume = %Stress

In the example below, we can see that my Stress percentage is 20.63% on labesx04.home.virtlab.com.

If we use the formula, we get the following results:

Total Stress Zone Volume

720 hours (30 days) * (100% total capacity - 70% stress line) = 21,600 Stress Zone volume for the past 30 days

Measure Object Stress Volume

148.436 hours * (100% total capacity - 70% stress line) = 4,456.08 Measured Object Stress Volume for the past 30 days

Measured Object Stress Volume 4,456.08 / Total Stress Zone Volume 21,600 = Stress Percentage of 20.63%

Roughly speaking, over the past 30 days my host has seen stress for 6 days. It will use the percentage of stress with the peak value to understand the amount of stress over time.

Now we turn our attention to Capacity Remaining, we notice that due to capacity restraints with memory I am able to add 0 virtual machines to this host. The Peak Value was 13.08 GB or 126.04% of the Usable Capacity; however the Stress Free Value is 105.86% because it has taken the percentage of stress into consideration. Keep in mind this is my home lab environment; hopefully you are seeing a more modest trend line in your corporate environment.

Usable Capacity * Stress Free percentage = Stress Free resources require for an object, the difference between the Stress Free Value and the Useable Capacity is what is Remaining on the host. Obviously, because of the memory demand, I have 0% remaining. My average demand is 5.22 GB, which is only half of the total capacity, but vRealize Operations is going to take into account the Peak Value and percentage of Stress over the sampling period because it could have application workloads that have high workload periods, such as monthly batch processing or seasonal database transactions.

When we move to Time Remaining, we can see that there are 0 days left due to memory demand and 67 days left due to disk space. Again, it is using the Stress Free Value of 10.99 GB against the Usable Capacity of 10.38 GB.

When using the stress policy in conjunction with Capacity Remaining and Time Remaining, you can identify hosts and virtual machines that require additional resources before you impact business applications. In addition, you could use the VM:Migrate action in vRealize Operations to move virtual machines to an alternate host with more capacity to balance out the environment. This helps you avoid performance problems and helps you get some sleep at night!

ManageIQ with vSphere: Providers

2015-07-09T18:30:00.000-04:00

In a previous post, I went through the basic configuration of setting up a ManageIQ deployment. Since that time, I have configured my ManageIQ lab environment to administer my vSphere environment, Amazon EC2 account, and a RHEV environment. Today, I am going to walk you through the process of adding Providers to ManageIQ and working your infrastructure resources.

To start, I am going to add my VMware vCenter server, which will inventory my clusters, hosts, virtual machines, resource pools and datastores. I add in all the basic information including:

Name
Type
Host Name or IP address
Zone

I then enter the credentials to connect to the vCenter server and click Validate. After it has been validated, click the Add button.

Next we are going to add our Red Hat Enterprise Virtualization Manager. Like before, we are going to add in the basic information and the credentials to authenticate. After our account has been validated, we are going to click the Add button.

Now I have both infrastructure providers showing under the Providers tab.

If I click on the Hosts tab, it shows my ESXi and RHEL host. I like to change the view to list mode so I get more detail, this can be done by clicking the button in the upper right had corner.

In order for us to gather information on the host, we want to make sure that we have the proper credentials in place. We are going to check the box next to our host, in my example below labesx02.home.virtlab.com, and then click Configuration and Edit Selected Hosts.

In the Credentials section, we are going to add the user account with privileged access to our host. I setup credentials for Default and Remote Login.

I do the same for my RHEL host; I enter the Credentials of my user account with privileged access.

If we double click on a host, it brings up a comprehensive list of all the information on the host. Below, we can see a summary of my ESXi host labesx02.home.virtlab.com. It provides us information on IP address, operating system, power state, devices, services, and advanced settings, to name a few.

For more detailed information, you can click any of the items. I am going to click on Services to get details on the 14 services that are running on my host.

After clicking on Services, it brings me to a complete list of the services running on my ESXi host. It provides the service name, display name, and a status on the service state.

If you click the Properties drop-down on the left hand Navigation panel, you can then select Capacity & Utilization which gives performance metrics. You can do hourly intervals for the past 12 hours and daily intervals for the past four weeks. Although it doesn't provide comprehensive information like vRealize Operations Manager, this gives a good general overview of the systems health.

The different drop-downs on the Navigation panel provide the following options:

Properties

Summary
Devices
Network
Storage Adapters
OS Information
VMM Information
Capacity & Utilization
Timelines

Relationships

Cloud Infrastructure Provider
Cluster
Datastores
VMs
Templates
Drift History

Security

Users
Groups
Patches
Firewall Rules

Configuration

Packages
Services
Files
Advanced Settings

If we select the Virtual Machines tab, it gives us the list of virtual machines and templates in the center panel. On the Navigation panel to the left, you will notice that it shows both providers in the tree with tree branches that include datacenters, resource pools, virtual machines, and templates.

Similar to hosts, if we want more detailed information on a virtual machine we can double click on the VM. In the example below, I have clicked on my Management Server, which is a Microsoft Windows Server 2012 server.

If we scroll all the way down the window, it gives us performance information for the virtual machine over the past 30 days.

If you want to find out if the virtual machine is a candidate for right-sizing, click on Configuration and then Right-Size Recommendations.

ManageIQ provides you a general report showing the capacity ranges for CPU and memory; and the capacity that can be save by right-sizing your virtual machine. In my case, I can save 0 on processors; however I can save in-between 284 MB and 3764 MB of memory depending on the recommendation.

In my next ManageIQ post, we will look at setting up tags for virtual machines, setting up automation, and deploying virtual machines from ManageIQ.

VMworld 2015 Session Recommendations

2015-07-06T17:02:00.001-04:00

The last couple of years, I carved away the VMworld content catalog to 50 sessions I recommend to my customers. This year, there are 658 session available at VMworld 2015, which is an overwhelming amount of sessions! My recommendations are from past experience of seeing the presenters at VMworld, VMUGs, and interacting with them at work. Moreover, I try to make sure I have all our major product lines included, and a mix of technical and business tracks.

Going along with the VMworld 2015 theme. You can be ready. Ready for VMworld 2015 and the brave new model of IT.

Ready for Any.

50 Recommended Sessions:

MGT6525 - Automation for Service/Catalog Item Provisioning and Management with Naomi Sullivan

Presenter: Naomi Sullivan

Group Discussions are a good way to join together with peers, guided by a VMware expert, and discuss a VMware key topic as selected by the group. Come to this session prepared to dive-in, engage, and share best practices

EUC5052 - Beyond the Marketing: Horizon 6 Technical Deep Dive

Presenters: Ray Heffer and Jim Yanik

Horizon 6 continues to evolve with new features and functions that increase the breadth of use cases and enhance the end user experience. This session will provide a technical look at each feature and how it works. We will provide examples of how these features are used in real-world use cases. We will cover topics including vGPU, Virtual SAN, Virtual Volumes and end use experience enhancements driven by client and agent functionality. This session will focus on the new features and functionalities of the latest versions of Horizon 6.

EUC4827 - VDI Sizing Deep Dive - The Horizon Sizing Tool

Presenter: Ray Heffer

Direct from the EUC Technical Marketing and Enablement team, we reveal the Horizon Sizing Tool. We will show you best practices and sizing techniques using our latest sizing tool aimed at consultants and architects. Learn how to avoid common pitfalls when sizing for large scale deployments. We will show you sizing methods for linked clones, full clones, RDS sessions, including Horizon server components. We will answer the most commonly asked questions around sizing for Horizon, including when or when not to use TPS (Transparent Page Sharing), SE (Space Efficient) Sparse disks, Virtual SAN and all flash arrays. To finish, we will show you a demo of the Horizon Sizing Tool and walk you through a large scale Horizon sizing exercise.

INF5211 - Automating Everything VMware with PowerCLI – Deep Dive

Presenters: Alan Renouf and Luc Dekens

PowerCLI is the number 1 automation product from VMware, in previous years you have seen how to get the most out of your infrastructure using PowerCLI. This year we will take it further than ever including new product automation never seen before, automating your public cloud instances and showing migration and reporting for your cloud workloads. This session will demonstrate how to bring DevOps working practices into your vSphere automation with PowerCLI and Desired State Configuration (DSC).

INF4448 - Design Advice for Small and Midsize Business

Presenters: Brian Atkinson, Sean Crookston, Jason Langer, Harley Stagner, and Jason Gaudreau

Join several industry experts to talk about the considerations that come into play for small and midsize organizations. IT administrators at small and midsize businesses wear many hats, which requires a breadth and depth of knowledge of not just the technology, but the business as well. In this interactive design workshop, we'll go through some of the most common challenges we see customers facing when moving towards the next generation of virtualization. Considerations will be made including management, backups, storage, & networking.

INF4529 - VMware Certificate Management for Mere Mortals

Presenters: Adam Eckerle and Ryan Johnson

SSL certificates have become essential in securing access to our data centers and infrastructure. As vSphere has evolved so to have the processes, application, and management of SSL certificates. Gone are the days of simply relying on self-signed certificates and never having to validate, renew or really manage those certificates. In this session we will discuss and demonstrate the new VMCA to show how to deploy & manage SSL certificates in vSphere 6. We'll also discuss upgrade considerations and how to evaluate the different deployment models for the VMCA in order to decide which is best for the user's organization.

MGT5973 - Starbucks Case Study: Self Service Provisioning of VMs with NSX Micro-Segmentation

Presenters: Justin Jones and Jason Horn

Over a 9 month period, Starbucks and VMware collaborated on the design and deployment of a self-service provisioning platform. The platform offers user-controlled application micro-segmentation and self-service provisioned virtual machines, using a wide range of technology and software including vRealize Automation, vRealize Orchestrator, and VMware NSX. We will recap the capabilities of the system and summarize the realized benefits as well as challenges and lessons learned.

INF4790 - A ROBO Technical Deep Dive, Understanding Remote and Branch Office (ROBO) Best Practices

Presenters: Brian Graf and Rick Terlep

In this session we will explore best practices of implementing vSphere in Remote Office / Branch Office deployments. We will perform a technical deep dive into subjects such as planning and architecting the initial deployment of hosts and virtual machines, discussing what we see our customers doing as well as design decisions that should be considered carefully. We will jump into ESX and VM lifecycle management for remote deployments and discuss best practices for upgrading and patching these environments as well as ways to maintain configuration consistency across sites and avoid forms of configuration drift. Join us for a great informative and technical session.

HBC6562-GD - vCloud Air with Massimo re Ferre and David Hill

Presenters: Massimo Re Ferre’ and David Hill

HBC4849 - vCloud Air 2015 - Getting Started with Hybrid Cloud

Presenters: Jenny Fong and Greg Herzog

vCloud Air is VMware’s public cloud platform, designed to be hybrid-first and compatible with your existing vSphere workloads. This year has seen many exciting developments including OnDemand, Disaster Recovery and Advanced Networking. In this session we will cover the basics of vCloud Air, what it is, what's new, and the flexible options for purchasing it. We'll also provide a live demo of the simple steps to get you started on your hybrid cloud journey.

MGT5318 - Becoming a vRealize Automagician: Why Automation isn’t Automatic

Presenters: Steve Kaplan, Jad El-Zein, Alan Keeler, and Trent TeSelle

As any IT professional knows, being able to provision systems in a consistent manner, reliably, is the holy grail. Most organizations have tasks they are performing during various stages of provisioning, but these things are, in many instances, manual. While that can be a great starting point, this approach does not scale and routinely falls victim to things like "Oh, I forgot about that” or "I didn't know about that!". The longer the list, the more likely there is for discrepencies between systems being provisioned, regardless of if it's the same person or multiple people performing the same steps. What if there was a way to reduce the possibilities for human error? To do that, you have to become an “Automagician". For vRealize Automation (vRA) managed environments, becoming an Automagician means leveraging vRealize Orchestrator (vRO) to extend native automation. Applying custom properties effectively in vRA to invoke workflows included in the vRA plugin for vRO, the Automagician gains the ability to incorporate automation and transcend the need for manual processes while removing the error-prone human factor in much of the decision process. This session will delve into the symbiotic relationship between vRA and vRO, and how custom property attributes in vRA can be used to power workflows within vRO for enterprise-scale automation.

INF5116 - A Practical Introduction to DevOps for VMware Administrators

Presenters: Josh Atwell, Trevor Roberts Jr, Yvo van Doorn, Egle Sigler, and Chris Sexsmith

You’ve heard of DevOps, but you may not be 100% clear on what benefits it can bring to your IT organization. Join this session to see, and possibly participate in, live demonstrations of popular DevOps tools like Puppt, Chef, Ansible, PowerShell DSC, Razor, Jenkins, Docker, and more. DevOps experts will be on-hand to present information on DevOps tools and practices and to answer your questions about how these solutions integrate with VMware technologies. In this session, you will also learn how to keep your developers happy whilst maintaining control of the infrastructure you know and love.

SDDC5094 - A Technical Deep Dive into VMware Integrated OpenStack

Presenter: Trevor Roberts Jr

You may have tried deploying OpenStack either from Operating System package managers or from OpenStack distribution vendors. You heard that VMware has a formal offering in this space, and you want to find out more. This session will begin with an architecture review of the latest OpenStack release and then progress into a deep-dive of VMware Integrated OpenStack internals. We will round out our discussion by covering how VMware’s solution simplifies “Day 2” OpenStack operations.

STO6407-BC - vSAN Workshop

Presenters: Noel Nguyen, Bo Bolander, Mostafa Khalil, and Greg Mulholland

Software-Defined Storage represents the critical next phase to realize the Software-Defined Data Center vision and journey. VMware Virtual SAN is VMware's Software-Defined Storage approach and solution to managing storage for virtual environments. This Technical Sales Workshop will provide training on how to use Virtual SAN (VSAN) to address the customer's storage pain points around management and provisioning, performance, and costs. Including, i.e., what the product does, benefits, use cases, when and where to position, objection handling. The Workshop will also feature How to Sell VMware Virtual SAN from a Partner perspective, technical deep dive, sizing and design, and troubleshooting. PLEASE NOTE: If you are staying in San Francisco, Shuttle service will be provided to and from Palo Alto for this workshop.

INF5177 - VMworld 2015 - vSphere Security - Fact .vs. Fiction

Presenters: Mike Foley and Yuecel Karabulut

VMware vSphere has been developed from the ground up to run virtual machines in a secure manner. vSphere addresses the security concerns of the most demanding datacenter environments for enterprises and government organizations. Get a better understanding of exactly how!

SDDC6121 - VMware and the 'Internet-of-Things

Presenters: Greg Bollella, Matthias Schorer, and Chris Wolf

Join us for a session on VMware in the ‘Internet-of-Things’ (IoT). We’ll share insights and feedback from some of our customers, discuss internal IoT projects and highlight innovative solutions for building elastic IoT infrastructure; for connecting, managing and securing devices; and for using hardware virtualization for application execution in IoT gateways. We’ll also discuss why we think IT is the right internal organization to own the management, provisioning, configuration, and maintenance of IoT gateways as well as operating tools that perform behavioral analysis on telemetry streams for real-time alerting of anomalous patterns in the values of the metrics. (Hint: Like most things IoT, a big part of the answer is about scale, scope, and existing expertise.) Attend this session and learn: How application of AirWatch, vRealize Operations Manager, LogInsight, and other datacenter management tools, appropriately tweaked, gives you a jump start on the management, application provisioning, and configuration control for IoT gateways. More about our special project (Helix), a vRealize Operations Manager adapter specifically tailored for IoT and an agent for the Intel IoT Solutions Gateway as a telemetry acquisition, behavior analysis, and real-time alerting IoT solution. Why many technologists think an application execution model based on hardware virtualization for IoT gateways makes a lot of sense.VMware can excel in the Internet-of-Things (IoT) Agenda: Customer use cases and application of traditional datacenter management tools to IoT. A deep dive into project Helix.Body: Application of AirWatch and other datacenter management tools, appropriately tweaked, can be used for managing IoT gateways and applications. Our special project (Helix), a vRealize Operations Manager adapter specifically tailored for IoT and an agent for the Intel IoT Solutions Gateway as a telemetry acquisition, behavior analysis is real-time alerting IoT solution. An application execution model based on hardware virtualization for IoT gateways solves some very difficult problems in IoT. Summary: Application of traditional datacenter management tools to IoT Project Helix Hardware virtualization for IoT gateways Conclusion: VMware and IoT, a great match.

NET4976 - vSphere Distributed Switch 6.0 – Technical Deep Dive

Presenters: Jason Nash and Chris Wahl

VMware’s latest 6.0 version of the vSphere Distributed Switch (VDS) provides unparalleled networking features, control mechanisms, and new ways to construct a LAN topology. In this year’s version of one of the highest rated VMworld sessions, you’ll get front row seats to a deep dive on the VDS in a variety of real world use cases. Gain new knowledge for building multi-tenant capable configurations with NIOC version 3 and CoS marking, learn how to protect critical vSphere management workloads consuming the VDS, and follow along as we review the steps necessary to leverage the TCP/IP stack for routed vMotion.

SDDC4595 - Ask the Expert Industry Titans

Presenters: Kit Colbert, Rick Scherer, Nicholas Weaver, Vaughn Stewart, and Chad Sakac

Back for it's 8th year at VMworld, Ask the Experts is back with an awesome panel of the industries top experts. In this session there are no powerpoints, no sales pitches and no rules! Experts in the industry are here to answer your questions while having some fun in the process. Bring your topic, anything from Software-Defined Data Center, to End-User Computing to Hybrid Cloud... Storage, Networking, Security. No questions are off limits.

NET5422 - Using VMware NSX in Your Open Source Environment

Presenter: Scott Lowe

You're building out your next-generation cloud infrastructure, using open source projects and technologies like OpenStack, Open vSwitch, Open Virtual Network, and containers. Can VMware NSX offer value in such an open source environment? Absolutely! This discussion will look back on the rich history that VMware NSX has in the open source world, reviewing some of the innovation that NSX's development has driven into various open source communities. We'll provide a comprehensive review of the various ways in which VMware NSX can be used and integrated into environments that are heavily based on open source projects and technologies, and discuss the value that VMware NSX brings to such environments. If you are involved in building environments that will heavily leverage open source projects and technologies, this is a session you won't want to miss.

VAPP5483 - Virtualize Active Directory, the Right Way!

Presenters: Deji Akomolafe and Matt Liebowitz

Active Directory Domain Services (ADDS) allows organizations to deploy a scalable and secure directory service for managing users, resources and applications. Virtualization of ADDS has been supported for many years now, however has required careful management to avoid pitfalls around replication, time management, and access. Windows Server 2012 provides greater support for virtualization by including virtualization-safe features and support for rapid domain controller deployment.

SDDC5260 - Active-Active-Active Borderless & Always-On Virtual Data Center

Presenter: Scott Carpenter

Case study of how VMware's Software-Defined Data Center (SDDC) enabled a leading Healthcare organization to meet ever increasing demands for high availability while facing IT budget cuts. Learn how VMware Virtual SAN (VSAN) and NSX enable vSphere automation, such as vMotion & High Availability (HA), across three data centers in this active-active-active design.

STO6137 - Site Recovery Manager and Policy Based DR: Deep dive with Engineering

Presenters: GS Khalsa and Aleksey Pershin

Policy based management greatly simplifies the work of IT Administrators making it easy to ensure that applications and VMs receive the resources, protection and functionality required. Learn about the latest enhancements of Site Recovery Manager in this space, which represent a huge step towards providing policy based DR. In this session we'll dive deep into how this approach works and how to work with them.

STO5333 - Building a Stretched Cluster with Virtual SAN

Presenter: Rawlinson Rivera

In this session we will present how to build a stretched cluster configuration on VMware Virtual SAN, in order to expand your enterprise capabilities. A stretched cluster on Virtual SAN can provide zero recovery point objective and close to zero recovery time objective for applications even in case of site failures and enable planned maintenance, disaster avoidance, and automated recovery.

CNA5379 - Panel: Enterprise architecture for Cloud-Native Applications

Preseners: Martijin Baecke, Joe Baguley, Robbie Jerrom, Aaron Seemer, and Andy Jenkins

There are tectonic shifts happening in the enterprise, particularly in how applications are developed and operated (e.g. DevOps), how they’re architected (e.g. micro services and 12-factor apps), and how they’re deployed (e.g. Docker and containers). We call these applications “cloud-native” as they’re designed for the mobile-cloud era. VMware launched Cloud-Native Apps at the beginning of the year to address the shift that is currently taking place how applications are developed today. This session will provide you with a dive into that world of Cloud-Native Apps. In this panel discussion, lead by VMware CTO for EMEA Joe Baguley, we will discuss all the topics surrounding Cloud-Native Apps and walk you through the vision that VMware has for next generation application development and hosting. The panel will include VMware subject matter experts that will provide insight into what can be achieved with containerization, micro services and DevOps. They will provide you with examples and real-world scenarios from field experience. And of course we encourage everybody to come and join in the conversation. This session is meant to be interactive and open discussion around the topic of Cloud-Native Apps. We welcome everyone’s input and questions to learn and share with one another.

INF6394 - vSphere lifecycle with Brian Graf and Salil Suri

Presenters: Brian Graf and Salil Suri

MGT5151 - vRealize Automation or VMware Integrated Openstack or Both?

Presenters: Ryan Kelly and Long Wang

Which IaaS platform is right for your organization? vRealize Automation or VMware Integrated Openstack or combination of both? The focus of this session will be to familiarize you with vRealize Automation and Vmware Integrated Openstack features and concepts. We will provide customer examples of common use cases as to when you would use one or the other. Most importantly the different consumption models each solution provides. We will help to answers the question "What does my end user what to see? Portal? API?" and "How virtual networking with VMware NSX integrates into both solutions. For those interested in operationalizing your private cloud, you will also learn how vRealize Operations and Loginsight can assist with day 2 operations for both vRealize Automation and VMware integrated Openstack. Experts in both vRealize Automation and VMWare integrated Openstack will be presenting and answering your questions throughout the session.

SDDC6541-SPO - Workload Consolidation- From Silos to the Software Defined Data Center

Presenter: Jeramiah Dooley

Storage has become an anchor on the ship of progress in the modern, virtualized data center. Up until recently it has been slow, unpredictable, inflexible, and requires far too much operational overhead. Showing its age, storage is still sized, deployed, and managed using methods created in the 1980s. Worst of all, storage capacity and performance are often still locked together in fixed ratios, making it inevitable that even the best-run storage environment will be wasteful and inefficient. It doesn’t have to be this way. The technology and know-how exists to have storage that is a predictable, flexible, and easily managed part of an overall virtualized platform, designed to provide the performance and availability that today’s end user workloads demand. Join us as we examine a year-in-the-life of an enterprise data center. We’ll show how migrating workloads onto a single platform can be the crucial missing step as VMware customers transition from legacy architecture into a Software Defined Data Center (SDDC).

STO5207 - Troubleshooting Storage Devices Using vRealize Operations

Presenter: Bill Roth

Storage troubleshooting has always been a big pain-point for vSphere admins. Hear from a customer, SalesForce.com on how they are using vRealize Operations to troubleshoot storage issues and learn about the new native storage tools that help you to isolate problems between VM and LUN.

STO5822 - Putting Virtual Volumes to Work -- Storage Best Practices for vSphere 6 and Beyond

Presenter: Howard Marks

The Virtual Volumes (VVol) feature in VMware vSphere offers tremendous operational benefits for IT departments, including more efficient storage utilization and finer control of hardware resources. To take full advantage of these benefits, virtualization managers will need to adapt their systems and processes to leverage the new management objects, mapping considerations, and performance implications inherent in VVol. In this session, Howard Marks, NetworkComputing Contributing Editor and Chief Scientist of DeepStorage, a leading hands-on testing lab, will provide best practice storage considerations for vSphere He will base this on years of research and testing around VVols and will share findings and storage integration best practices. NexGen Storage will sponsor this vendor-neutral, educational session.

VAPP5719 - Monitoring and Managing Applications with vRealize Operations 6.1 and vSphere6

Presenters: Jeff Godfrey and Ben Todd

This year VMware vSphere 6 combined with vRealize Operations 6.1 (vR Ops 6) adds critical features to increase technical agility in the infrastructure, and reduce Mean time to Repair. With a new Automated remediation action framework in vR Ops, vSphere 6’s ability to vMotion Physical Raw Device mappings (RDMs), and a complete Management Pack Ecosystem for monitoring Infrastructure to applications, administrators have the tools needed to get to maintain 5 9’s uptime, shorten Mean Time to Repair (MTTR), and predict capacity requirements as and when the business requires.. This session will be a deep technical explanation, and live demonstration of these tools. It will give administrators a solid understanding of how they can use these tools to monitor and manage their application clusters, keep applications running during Infrastructure maintenance, and get deep holistic visibility into the entire Application ecosystem, from Storage to Networking

INF4528 - vCenter Server Appliance (VCSA) Best Practices & Tips/Tricks

Presenter: William Lam

In this session, I will go through some of the best practices and considerations when deploying and managing the vCenter Server Appliance (VCSA). I will walk through some of the changes between past releases and the new vSphere 6.0 release. In addition, I will also provide tips and tricks on using the VCSA as well as ways in migrating from a Windows based vCenter Server to the VCSA

INF6396 - Platform security with Mike Foley

Presenter: Mike Foley

MGT6003 - How to Win Executives and Influence Outcomes with vRealize Operations and vRealize Business

Presenters: Steven Jones and Jeff Godfrey

Become a true broker of IT services and help business units gain insight into their IT spend using vRealize Business (vRB) and vRealize Operations Manager (vROps). Help Business Units in the way they work and move IT into the realm of a critical contributor to their future success. Give them the whole picture on how IT saves money, makes money, saves time, and provides peace of mind, while earning respect at the same time. Shadow IT is financially inefficient and VMware is providing the solutions you need for a comprehensive cure. Join Jeff and Steve as they plunge into the story of three powerful insights every business needs that vRealize delivers. Walk out with a vital checklist of concepts to deliver value to those that you serve.

SDDC5273 - Growing from EVO:RAIL to EVO: RACK – Scale with Investment Protection

Presenters: Jason Lochhead and Sujit Pania

What's all the hype about with hyper-converged infrastructure (HCI)? In this session, we'll discuss the different alternatives to building a software-defined data center (SDDC), and how the newest option, HCI, is differentiated in terms of simplicity, faster time to productivity, flexible scalability and lower TCO competitive with public/private cloud offerings. Furthermore, we'll do a deep dive into features and capabilities of the 2 members of VMware's EVO family--EVO:RAIL and EVO:RACK--and explain how they are based on the same foundational pieces, but also very different in terms of architecture, capabilities, use cases and target customer. In particular, we will also discuss how an existing investment in EVO:RAIL can be preserved by migrating to a new EVO:RACK instance.

STO5920 - Considerations for Virtual SAN Running Business Critical Applications

Presenters: Vahid Fereydouny and Christian Rauber

In this session we will be discussing the sizing, best practices, and monitoring for architecting Microsoft Exchange and SQL on Virtual SAN. This will be a deep dive session into Microsoft Exchange and SQL Server on VMware Virtual SAN, touching on SharePoint. Also we will discuss how to utilize Virtual SAN for your Oracle RAC deployments. After this session you will be able to understand how to utilize Virtual SAN as the best hyper-converged infrastructure architecture for Exchange, SQL, SharePoint, and Oracle RAC.

OPT5279 - Chargeback in the Department of Defense

Presenter: Daniel Phillippi

Is chargeback in the DoD impossible? Ever wonder why that is? While organizations like DISA and Cyber Command are looking to leverage a service provider model across the DoD, many other COCOMs are left with the internal struggle of how to handle a chargeback or showback model and often fall back to operating in the dark in regards to the true cost of their production environments. Whats worse is that too often the move to a private cloud is seen as unachievable. The truth is that within the Federal space we have done a great job of building barriers to our own success, but the failure to implement a proper cost model for chargeback within each organization has left us paralyzed to move beyond the basic capabilities that virtualization technology brings. Operating on the assumption of costs based on contracting mechanisms and colors of money has disabled the drive to move further into the future and has cost greatly in terms of efficiency and capability; giving our enemies the chance to eclipse us in the cyber arena. The good news is that we can change the dynamic and move into the datacenter of tomorrow. Join us in exploring the possibilities that can come from implementing a proper chargeback or showback cost model, and how it can be used to advance your IT operations in the DoD.

SDDC5440 - VMware Validated Reference Designs - A Reference Architecture for the SDDC

Presenters: Mike Brown and Karthik Narayan

Deploying ESX, vCenter, vRealize Ops, vRealize Automation, and NSX together in a production deployment takes care, thoughtfulness, and planning. Downstream operational issues such as monitoring, capacity planning, and maintenance must be considered. Backup, disaster recovery, high availability, and other enterprise readiness issues must be considered. And, it's not just about VMware's products, but also about how storage, networking, load-balancers, and other non-VMware services integrate into the total deployment. This session will explain the VMware Validated Reference Design, a new project designed to simplify this process for VMware's customers.

VAPP6431-GD - Oracle Migration from Big Iron to Virtual Machines

Presenters: Sudhir Balasubramanian and Mukund Yadav

Migrating large Oracle RAC database from Physical to Virtual

INF5229 - Docker and Fargo: Exploding the Linux Container Host

Presenters: Benjamin Corrie and George Hicken

OPT5793 - Organizational Change Management and SDDC: Why Getting Your Organization and People Aligned Are the Key Ingredient in Ensuring Maximum Value from You

Presenter: Brad Wolfe

The Software-Defined Datacenter (SDDC) delivers a compelling vision of infrastructure transformation and a modern means to IT service delivery across the Private, Public or Hybrid clouds. Moving your enterprise to a SDDC is a tremendous investment in time and resources. Unfortunately many IT organizations miss the most important factor when making that investment, its people. Organizational Change Management (OCM) is a critical component of moving to SDDC and overall IT Transformation. A sure way to be successful when embracing emerging technologies, such as the SDDC, is to ensure the people who will be responsible for architecting, deploying, and managing the solutions are in sync with a common set of goals. OCM begins with focus on these goals to align an organization’s vision, strategy, objectives and mindset thus ensuring alignment between key IT services and broader business objectives. Attend this session to learn how to address the software-defined people problem and build a modern SDDC architecture team. Also learn how building your architecture team is just one of the many components of OCM that are brought together to align your people and your organization with an SDDC strategy. We will address key SDDC and OCM questions on the minds of business and IT leaders regarding the risks of undertaking this enterprise transformation.

OPT4707 - Integrating vRealize Automation with ITSM and Service Catalog Tools

Presenters: Reginald Lo and John Worthington

Many IT organizations are grappling with the decision on which service catalog tool they should use: the one from vRealize Automation or the one from their ITSM tool, i.e. ServiceNow, Remedy SRM, HP Service Catalog, etc. Based on real world experiences, this session describes different implementation options and their pros and cons and common integration points between vRealize Automation and your ITSM tool.

MGT6523-QT - Selling the Value of IT to Your Business

Presenter: Kathylynn Brunner

MGT6528 - Capacity Planning and Management with Zubair Ansari

Presenter: Zubair Ansari

SEC4488 - WestJet - Fast and Simple Security Using NSX and VCO

Presenter: Richard Sillito

In the fast world of Agile development and DevOps, there are many security roadblocks in the race to production. Businesses can't slow down, so security must speed up. This requires a combination of good architecture and strong technology. WestJet’s unique Security Architecture Made Simple (SAMS) combined with Software Defined Security and Automation technology has transformed security operations.

OPT5972 - 80,000 VM's and Growing! VMware's Internal Cloud Journey Told by the People on the Frontlines

Presenters: Ahmed Al-Buheissi, Brian Smith, and Anees Iqbal

STO4856 - Designing for Mass Failover with Site Recovery Manager: Lessons Learned from a Customer Deployment

Presenter: Christopher Chua

How do you design a system to fail over 1000 Virtual Machines? How different is it from designing to fail over 10 Virtual Machines? VMware Site Recovery Manager allows you to protect up to 5000 VMs using array based replication, but can you really just set it and forget it? Explore some of the ins and outs of using Site Recovery Manager at scale through a review of Chevron’s Global Deployment of SRM. Review lessons learned from the design phase, implementation phase, and operational turnover.

EUC5523 - AirWatch 101: Enterprise Mobility Management Simplified

Presenter: Harit Patel

Discover AirWatch by VMware, the world’s leader in enterprise mobility management. The AirWatch platform includes industry-leading mobile device, email, application, content, and browser management solutions for corporate, BYO and line of business devices. Learn how AirWatch enables end users with a digital workspaces while empowering IT with a platform for business mobility.

EUC5921 - App Volumes Tips, Tricks and Troubleshooting

Presenter: Stephane Asselin

App Volumes is a great solution for Just-In-Time application delivery to your virtual desktops or your remote sessions (RDSH or Citrix). There are a lot of ways to deploy and deploy it right. Come here us show you lesson learned from real-life customer deployments, how we can save you time in your future deployments. We will also go over troubleshooting ideas and what to look for when deploying App Volumes in your enterprise.

EUC6051-SPO - How Florida Atlantic University Made Their High Performance Horizon View Deployment Look Easy

Presenter: James Mauser

When the College of Engineering and Computer Science at Florida Atlantic University (FAU) looked to replace their high performance physical desktops and workstations with VMware Horizon View virtual desktops, they faced a challenge in architecting a affordable solution that could stand up to the task of delivering stable, non-persistent virtual desktops for high performance 3D applications to their concurrent users. Simple, right? Attend this session to learn how FAU’s overcame high latency, unstable virtual desktop challenges and delivered a cost-effective, simple to manage, end-to-end solution that met the demands of their high end VDI deployment running intensive 3D graphics applications.

OPT5814-QT - AGILE for Infrastructure: Utilizing Agile Methods to Drive Iterative Infrastructure Development and IT Service Delivery

Presenter: Josh Miller

Is your infrastructure organization struggling to adapt to meet the ever-changing needs of business stakeholders? Have you been trying to stimulate the cultural change required to initiate the adoption of DevOps or IT as a service (ITaaS)? Are you (still) under the impression that the Agile framework is only used to support application development? The widespread adoption of VMware’s Software Defined Data Center (SDDC) technology stack opens up a whole new world of opportunity for infrastructure organizations. Infrastructure practitioners are no longer bound by the proprietary compute, storage, and network siloes of legacy IT environments and now enjoy an unprecedented level of control, flexibility, and freedom. So now what? In this session, Josh Miller, VMware’s Accelerate Transformation Services Agile Operations offering lead explains how your infrastructure teams can leverage Agile frameworks and scrum practices to: • Maximize your VMware technology investments through increased adoption and quicker development of functionality and capabilities to support business needs • Accelerate IT transformation initiatives by enhancing communication, collaboration, and integration within your organization • Create the cultural change required to speed adoption of DevOps/ITaaS as demanded by the business

vRealize Operations 6 Trending

2015-07-02T11:01:00.000-04:00

This week I was working with a customer; they were interested in looking at trending for host and cluster memory resources. But, when we started looking at the default views it was taking into account two memory spikes that were 1000%. Now, having memory spikes at 1000% isn't good to start with, but for capacity trending purposes it was making it difficult to gain perspective on the overall trend of resources.

To remedy the problem, you can change the roll-up interval to flatten the line and gain a better understanding of the overall trend.

Now, because I am not taxing my lab environment, it isn't going to be as dramatic as it was for this particular customer. For my host labesx04.home.virtlab.com, I am going to select the Details tab, which provides me with a list of Views. I am going to scroll down until I see Host Memory Usage and Demand (%) Trend View. When I click on the name of the View, it is going to display in the panel below.

Next, I am going to click on the clock icon and change my roll up interval to 1 Day.

This flattens out my trend line and provides a more consumable perspective of the overall capacity, especially when you have dramatic spikes.

If I wanted to make a View that defaults to 1 day, I would click on the Clone View icon while having the Host Memory Usage and Demand (%) Trend View selected.

I am going to call my new View 1-Day Interval Host Memory Usage and Demand (%) Trend View. Also, we are going to expand advanced settings and change the roll up interval to 1 Day. You have to do this for both Memory|Usage (%) and Memory|Demand|Workload (%).

When you select the newly created View from the list, it will show the flattened trends lines in the chart.

If we wanted to create a scheduled report based on this view to keep track of capacity trending, we would click on the Reports tab and then click the green + icon.

We are going to enter the name for our report; I used Host Memory Usage and Demand (%) Trend View and noted that it was 1-Day Interval in the description.

Drag the 1-Day Interval Host Memory Usage and Demand (%) Trend View to the right panel, it should show you a preview of the report.

I kept the defaults for Formats and Layout Options then clicked Save. Like the image below, now your report should show in the list of available reports. We are going to click on the Schedules link.

I setup the schedule to run on a weekly basis and include the email distribution list to mail this out to the IT operations management team.

After the report runs, I get a PDF report showing me the memory usage and demand trend for the host.

ManageIQ with vSphere: Deploy and Configure

2015-06-30T12:09:00.000-04:00

ManageIQ is company that Red Hat acquired in 2012. ManageIQ created a cloud management program by the same name, which is available with a Red Hat subscription as CloudForms. ManageIQ provides a single pane of glass for the unified management of infrastructure resources. It can manage vSphere, Hyper-V, RedHat, OpenStack, and Amazon EC2 environments.

ManageIQ was made open source last summer; since it is a free download, I decided to try it out in my lab environment. I have it managing my vSphere lab infrastructure and my Amazon EC2 account.

The deployment couldn't be much easier; you go to http://manageiq.org/download/ and click on VMware vSphere.

You then follow the three step instructions to deploy the appliance in your vSphere environment and start collecting information.

I logged into my vSphere Client, then clicked on File and Deploy OVF Template. In the Deploy from a file or URL field, I paste the URL that I copied from the ManageIQ download page, which is http://manageiq.org/download/manageiq-vsphere-stable.ova and walk through the process of deploying the ManageIQ appliance.

Once the appliance has been deployed, I open up a browser and connect to the appliance with the IP address. I then enter the default username of admin with the default password smartvm to log into the ManageIQ appliance for the first time.

The next step is configuring the appliance. This is going to be found on Configure on the top ribbon and then the Configuration link. You are going to want to select your Default Zone that was created when it was deployed, that is the highlighted item in the image below on the Settings window.

The options I am going to modify are:

Company Name
Appliance Name
Application Time Zone
NTP Servers

I am also going to check off 3 additional Server Controls:

Capacity & Utilization Coordinator
Capacity & Utilization Data Collector
Capacity & Utilization Data Processor

The Capacity & Utilization settings collect usage data. For example, if we select to collect data from a cluster, you are choosing to collect data for all hosts and virtual machines that are part of that cluster.

For authentication, I wanted to use my home.virtlab.com domain; I clicked on the Authentication tab and entered my LDAP information.

If you do not check Get User Groups from LDAP, the user must be defined in the VMDB using the console where the User ID is the same as the users name in your directory service typed in lowercase. For example, when using User Principal Name for my account it is cn=jason gaudreau,ou=users,dc=home,dc=virtlab,dc=com (CN=) and when using Distinguished Name for my account it is uid=jason gaudreau,ou=users,dc=home,dc=virtlab,dc=com (UID=). Then, when logging in, the user would type either jgaudreau (User Principal Name) or jason gaudreau (Distinguished Name). If the user is not defined in the VMDB, they will be denied access to ManageIQ Management Engine.

Suffice it to say, it is much easier to enable Get User form LDAP.

Now we are going to add our LDAP group as a super administrator to the appliance, we are going to click on Access Control, which should be at the very bottom left side of the screen after the Settings. Select Groups, then click the Configuration down-down menu and click Add a new Group.

We are going to provide a description for our new group, I am going to call mine Administrators, click the (Look Up LDAP Groups) option, and select the Role EvmRole-super-administrator.

Under the LDAP Group Look Up, you are going to search on an account that is a member of the group you want to add; in my case, I look up jgaudreau. You then enter the user account and password that has access to Active Directory.

When you hit Retrieve, another field will show up for LDAP Groups for User. These are the groups that my account is a member of in Active Directory. I am going to select Administrators and then click the Add button at the bottom of the screen.

Before we leave Access Control, we should change the password for the default Administrator account.

The last step for today's post is configuring our Capacity & Utilization Collection. We are going to go back to the Settings option and then highlight the very first entry, which should be your default region. Next click on the C & U Collection tab. Because I am going to be adding an Amazon EC2 provider, I click on the Collect for All Clusters option for Clusters; and Collect for All Datastores option for Datastores. After I add my Infrastructure and Cloud Providers, it will start gathering information.

In my next post, I will go through adding the Providers into ManageIQ, which gives you the ability to manage hosts and virtual machines.