Overall, this post touches on some of the options and strategies administrators have when planning to provide application consistency for both Windows and Linux VMs. Various links to KB articles, blog posts, and white papers are provided for additional reading.

This post is part of the VMETC.com From the Field Series on Veeam v6 Replication

The rest of this post is divided into 4 sections:

• Agentless options for Windows VMs
• Agentless options for Linux VMs
• VM Cluster nodes
• Should you truncate logs for a replication job?

A good download for starters

A few years ago Veeam presented Considerations for backing up Exchange and Oracle for a VMware User Group. These are popular applications, and I refer to this information in the rest of this post. The methods discussed for each provide a great example of the process and strategy involved in both Windows and Linux VMs. Download a copy here:

Veeam Backup and Replication Considerations for Exchange and Oracle (1340)

Agentless options for Windows VMs

Microsoft’s VSS framework makes quiescing applications running in Windows VMs a relatively easy process for the replication software without the need for installing agents in the VM. In short, code already exists to make the application consistent and put the OS in a recovery mode necessary to safely, and properly, restart the VM. For applications like SQL, Exchange, Oracle (when running on Windows), and Sharepoint, the VSS framework can even be leveraged to commit transaction logs. The VSS framework provides 2/3 of the solution. Veeam Backup and Replication provides the remaining pieces to ensure reliable application consistency.

As shown in the screen shots below, enabling transaction consistent replication with Veeam is as easy as choosing a checkbox and providing VM local admin credentials.

Veeam Backup & Replication – Step 12. Enable Application-Aware Image Processing

Some more recommended reading if you need to understand more about VSS:

What is Windows VSS & why you should care | Mr. Backup Blog

VMware and Microsoft VSS: What You Need to Know

Agentless options for Linux VMs

Applications running in Linux VMs require a little more set up today outside of the Veeam GUI, but the minor effort is well worth the time. The answer is to utilize VMtools and their ability to call pre and post thaw scripts.

Here are a few well documented how to examples from other IT Pros for the most popular Linux apps – MySQL and Oracle:

How to Quiesce Linux Virtual Machines with Veeam Backup – MySQL | HiperLogic

On Windows, Veeam can do application consistent backups through the Veeam Microsoft Volume Shadow Copy Services (VSS) integration for VSS aware applications. Unfortunately Linux does not have a system like Microsoft Volume Shadow Copy Services (VSS) that allows applications to be made application consistent by backup applications like VMware vDR or Veeam Backup.A simple workaround is to leverage VMware tools ability to call custom freeze/thaw scripts during a quiesced snapshot to enable customers to do the required application quiescence.The example [in the post] is for MySQL, for other applications you need to know the application specific command to quiesce the application and modify the scripts below appropriately.

Follow the link above to see the specific scripts for the MySQL example

Hot Backup of MySQL on a Linux VM

This whitepaper written by Veeam also shows a detailed process for MySQL VMs and provides sample scripts

Moving on, there are 3 common strategies for Oracle as outlined in the PDF copy of the VMUG presentation (downloadable above) :

The PDF provides the technical details of all 3 methods for Oracle running in Linux.

A possible solution of the future for Linux VMs may be the experimental vmsync option of the VMware tools. If you pay attention when installing VMTools for Linux you will notice the option for vmsync with a default answer to leave it disabled. vmsync seems to be the “VSS-ish” solution to enable the future GUI config of non-windows application quiescing.

Although I have not personally tested it, others have blogged about their experience enabling vmsync.

Filesystem consistent Linux Backups with VMware | Virten.net

Solution 2: vmsyncAs mentioned above VMware has created a sync driver that allows to create consistent backups. Unfortunately i couldn’t find any information about this driver and it is disabled by default, giving a small explanation:[EXPERIMENTAL] The VMware FileSystem Sync Driver (vmsync) is a new feature that creates backups of virtual machines. Please refer to the VMware Knowledge Base for more details on this capability. Do you wish to enable this feature?This comment emphasizes that this feature is not supported at the moment. I also couldn’t find this Knowledge Base details.

Replication of VM Cluster nodes

Unfortunately, agentless, host based replication solutions like Veeam cannot integrate with a clustered application. What you are replicating is the entire VM for one or more nodes. This means that the quorum and shared configuration of the cluster must also be replicated to the fail over location. This is often done outside of Veeam unless the quorum is somehow configured within a virtual disk that can be replicated. This is rarely the case. If you are able to account for all of the shared storage and networking at a DR site, then failing over may work.

I would suggest first analyzing how your cluster behaves when a node fails. What are the necessary pieces in your virtual infrastructure to keep the application available to the users running with a failed node? Essentially, the application cluster fail over state is what you need to replicate.  You may not need to replicate all nodes, for example. If you only need a single node for the application to operate then save the storage space and the bandwidth and only replicate one VM node. Better yet, replicate the passive node if possible to lessen the impact on production while the job is running.

Should you truncate logs when replicating?

The gotcha with replicating VMs that you are already backing up could be that you introduce a double log truncation schedule from 2 different jobs. Veeam replication jobs are independent of backup jobs and only provide cross intelligence for schedule queuing and conflicts.

To avoid the double schedule simply disable log truncation for the replica jobs (select “do not truncate logs” in the Veeam screen shot from earlier). Since Veeam also provides file level restores from replication jobs, you might consider only replicating VMs and skipping the backups for certain VMs.

Summary

As always with virtualization, the total solution for application consistency requires the cooperation of multiple technologies. A host based replication product such as Veeam will act as the “conductor” of the over all process, but one or more of the “instruments” in the DR “concert” may need to be fine tuned. This post hopefully provided several examples to help you get started.

Posted with Blogsy

This post explains the new proxy architecture and requirements at a high level and offers some tips on making sure things work right. There are a few screen shots from the Replication and Proxy wizards, but If you need more technical details on Veeam Proxies and the actual features or job configs please be sure to check the Veeam User Guide and the Veeam Forum F.A.Q (linked below).

This post is part of the VMETC.com From the Field Series on Veeam v6 Replication

Proxy Primer

The easiest way to understand the Veeam v6 Proxy architecture is to realize the data flow of a Veeam replication job. That is explained very simply in the Veeam Forums Backup and Replication F.A.Q:

READ THIS FIRST : [FAQ] FREQUENTLY ASKED QUESTIONS

Architecture

Q: What is the data flow in case of replication?

A: Disk > Source proxy > Network > Target proxy > Disk

Q: Can I use the same source and target proxy for replication?

A: Yes, but only when replicating locally (on-site replication).

“Danger Will Robinson!” Notice the last Q&A? You will need at least one Veeam Proxy at both the Source and Target sites if they are different locations. You may need to add more than one Veeam Proxy per site. More on why later in this post.

The basic Veeam replication architecture is illustrated in the following whiteboard diagram:

Architecture Overview

• Install Veeam at the target location for replication jobs. Installing Veeam also creates the target proxy.
• Use the Veeam console to create a Proxy at the source location.
• A Veeam Proxy is any supported Windows OS (virtual or physical). Linux servers are not an option.

Note that I am dicussing replication only in this post. Many customers do install a separate instance of Veeam at the source site for nightly backups. In that scenario both Veeam installs are independent and do not share jobs or configs. Veeam Enterprise Manager (not illustrated in the whiteboard) then consolidates both installs in a single pain of glass for monitoring and managing all jobs at both sites.

Proxies – where and why

Why put a separate proxy at each location? As already established, replication data flow moves from source disk to target disk through the proxies (no longer through ESX management nics!), but there is more than just the data path to understand.

The source proxy will read the VM blocks from the local datastores, compress and dedupe the VMs, and then send it across the WAN in the compressed form. The target proxy will then re-hydrate the VM as it writes it to the local datastores at the target. As you can see, each proxy not only moves data but performs several other tasks – read, write, manage, compress, inflate, etc. The correct selection of the proxies local to each site is critical to optimized and efficient replication.

Here are two commonly made mistakes.

1. Veeam installed at source site only – This means use of a single proxy (the default Veeam Server install). VM blocks will be read and compressed but then inflated again before it ever leaves the source site. The replica VM will be written to the target datastore across the WAN uncompressed.
2. Veeam installed at target site only – Again, the default install means a single Proxy. VM blocks will be read and compressed at the target site – remotely. Data traverses the WAN before being compressed. After reaching the target proxy data is compressed (needlessly) and inflated again, and then written to the target datastore.

The above two scenarios are oversimplified. With both there could also be a lot of traffic “ping pong” bounced back and forth between sites as the Veeam Console (GUI) attempts to manage and send commands between the proxies.

Pick your Proxies Manually

After deploying a Veeam Proxy at both locations, I recommend manually configuring specific proxies for each job. Do this on the Job Settings screen in the Data transfer section.

In the screen shot above you can see the fields for the Source proxy and the Target proxy. Clicking the “Choose” button allows you to tick checkboxes and select from all your configured proxies. Choosing multiple proxies for each job is possible for redundancy.

Be careful with Automatic Proxy Selection

The default data transfer settings is “automatic”. This means that Veeam will use it's own A.I. to pick proxies for you. This is done by watching the availability and load on every configured proxy. This can be a bad thing, however. As proxies take on jobs and reach their max concurrent tasks (more later in this post), Veeam mght pick a remote proxy to handle local tasks just because it is the only proxy available for the “automatic” selection. Personally, I would rather a job queue and wait for the right proxies to become available as opposed to introducing uncompressed replication or management “ping pong” traffic on the WAN.

Repository for Replication

In the replication Job Settings screen shot above, the Repository field is below the Source and Target Proxy fields. Many customers confuse this setting with Veeam's Backup Repositories. The same terminology leads one to believe their target disk for backup jobs must be configured as part of a replication job. Add to that the repository for replication is created exactly the same as a repository for backup. Despite the similarity, a replication repository most often should be a new and separate server at the source site. A recent Forum thread clearly explains why:

It's basically just a place to store the hashes for deduplication during replication. Since dedupe is performed by the source proxy it's best to keep it on the source side, otherwise the metadata has to be read/written across the WAN by the source proxy.

If the metadata is missing, Veeam will simply recreate it by rescanning the target side and recalculating the hashes, but it's only needed during a replication, not for failover.

A replication job repository is significantly smaller (a few gbs in size so no extra space is needed) than it's backup cousin. When in doubt use the source site Proxy (add it as a Repository in Veeam too). Recalculating the hashes can take some time, so try to pick a server that will be consistantly available at the source site.

DNS and Permissions

A Veeam replication job to a DR site spans multiple datacenters. That means there is often different security, resolution, and permissions. – both in the virtual infrstructure and the directory services of the guest VMs. Looking at the whiteboard from earlier in this post will help you visualize this design challenge.

I have found that the following two Veeam KB Articles are frequently needed by customers setting up new replication jobs:

KB1518: Ports needed for use with Veeam backup and replication 6

KB1198: NFC server connection is unavailable: troubleshooting steps

Permanent failover

Note that the placement of the Veeam Console ( install ) is no longer critical to the actual performance of the replication job. Since Proxies now handle the actual flow of data between sites, Veeam simply sends commands to start the job and then collects the statistics and results. Should the Veeam server be lost during a DR event, all replica VMs can still be started manually via the hypervisor's native management client.

Concurrent Jobs (or Tasks)

The Add Proxy Wizard initially allows you to configure the Max concurrent tasks. You can right click a previosuly configured Proxy and change the Properties later.

The Proxy rule of thumb is this – For every 2 CPUs on a Proxy Veeam can send it 1 job (task).

In the screen shot above, this proxy will only handle one job at a time. I borrowed the image from Rick Vanover's post How to install a Veeam Proxy (vSphere). He does a great job explaining the process there. Vanover also has a walkthrough post on Veeam's blog about how to add Veeam Proxies.

Bottom line is that for all Veeam proxies, both at the source and target sites, give your proxies the most amount of CPUs you can afford to. That allows you to make 80-20 VM adjustments, create more jobs to run simultaneously, and get the most done in your replication window.

More about Proxy concurrent tasks can be foud in the online Veeam Help – Veeam Backup & Replication – Limiting the Number of Concurrent Tasks.

Summary

Veeam replication has many options to meet any VMware or Hyper-V infrastructure demands. Granular feature configs and a flexible architecture of Veeam Proxies create a disaster recovery solution that can scale to any size as needed. With a deeper understanding of all the options available, an admin can make smart changes to the deafult Veeam install that allow for fast and efficient replicas in the job window desired.

Posted with Blogsy

This post is part of the VMETC.com From the Field Series on Veeam v6 Replication

Fact is, if you want cost effective replicas of your VMs for disaster recovery fail-over using hypervisor based technologies you are about to ask your V.I. to “turn your head and cough”. Reliable, fast and efficient replication in a minimal window starts with providing good health in your storage, hosts, and virtual machines. You will probably have to make some changes to your architecture and design. As I've pointed out before, few planned and then actually built their virtual datacenter with a specific tool for disaster recovery in mind. The good news is that by optiizing snapshots you will probably fix many those nagging performance issues you've struggled with for a long time.

I know. I know. I and others have said in the past that snapshots in in your virtual datacenter are a bad idea. Well, the context of the discussion before was not to leave running snapshots open. That is not the context here. In fact, hypervisor based replication should never leave behind snapshots in your environment. Veeam Backup and Replication does not, nor do all of the other alternatives that I know about, leave behind snapshots. Sure, things can go wrong, but that is usually because of abnormal events in the environment (connectivity loss, Veeam architecture server reboots during jobs, etc). Things go wrong with snapshots without Veeam in the mix all the time. More on this later in the post.

BTW, Veeam is also smart enough to clean up left behind snaphots from previous jobs, and smart enough to leave a snapshot alone that Veeam did not create. There is never a “delete all” command issued to the hypervisor. If a snapshot already exists then Veeam does it thing with a new snapshot without touching the existing one.

The point is snapshots play a major role in hypervisor based replication and you can optimize their performance. An unhealthy snapshot process means that the replication job will take excessively long or will fail. This post explores what can be done.

Snapshot Best Practices

Both VMware and Microsoft have snapshot best practices. For starters, both vendors plainly state in almost the same words: “do not use snapshots as backups”. In fact, both state the issue with snapshots is not creating or using them but instead recognize the potential issues of filling up a datastore and committing the data accumulated inside a snapshot back to the production VM.

Hyper-V Virtual Machine Snapshots: FAQ

We do not recommend using virtual machine snapshots as a permanent data or system recovery solution. Even though virtual machine snapshots provide a convenient way to store different points of system state, data, and configuration, there are some inherent risks of unintended data loss if they are not managed appropriately. A backup solution helps provide protection that is not provided by snapshots.

Since Veeam does not use Hyper-V snapshots to backup or replicate VMs, I will focus on VMware for the rest of this post.

VMware KB: Best practices for virtual machine snapshots in the VMware environment

Use no single snapshot for more than 24-72 hours.

This prevents snapshots from growing so large as to cause issues when deleting/committing them to the original virtual machine disks. Take the snapshot, make the changes to the virtual machine, and delete/commit the snapshot as soon as you have verified the proper working state of the virtual machine.

Be especially diligent with snapshot use on high-transaction virtual machines such as email and database servers. These snapshots can very quickly grow in size, filling datastore space. Commit snapshots on these virtual machines as soon as you have verified the proper working state of the process you are testing.|

Read the entire best practices for both hypervisors using the links above.

Problems when using snapshots

VMware has a KB article that lists 9 things that can cause headaches with snapshots. Here's a quick summary in my own words: Snapshots are special virtual disks that need space on your datastores, and if you have permissions to create and commit them they will cause extra I/O that might have a negative impact. Make sure you have resources to support them.

Review the KB article for yourself, and, especially if you have had problems in the past, manually create and commit some snapshots to get an idea of where you stand before you start to replicate VMs.

Improving snapshot commit times to minimize replication windows

After you've got your snapshots to work and have successfully run your replication jobs examine the replication job report to see the total time the snapshot creates and commits take. With Veeam you can click on each VM in the Statistics window and find the lines where Veeam is waiting for the snapshots. You could also look at the tasks in the vSphere Client to see the snapshot progress.

Excessive time to commit a VMware snapshot, especially with a database or messaging VM, is often the result of too much I/O for the underlying datastore. You've got the normal production I/O of the VM combined with the extra I/O of commiting the changes in the 0001.vmdk (for example) made during the time the snapshot was open (the job).

If you are using vSphere 4 or earlier, the snapshot is actually created on the datastore where the VM's .vmx file lives. I've seen the time it takes to commit snapshots drastically reduced just by moving the VM's .vmx file to another datastore. Oversimplified, you are then dividing the production and snapshot I/O across the different datatstores. If you have different spindles and controllers for the different datastores performance gets even better. Beware – make sure the datastore you move the .vmx to has a block size and enough free space to handle the snapshot. I've known customers to build a 1TB datastore just for .vmx files so VMware snapshots would perform better.

If you are using vSphere 5 or newer then the snapshots are now created on each datastore where there is a vmdk. If you want to, you can still specify a working directory for snapshots even in ESXi 5.

Of course you can also improve the disk performance of your datatstores. The number of disks in the array, the RAID level, or even the type and number of disks all make a difference. SIOC might even help out.

Summary

Identifying the VMs that have to wait on poor snapshot performance is part of the 80-20 VM adjustment phase I talked about in the previous post of this series. If you can't make any changes to your VMs or your datastores right away then consider separating these VMs from the ones that finish quickly by creating a new job. As long as jobs are finishing and the problem seems to be isolated to a few VMs then you can live with this for a while.

Be sure to check out the other posts from this series!

Posted with Blogsy

Browse to the .edb with VEX

Psst! Let me let you in on a little secret. Veeam Explorer for Exchange (VEX) is a little more versatile then you might think.

There's been some great coverage on the new public beta and VEX's awesome ability to crack open Exchange 2010 Veeam Backups like a piñata of messaging goodness, but the fiesta doesn't stop with just Veeam Backups.

The truth is, no matter how you backed up your Exchange 2010 server(s), all you have to do is recover your Exchange .edb files. You can then use VEX to browse to the restored location of the unmounted database file. That means that whether you snapped a LUN, backed up Exchange with an guest agent, or even have backups from alternative products, if you can restore the .edb files VEX can do it's thing! The first time you run VEX you will need access to an Exchange .ese file since Veeam is not allowed to distribute that file, but that's another simple file restore or you can always grab a copy from your Exchange server.

Yes, you will have to install the next version of Veeam Backup and Replication to use VEX, but VEX will be available in the free, trial, and licensed versions.

That's right, you will be able to eliminate the time, effort, space, and cost of brick level Exchange 2010 backups but still granularly recover the mailbox items. Even with Veeam's free version.

Here's the VEX Overview details from Veeam's current product page:

Instant visibility

Veeam Explorer™ for Microsoft Exchange gives you instant visibility into your Exchange backups. You can browse, search and selectively export items (emails, notes, contacts, etc.) directly from Veeam backups of your Exchange virtual machines (VMs).

Fast

Veeam Explorer for Exchange works directly with your Exchange VM backups—even compressed, deduplicated and incremental backups. There’s no need to provision storage or restore entire mailbox stores—just choose the Exchange Servers and restore points you’re interested in, and Veeam Explorer will present the contents of those mailbox stores for browsing, searching and export.

Free

Veeam Explorer for Exchange is a new feature of Veeam Backup & Replication. Currently available as a beta, Veeam Explorer will be included in all editions of Veeam Backup—even Free Edition. You no longer need expensive standalone tools for Exchange item recovery that are licensed per mailbox.

E-discovery

Use Veeam Explorer for Exchange to find and retrieve information for internal or external investigations. The Advanced Find feature lets you zero in on exactly the items you need.

Item-level recovery

When users accidentally delete emails or scripts corrupt calendar items, you can save the day with Veeam Explorer for Exchange. Simply export a good version from a previous Veeam backup and transfer it to the affected user.

Mailbox archive

Veeam Explorer for Exchange makes it easy to archive an entire mailbox—from any point in time—to a PST file. This helps ensure a smooth, complete handoff when employees leave or change jobs.

Try it for yourself

Veeam Explorer for Exchange is a new feature of Veeam Backup & Replication. A beta version is available now by invitation, for you to install on your existing Veeam Backup server. The released version will be included in the next release of Veeam Backup & Replication and available in all editions (Free, Standard and Enterprise).

Get an Invite to Test Veeam Explorer for Microsoft Exchange!

Posted with Blogsy

This post is part of the VMETC.com From the Field Series on Veeam v6 Replication

I suggest that you give yourself at least 30 -45 days to optimize and finalize your replication design and schedule. Breaking it down in phases, I would recommend an approach using Veeam v6.x as your solution like this:

1. Phase 1 (2 weeks) – install, configure, and get jobs running
2. Phase 2 (2 weeks) – The 80/20 VM(s) Adjustment
3. Phase 3 (1 week) – Final Tweaks
4. Ongoing monitoring (forever!)

I am not diving into the technical details in this series, but there are already many posts on installing Veeam v6 and creating Proxies, creating replication jobs, or you can follow the Evaluators Guides for vSphere or Hyper-V. I've also linked to various blogs and helpful info all throughout this post.

Let's expand on what to do and what to expect in each phase.

Phase 1 – Install, configure, and get jobs running

Phase 1 should be looked as a proof of concept or as a test phase regardless if implemented in production and regardless whether you plan to keep the initial Veeam install. In fact, agentless replication via the hypervisor API makes it easy to start in production and adjust later. Why? Because the infrastructure you introduce to the production environment is minimal, and unless your utilization is pushing 80% across your clusters you should not see a significant negative impact.

Before you start I recommend you think about the last time you committed a snapshot, however. Did it cause any hiccups? I'll talk more about snapshots in another post of this series, but on the production side that will be the biggest reason not to move forward with running replica jobs.

At the target, or fail-over location, you need a minimal number of hosts with configured datastore(s). Veeam replica VMs will be created powered off, so the storage is really the key until you are ready to test or run the replicas. You do have the option to create thin provisioned replica VMs, but I would recommend matching available storage at the target to the amount of storage at the source. You don't need to worry about similar storage (vendor, type, etc). You just need to have the room to store the replicas.

The following is an oversimplified whiteboard of minimal Veeam architecture needed to get jobs started

Summary install and configure points

• Veeam is only installed once at the target location (more on picking an install location later in this series).
• A Veeam proxy is required at the source location. The default Veeam install provides the required proxy at the target location.
• All Veeam servers must be able to resolve and communicate with each other across both locations.
• vCenter or SCVMM is not needed (but certainly can be used).
• Veeam architecture can be all virtual, all physical, or any combination you want.

Once Veeam is installed and the Proxies are deployed, it's time to think about running jobs.

I always suggest running backup jobs first. Running local backups establishes a Veeam processing baseline and allows you to create a set of files that can be used as seeding for the replication jobs. Run the backup jobs using portable disk as a backup repository – USB drive, server with local storage, a notebook, whatever. You just need some way to get the backup files to the target site.

Here's the whiteboard updated to show how a disk drive was configured first a Veeam Repository at the source for backups and then moved to be a Repository at the target for seeding. Note that Proxies at each location need to use this Seeding target.

Veeam Replication Whiteboard w/ Seeding

Seeding is not required if your WAN is fast enough to transfer replicas of your VMs in a reasonable window, or if you have the patience and can wait for the first full replication to complete across your WAN – no matter how long it takes!

Another benefit of running a backup first is it lets you get a better idea of how much data you actually need to move, and then how much data rate of change you might expect in your VMs. ReplicaCalc is a tool you can use to guestimate the time your actual replication jobs will take based on info from Veeam backup jobs combined with your available bandwidth between locations.

Eventually, you can then divide your VMs into replication jobs. The strategy I suggest is that you split your VMs into groups that create replication jobs that will take 2 – 4 hours each. Then you want to schedule the jobs to run one at a time and on a staggered schedule so all jobs will fit in your replication window.

It doesn't matter how many VMs are in a single job or how many jobs you create. Just make sure you pay attention to the total time it takes to run a job. You only run into trouble when you schedule many jobs simultaneously, and I am not suggesting that for the first phase.

Run your jobs! If you use seeding then Veeam will use the files you put at the target site to do restores there and then only replicate the current changes across the WAN. Problems on the first run usually are the result of connectivity, DNS resolution, permissions, or other environmental scenarios. Look again at the whiteboards earlier in this post. Often the Veeam replication infrastructure stretches across domains, DNS servers, subnets, firewalls, and even virtual infrastructures with separate permissions. More on these gotchas later in the series.

Phase 2 – The 80-20 VM(s) Adjustment

I have seen very few customers get their replication job / schedule / proxy mix right the first time. It doesn't matter if a customer is an SMB or an Enterprise, the initial replication jobs are rarely the final, go forward configuration. What I do see is that for 80% of the VMs the first runs are fine. Then there is always that other 20%.

Databases, messaging, or any other applications that generate significant I/O during the replication window usually result in VMs that take significantly longer to replicate then the rest of the VMs. Look at the Statistics window or html report of each job and you will see how much time each VM needed to complete. Furthermore, look at the Statistics of each VM (click on the VM in the Statistics window) and see what part of the job took the longest for that VM. Be sure to analyze your jobs after you have successfully run jobs more than once. Get a feel for the average time each one of your VMs takes to replicate incrementally and not just during that first full (regardless of seeding).

A simple adjustment example

A customer has 2 replication jobs each with 20 VMs. Looking at the Statistics of Job 1 they see a job usually takes around 4 hours and 15 minutes. When you look at the time it takes for each VM in that job you see that 18 VMs usually finish in 5 minutes each (90 minutes total) but the other 2 VMs take 2:45. For simplicity lets say Job 2 has the same results.

The customer started with 1 proxy at each location (like the whiteboard). Each proxy has 4 CPUs and can run 2 concurrent tasks (jobs).

Job 1 is scheduled to start at 10 pm. Job 2 is scheduled to start at 2 am. Job 2 is finishing between 6:30 and 7 am every morning.

As you can see the jobs results show that 36 VMs finish quickly and 4 VMs do not. We won't worry about why they don't for this example. That's for later in the series. Let's focus on what you can do in Veeam to make these jobs finish faster.

If you removed the 4 VMs taking the longest from their current jobs, Job 1 and Job 2 would finish in 90 minutes. Since the proxy can handle 2 jobs concurrently and only 1 job is running at 10 pm we have room for another job. Taking the 2 VMs that used to be in Job 1 and creating a Job 3 would be my advice. Doing the same to create a Job 4 at 2 am would also make sense. Now when the jobs run Job 1 is not held up by the VMs in Job 3 and likewise for Job 2 and Job 4. The net result is that Job 1 and 3 are finished by 12:45 am and Jobs 2 and 4 are finished by 4:45 am.

Taking this one step further, nothing is happening between 12:45 am and 2:00 am. In fact, the Veeam Proxy has an open task after 11:30 pm because Job 1 only took 90 minutes. The customer could move Job 2's start time to 11:45 -ish and move Job 4's start time to 12:45 -ish. Every job could be complete by 4 am.

Remember the Seeding option we used to establish the first fulls for Job 1 and 2? You can do the same for the new Jobs 3 and 4. This time you could even use the existing replicas of the 4 VMs instead of making new backups. The Veeam job also let you choose existing VMs at the target site as Seeds.

Of course there are many other factors that influence the speed and efficiency of replication jobs, but the point here is by going through the process of running replication jobs for a while and getting a true understanding of average job times you can make intelligent adjustments to the Veeam jobs and Proxies. As you can see, the results are vastly different for each customer. There is no easy best practice design or deployment guide document to refer to. Running jobs and adjusting is the best way to finalize the replication infrstructure design and DR solution.

BTW, I realize the math makes this a 90-10 example. You get the point!

Phase 3 – Final Tweaks

After the adjustments run your jobs and make sure things happen as expected. Using our example, what if all the jobs were over at 6 am instead of 4 am? This is better than before, but why could this be happening?

Perhaps the Veeam Proxies, the “muscle” for moving all data across the replication path, are undersized or resource constrained. Maybe the Proxy OS is pegged at 100% CPU during the replication jobs. If the proxies are VMs, maybe the hosts are resource constrained and can't dedicate the resources needed to the Proxies. If you have to, add more Veeam proxies or switch from VMs to physical servers. Maybe your network or even your disk arrays (datastores) are the bottleneck.

Ongoing monitoring (forever!)

Even though you've worked through all the jobs, made the adjustments, fixed the bottlenecks, and your jobs have run smoothly for weeks that doesn't mean things can't suddenly change. Using our example again:

• 1 of the 36 VMs now takes 1 hour to replicate
• 20 new VMs have been added and need to be replicated
• 20 new VMs have “stolen” resources from Proxies during the replication window
• The V.I. has been upgraded / changed in a way that could impact jobs

There are many, many environmental factors that can cause unforseen disruptions in your replication jobs. be sure to keep an eye on your jobs and “rinse and repeat” the adjustments as needed.

Be sure to check out the rest of the posts in this series.

Posted with Blogsy

This series will not contain technical answers, official best practices, or “how to” steps on designing a Veeam Replication solution. The series will attempt to make virtualization admins think about their virtual infrastructure architecture, the virtual machines, and suggest what might need to be modified to maximize host-based replication. I will try to cover the topics from both the vSphere and Hyper-V perspective.

If you need more technical details, information about Veeam features, or help with the “how to” on any of the topics covered please contact your regional Veeam Engineer or your Veeam Partner / Valued Added Reseller.

Here is my outline of posts to come in the FTF Veeam Replication series:

Phased Implementation Methodology (link)

1. Phase 1 (2 weeks) – Install, configure, and get jobs running
2. Phase 2 (2 weeks) – The 80-20 VM(s) Adjustment
3. Phase 3 (1 week) – Final Tweaks
4. Ongoing monitoring (forever!)

Optimizing Snapshots (link)

If you are planning on replicating VMware VMs then you have to be able to create and commit healthy VMware snapshots before you ever introduce Veeam in your environment.

Veeam Architecture (link)

• Proxies – where and why
• DNS and Permissions
• Where to install the Veeam Server?
• Concurrent Jobs (or Tasks)

Applications in the VMs (link)

• Agentless options is Windows VMs
• Agentless options in Linux VMs
• VM cluster nodes
• Should you truncate logs for replication jobs?

CBT and VM data rate of change

Once Veeam establishes a full replica of your VM on the target datastore it will only replicate changed blocks. How many changed blocks each job run depends on each VM’s personality.

Replication and Network Physics

Veeam can help a little by throttling itself, but the replica data has to be sent across your WAN!

Other Gotchas

• ESX Versions
• Upgrades and the API
• Backup Repository for Replication
• Building jobs
• Datastore space at the target site

The information provided in this series is not necessarily supported or endorsed by Veeam software. This series is based solely on my experience and ideas. FWIW I have been acredited as a VMSP and VMTSP, and certified as a Backup Expert by the Backup Academy.

I’ll expand on the topics in each post! Once a post is published I will add links to it here. This post then becomes a table of contents for the series.

Let me know in the comments if there is something else you would like to see covered.

Posted with Blogsy

So what exactly is VeeamZIP and when would you use it? Here is how it is explained in the What’s New document:

A questionable reputation

Most people so far seem to be associating VeeamZIP with just the free version of BaR v6.1. Fact is that, although the free version does feature VeeamZIP, the full version allows you to use VeeamZIP with an extra benefit – You can create an ad-hoc backup of multiple VMs!

To use VeeamZIP in the licensed (or fully functioning trial version) go to Virtual Machines, select the Management Server or Hypervisor, and then shift select or ctrl select your VMs.

You will then be able to use the VeeamZIP button or right-click the selected VMs to start your job.

select multiple VMs and right click

Don’t let the name fool you

Contrary to the name, a VeeamZIP file does not have a .zip extension. It is actually a Full Veeam backup file with a .vbk extension. Just like all Veeam .vbks, it is deduped and compressed to use as little space as possible.  You have full control where you save the VeeamZIP file.

It’s all about the restore

Finally, you can do restores from the VeeamZIP file. You don’t even have to import the .vbk back into Veeam. Just open Windows Explorer on the Veeam Server, Browse to where you saved the VeeamZIP .vbk and double-click it.

Related articles

• More free stuff – Monitor VMware within System Center with Veeam for free! (rickvanover.wordpress.com)
• Free Backup Utility for Virtualization Environments (garvis.ca)
• Veeam Backup and Replication 6.1 is Virtualization’s Best Friend (virtualizationreview.com)
• Backup Academy featured at IT Certification Master (rickvanover.wordpress.com)