Test Exercise Roles & Responsibilities

Test Design Team
A Test Design Team is responsible for designing the test exercise scenario. These individuals should have in-depth knowledge of the organization and departments being tested, and are able to produce "credible scenarios" and yet stay on course with the test plan. Typically, the Disaster Recovery Coordinator serves in the capacity of the Test Design Team.

Assistance from a third party to help design your exercise can provide an independent evaluation of your exercise. Firestorm® acts as the Design Team for many organizations' test exercises.

The Simulation Team
The Simulation Team will guide the participants through the test or simulation.

Simulation Team Guidelines:

• Know the test plan and the messages
• Know where the test is going
• Know your resources
• Know your messages

Simulation Room:

• The simulation room should be located near the test room, but far enough away where occupants cannot be heard.
• Have a sufficient number of phones.
• Have white boards or flip charts for scribes to note the current status.
• Key messages need to be noted for tracking.
• The room needs to have adequate room and wall space.

The Facilitator
The facilitator is responsible for central coordination of the test exercise. Firestorm acts as the facilitator for many organizations’ test exercises. The facilitator is responsible for overseeing the accomplishment of targeted objectives, and conducting a briefing following the exercise. Facilitators should be knowledgeable in the execution of the plan(s) being tested. The facilitator provides overall guidance and coordinates with the participants and should assure that:

• Participant instructions are prepared.
• Full scenario and player scenarios, as well as a master sequence of events are prepared.
• Evaluation forms are prepared.

The Test Assistant
A Test Assistant may be assigned to assist the facilitator throughout the testing process. Evaluators should be very knowledgeable of the plan(s) being tested. Evaluators should be assessing command, control, coordination, and communication activities, and should be observant and objective. The role of a test evaluator is to:

• Monitor test play.
• Evaluate actions, not players.
• Determine if the objectives and related actions are being met.
• Identify problems to the facilitator.
• Track key messages and report findings to facilitator.
• Evaluator Activities:
• Attend the pre-test briefing.
• Assist in the development of evaluation form.
• Review and know the test plan.
• Know the objectives, narrative and messages.
• Know the test organization.
• Report early to the test.
• Be positioned near intake phones so you will see where messages go and how they are handled.
• If key messages are lost, advise the facilitator so the message can be resent.
• Assign certain messages to specific evaluators so they can track their progress.
• Note message processing on evaluator forms.

Test participants should be familiar with their specific roles within the plan(s) that are being tested. They should be specifically named as team members within the plan (s).

Messages
Messages drive the test, expose unresolved issues, and address the objectives. They add information to describe the disaster environment and/or situation. Messages stimulate action by the participants. Messages can escalate an initial (primary) problem and create secondary or tertiary problems. Example:

• Primary event — earthquake
• Second event — building collapse
• Tertiary event — building fire

Messages should influence action at least one of four ways:

• Verification — information gathering
• Consideration — discussion, consultation
• Deferral — place on a priority list
• Decision — deploy or deny resources

Message component examples:

• Time — what time is it to be delivered within the test?
• Who — who is the source of the message?
• Mode — how was the message transmitted?
• To Whom — who is the recipient?
• What — is the content of the message?

Followup
Following completion of the test, the facilitator should review the test plan with the participants and answers questions. If possible, audio-visuals should be used to add realism. The best time for a debriefing is immediately after the test. The test facilitator should facilitate the session. The purpose of the debriefing is to:

• Review and evaluate the test
• Provide feedback
• Review lessons learned from the test
• Obtain feedback from all participants on what worked and what didn’t work
• Note issues of command, control, coordination, and communication
• Have each function/business unit chair report on their group

Written Evaluations:

• Test participants should evaluate the perceived value of the test and their overall reaction to the experience
• They should evaluate the existing plan(s)
• They should evaluate the test
• They should identify the need for further training and tests
• They should make suggestions for improvement

The test facilitator should incorporate debriefing comments, evaluator observations and participant evaluations into a concise report of the event including lessons learned, issues that need correction, next steps, and additional training needed.

Test exercise analyses should include:

• An assessment of whether the test exercise objectives were completed.
• An assessment of the validity of test exercise data processed.
• Corrective actions to address problems encountered.
• A description of any gaps between the plan(s) tested and actual test exercise results.
• Proposed modifications to the plan(s).
• Recommendations for future test exercises.
• The report should be completed within five working days of the test and distribute it to all participants.

Summary: Keys To A Successful Test Exercise
Having a clear objective, management support, a realistic scenario, and active involvement are key to an exercise success. The updates and plan changes based on the lessons learned in the exercise must be made and shared. In evaluating a plan(s), look for:

• Top level support and involvement.
• Test design team expertise.
• Realistic test plan.
• Thorough preparation and attention to detail.
• Clear introduction and instructions.
• Participant feedback at debriefing.
• Follow-up.

Introduction
The success of any emergency, continuity, or disaster recovery plan depends upon its routine testing, audits and updates. Plans cannot be expected to work properly unless they have been tested prior to their actual implementation in an emergency. Don’t wait until an emergency unfolds to see if the plans and procedures you’ve implemented are effective in responding to and recovering from a crisis event.

Everyone has a role in a crisis. Some are strategic, some are tactical. How decisions are made in a crisis is critical to the outcome. Because of this, the following holds true:

1. Practicing emergency response helps assure that the response can proceed predictably during a crisis or disaster;
2. Participation in exercises familiarizes everyone with the vulnerabilities, impacts, plans, mitigation strategies, incident management and crisis communications;
3. Testing allows problems or weaknesses to be identified and used to stimulate necessary and appropriate changes; and
4. Errors committed and experience gained during testing will provide valuable insights and lessons learned that can be factored into the planning/updating process.

Exercises empower critical decisions in a crisis. Exercises focus participants to determine:

• What changed?
• What do they know?
• Are they concerned? If so, about what?
• What is their plan?
• What will they monitor & how?

Process

Text Exercise Process
Test exercises serve several purposes. Exercises:

1. Allow management to use and assess plans and procedures to determine their feasibility and determine whether they will work under actual conditions.
2. Assess and measure the degree to which personnel understand their emergency response functions and duties.
3. Enhance coordination, communication, and proficiency among response staff.
4. Identify areas for improvement.
5. Increase the ability of management and staff to respond to emergencies.

Test Exercise Strategies

1. Test exercise strategies detail the conditions and frequency for testing applications and business functions, including the supporting information processing. The frequency and complexity of testing is based on the risks to a company or organization.
2. Tests can be as simple as testing the call tree in one or more plans. Or, the test exercise can involve an integration of multiple business areas, the IT environment and link to outside vendors and customers. The complexity of the tests should vary to ensure that all components of the plan(s) are adequately exercised.
3. Companies should participate in tests with their core service providers and test other critical components of the BCP. The strategy should include test objectives, scripts, and schedules, as well as provide for review and reporting of test results. Best practices in the industry support testing at least annually, or more frequently, depending on the operating environment and criticality of the applications and business functions.

Test Exercise Scope And Objectives

1. The scope of the drill or test exercise is determined by what is required to ensure the learning objectives are achieved by the participants. For example, if the objective is to test the ability of senior management to make decisions as specified in the emergency management plan, a tabletop exercise would be appropriate, although the same objective could be tested during a full-scale exercise.
2. Companies must clearly define what functions, systems, or processes are going to be tested and what will constitute a successful test. The objective of a testing program is to ensure that the plan(s) being tested can remain accurate, relevant, and operable under adverse conditions.
3. Testing should include applications and business functions that were identified during the In/Out/Across™ Analysis (BIA). The BIA determines the recovery point objectives and recovery time objectives, which then help determine the appropriate recovery strategy.
4. The scope of individual tests should be continually expanded to eventually encompass enterprise-wide testing, including vendors and key market participants. Achieving the following objectives will provide progressive levels of assurance and confidence in the plan(s).

At a minimum, the clearly stated testing plan should:

• Not jeopardize normal business operations.
• Gradually increase the complexity, level of participation, functions, and physical locations involved.
• Demonstrate a variety of management and response proficiencies, under simulated crisis conditions, progressively involving more resources and participants.
• Uncover inadequacies, so that configurations and procedures can be corrected.
• Consider deviating from the test script to interject unplanned events, such as the loss of key individuals or services.
• Be sure to inform participants of the objectives and goals of the test exercises.

Test Exercise Expectations

• Setting the proper expectations will minimize frustrations or inadequate participation from key stakeholders. As general guidelines:
• Test exercises scenarios should be realistic.
• Test exercises should consist of a generic scenario, and be indicative of an event that could happen in the area.
• Test exercises should not be too complex for the situation. Test exercises should compress a two or three-day real situation into a few hours, so they will be kept relatively simple with only a few objectives.
• One or two key threats should be the focus of each test exercise.

The test procedures should be checked periodically to make sure they include:

• Emergency response procedures, including escalation and notification processes.
• Alternate processing procedures, including security procedures at an alternate site.
• Full recovery procedures, including returning to normal processing.

Types Of Test Exercises
Testing methods should vary from minimum preparation and resources to the most complex.

Orientation/Walkthrough — Briefing or low stress training to familiarize participants with team roles, responsibilities, and expectations. Provides a good overview of new or revised emergency response plans. This type of exercise helps orient new staff and leadership. Planning cycle: one month; Test time: 60-90 minutes.

Drill — Test of individual emergency response functions that involve actual field responses. Examples include fire drill, tornado test, etc. Planning cycle: one month; Test time: 10-60 minutes.

Tabletop — Limited simulation or scenario of an emergency situation to evaluate plans, procedures, coordination, and assignment of resources. Advanced table tops will introduce messages and test assistants who can answer questions. Planning cycle: two-three months; Test time: 90-120 minutes; Debriefing time: 30 minutes.

Functional — Limited involvement or simulation by field operations to test communication, preparedness, and availability/deployment of operational resources. Planning cycle: three-six months; Test time: 90 minutes – 4 hours.

Full-scale — Evaluates the operational capability of systems in an interactive manner over a substantial period of time. Conducted in an environment created to simulate a real-life situation. Planning cycle: three-six months; Test time: 2 – 8 hours.

Know What You Are Exercising
Companies can establish exercise testing for a variety of reasons. Focus can vary from a single department to incident management, crisis management, and to company-wide and Board involvement.

Examples include:

• Response/Recovery Team Alert List — Contact information for all personnel assigned to the team. As this list can change frequently, team leaders should send a copy of it to each team member to review and update.
• Critical Functions List — Critical functions that each team must accomplish during a recovery effort. Team leaders should review these functions to determine that they are relevant.
• Team Recovery Steps — Strategies for recovery of critical functions; should be reviewed to validate that strategies are meeting current business objectives and reflect the best possible solutions.
• Functional Recovery Steps — Step-by-step procedures to complete the desired operational recovery; should be carefully reviewed and validated to determine accuracy and completeness.
• Vendor and Customer List — Contact information for critical vendors and customers; should be reviewed to determine list accuracy and completeness.
• Work Area Requirements — Critical resources required to support recovery at a designated work area site; should be reviewed to determine list accuracy and completeness.
• Off-Site Storage List — Critical records or resources stored off site; should be reviewed to determine accuracy and completeness.