chapter 3 - Kendall Hunt Publishing | Kendall Hunt Publishers...According to William Rothwell and H. C. Kazanas, instructional design involves: analyzing human performance problems - [PDF Document] (2024)

75

chapter 3Analysis Phase

LEARNING OBJECTIVES1. Identify the kinds of needs that must beconsidered in developing training activities and how to

go about determining those needs for an organization.2. Explainthe importance of data collection and the benefits and drawbacks ofspecific data col-

lection methods.3. Describe strategies for gaining informationfrom clients and presenting results of your analysis

to clients.

In the previous chapter, we started a discussion about thenature of learning and the various theories related to learning. Weare now going to start moving through the traditional processtraining and development professionals use to create trainingprograms. The focus in this chapter is going to be

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

76 TRAINING and DEVELOPMENT

on the first stage of creating a training program, analysis.However, before we jump into analysis, we need to discuss theconcept of instructional systems design (ISD).

Instructional Systems DesignAccording to William Rothwell and H.C. Kazanas, instructional design involves:

analyzing human performance problems systematically, identifyingthe root causes of those problems, considering various solutions toaddress the root causes, and implementing the solutions in waysdesigned to minimize the unintended conse-quences of correctiveaction. Instructional design usually encompasses not just thepreparation of work-related instruction but also the selection ofsuch management solutions to human performance problems as thepreparation and use of job aids, the redesign of organizationalstructure and reporting relationships, the redesign of jobs andtasks, the refocusing of employee selection methods, thereengineering of job-related and task-related feedback methods, andthe design and implementation of employee reward programs.1

As you can see from this explanation, instructional designencompasses a wide range of issues that must be considered in thepreparation of training programs. Over the years a number of groupshave created systematic ways of handling the instructional designprocess. The one most commonly discussed in the T&D field isinstructional systems design (ISD), the practice of usingconsistent and reliable procedures while creating instructionalexperiences. In 1975 a group of researchers at Florida StateUniversity working on behalf of the U.S. Department of Defensecreated one of the most commonly used models for handlinginstructional design. 2 The group of researchers helped to createwhat they dubbed the Systems Approach to Training (SAT) model,which became the foundation for the field of ISD. In the early1980s, the SAT was updated for more general training purposes andthe ADDIE Model was born.

The ADDIE Model (Figure 3.1) stands for Analysis, Design,Develop, Implement, and Evalu-ate. The major purpose of this bookis to walk you through the ADDIE Model in an attempt to help youunderstand all of the important facets of the training process. Nowthat we’ve briefly explored the concept of instructional systemsdesign, we can start focusing on the first letter in the ADDIEModel, analysis.

Analysis Beginnings The analysis phase is a critical part ofinstructional design. Its value, however, is not always apparent toclients, who may come to the table with requests for specifictraining solutions. Anderson (2010)

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

77chapter 3 Analysis Phase

identifies four reasons why needs analysis is imperative.3First, clients often believe they know the causes of organizationalproblems but misdefine or misdiagnose them. Thus, it is importantto gath-er evidence to determine whether the putative causes arethe real ones. Second, clients often ask for solutions that do notaddress the true problem. Part of this might be that the clientshave misiden-tified the problem, but clients may also have aninterest in a particular solution because it’s trendy or they haveseen it used effectively in other organizations even if thissolution isn’t what’s best for their specific needs. Third, timeand money used for needs analysis lead to interventions that aredesigned to correct the real problems, and clients’ time and moneyare thus recouped more effec-tively than if interventions areinappropriate for the client’s specific needs. Finally, analysisyields clear value-added results for clients. Analysis enables anorganization to know exactly how far it is falling short ofexpectations, so post-training results can be used to show clearlythe degree of im-provement in performance.

Four Stages of Needs AssessmentThe first element of analysis isto identify the needs of all parties involved in the trainingprocess. Without knowing what the actual needs of the organizationare, it is unlikely that effective training or other interventionscan be developed for performance improvement. The sections below(seen in Figure 3.2) briefly describe four critical needs,including business needs, performance needs, learn-ing needs, andlearner needs, before moving on to additional steps in the analysisprocess.

FIGURE 3.1 ADDIE Model

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

78 TRAINING and DEVELOPMENT

Business NeedsBusiness needs are sometimes overlooked whendesigning training programs but they are the most fundamental setof needs that must be considered. Business needs are the goals thatthe organiza-tion needs to attain to be successful; in fact,organizational viability might be threatened if they are not.Business needs span the entirety of an organization’s function,potentially including externally focused goals such as increasingmarket share or meeting particular sales targets and internallyfocused goals such as retaining and developing talent or becomingmore efficient in business pro-cesses. Quantitative goals andbenchmarking are often associated with business needs, providing auseful starting point for needs assessment as one asks: how closeis the organization to achieving those goals? What obstacles arepreventing the organization from attaining them?

When identifying business needs, it may be tempting to concludethat this is a simple task and that the business needs presentlystated in the organizational strategic plans or other documents arethose being actively pursued by the organization. However, businessneeds may change over time and even senior leadership may disagreeon how to interpret existing statements of organizational goals.Moreover, some business goals might not be in the best interest ofthe organization. It is in-cumbent on the training professional notonly to identify business needs, but also to make sure that seniorleadership has a shared understanding of those needs and that thoseneeds are consistent with where the organization wants to move inthe future.

FIGURE 3.2 Critical Needs to Be Assessed

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

79chapter 3 Analysis Phase

Performance NeedsPerformance needs focus on the behaviors thatemployees need to engage in so that business needs can be achieved.If, for example, reducing customer complaints by 10% is a businessgoal, what are the specific employee behaviors that are likely toreduce customer complaints? Identifying both business needs andpotential obstacles to achieving those needs are important stepsbefore iden-tifying performance needs. Trainers must understand theroot causes of organizational problems rather than simplyunderstanding where the organization wants to go, so thatperformance needs can be accurately identified. If an organizationor trainer simply says “We want to reduce customer complaints, soour employees need to be trained to be more effective on thetelephone,” without looking at the root causes of customercomplaints, training on telephone behavior will be ineffective ifthe complaints are due to unclear assembly instructions withproducts, unkept promises made by sales staff, or delays inshipping. Identifying the gap between performance behaviors thatneed to be enacted to achieve business goals and actual behaviorsthat are presently being performed allows a trainer to identifylearning needs.

Learning NeedsLearning needs focus on the knowledge, skills, andattitudes that need to be mastered in order to meet performanceneeds. Learning needs flow directly from business and performanceneeds by focusing on the gap between performance needs and whatindividuals in the organization are doing at present. Byidentifying both where gaps exist and the causes of such gaps, thetraining profes-sional can better understand specific types oftraining that may be needed. Even more importantly, understandingthe causes of performance gaps can also point out where training isunlikely to be an effective remedy. If performance gaps are causedby lack of trust, lack of motivation, inability to access necessaryequipment, or flawed awareness of business needs, then training islikely to be of little help.

Learner NeedsLearner needs include elements of both theworkplace environment and personal capacity that in-fluencelearners’ ability to enact desired behaviors. These needs can focuson how satisfied learners are with the training they receive, butthese needs are also sometimes beyond the scope of training itself,as noted in the section on learning needs above. Some learner needsare tangible, such as ac-cess to computing or other equipmentrequired to enact behaviors. As discussed in later chapters,particularly when discussing assessment of training and learningtransfer, support from manage-ment and peers is a critical learnerneed that must be fulfilled if training is to be effective.

Individual learner capacities are also important to the learningprocess. Existing skills and behaviors can be readily assessed todetermine the extent of the gap between current knowledge

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

80 TRAINING and DEVELOPMENT

and skills and what the organization needs. If the gap is small,training might focus on the specific behaviors associated withperformance needs. If the gap is large, training might first needto identify intermediate behaviors to be mastered before behaviorsassociated with performance needs can be developed. Additionally,individual learners have different levels of motivation andinterest in par-ticipating in training and learning new processes,skills, or knowledge. These differences also con-stitute learnerneeds, as training might be approached differently for individualswho are constantly seeking more challenging work environments orfor highly motivated groups as contrasted with individuals andgroups who may be less motivated, either because of personalfactors or because of issues within the work environment such aschronic turnover or lack of recognition.

Needs Assessment ProcessUnderstanding the four types of needsabove are central to the analysis phase of training develop-ment,so the needs assessment process focuses on how to go aboutcollecting information to identify those needs accurately so thatthe foundation for training design is solid.

Conduct External and Organizational ScanFactors both inside andoutside of the organization can influence both the need fortraining and how training activities might best be designed.External influences on the organization might in-clude marketconditions, including competition and demand for products, changesin workforce availability or preparedness, as well as governmentalor other types of regulation. Many organiza-tions that were majorcompetitors in their industries just a few decades ago have goneout of exis-tence or decreased in stature because they did notadapt to new products or ways of doing business while theircompetitors did. As some products are replaced by others (e.g., themarkets for photo developing services and video rental stores havevirtually evaporated over the past decade), organi-zations need tobe even more aware of current shifts in the external competitiveenvironment and work to forecast future trends accurately in orderto survive.

Beyond the specific competitive environment, the generaleconomic climate impacts interest rates, availability of investmentfunds, and customer willingness to purchase products; these factorscan influence the degree to which an organization can elect tochange or must change in order to maintain its viability.Relationships with external organizations, including suppliers,consumers, local partners, and others, can also have an impact onthe business needs of the organization and what perceived needs arepresent.

Equally important as a scan of the external environment is anorganizational scan. Issues within the organization and its culturesuch as existing business processes and behavioral expec-tationsfor organization members need to be identified. In the 1990s, thenow-defunct Saturn car company aired commercials promoting the ideathat all of their employees were empowered to ensure the quality ofthe automobiles they produced and showed employees stopping theauto as-

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

81chapter 3 Analysis Phase

sembly line when they spotted a problem. Saturn was making thepoint that it was an organization that promoted employee initiativeand joint responsibility for the products being produced;orga-nizational culture questions such as these are an importantpart of an internal organizational scan. Concerns such as frequentstaff turnover might contribute to a number of difficulties,including many organizational members who are new to theirpositions and who may not understand their role andresponsibilities, task overload because the workload of exitingstaff is simply added to those who remain, or difficulty indeveloping the necessary trust and relationships across theorganization to move forward on business needs.

A critical component of an internal scan is to identify not justsymptoms, but also root causes of gaps between performance needsand actual performance. Training professionals need to ask probingquestions about the nature of performance gaps, the conditionsunder which they occur, and why the gaps appear to be occurring.Multiple organizational members need to be queried, at multiplelevels of the organization, so that perceptions can be checked andpatterns in responses evaluated for consistency. Key questions thatshould be part of initial attempts to identify the root causes ofperformance gaps include:4

1. Is it a knowledge gap?2. Do employees have clear performanceexpectations?3. Do performance incentives align with goals?4. Arethere breakdowns in process?5. Is the employee in the correctassignment?

Collect Data to Identify Business NeedsIn some organizations,business needs may seem relatively easy to identify. Organizationalstrategic plans often include very tangible goals to be achievedover the course of the coming year or in the longer term. Existingdocuments such as mission and vision statements can also provideinsight into possible business needs and form the basis ofquestions to ask senior executives about related business needs. Itis important, however, to make sure that these documents are up todate and that there is agreement among senior leadership both aboutwhether these goals are still important to the organization andwhat each of these goals means.

During the initial stages of attempting to identify businessneeds, it is important to meet cli-ents where they are; some arefocused on actual business needs while others are focused on aspe-cific issue of concern or training desire. Stone (2009)identifies six situations that typically drive perceived trainingneeds: deficiencies in meeting business outcomes, deficiencies inhow employees are executing a specific job task, addition of newcompetencies or behaviors needed to perform new or existing tasks,changes in business processes that require employees to go abouttheir work in different ways, prospective changes in the market orbusiness opportunities that lead to proactive attempts to prepareemployees for change, and new processes driven by the need tocomply with regulatory or other external requirements.5 Determiningwhich of these factors are driving the

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

82 TRAINING and DEVELOPMENT

request for training is important in developing appropriatefollow up questions and to identify po-tential interventions,whether or not those interventions involve training.

Identify Possible InterventionsPossible interventions may take anumber of forms. The ATD identifies six potential categories ofinterventions that might address performance gaps:6

1. Workflow structure or processes: Interventions focus onremoving inefficiencies or redun-dancies in how people go aboutcompleting their work.

2. Resources: When lack of material resources or problems in thephysical environment (e.g., lack of temperature control orventilation) hamper performance, interventions focus on im-provingavailable resources.

3. Information resources: Interventions address deficiencies inhow information is shared or the timing or quality of sharedinformation.

4. Knowledge and skill deficits: When employees lack necessaryknowledge or skills to complete tasks or have difficulty with tasksequencing or improperly learned skills, interventions mightaddress these issues.

5. Motivation: Organizational members who feel devalued, do notreceive feedback or recogni-tion, are rewarded for inappropriatebehaviors, or are not committed to the organization may benefitfrom intervention.

6. Wellness: Interventions may target both physical capacity toperform the job and mental at-tentiveness that is unencumbered bystress from problems outside of work or work-related issues such astask overload.

Talking to a ClientClients are individuals who seek out andprovide the financial resources for training solutions and may beinternal to an organization or external to it. Often, clients seekout training professionals to address what they perceive as theorganization’s training needs. As noted several times in thischap-ter, it is important to distinguish between what a clientwants and what actions will yield the results the client seeks toachieve; these are not always the same.

It is important to recognize that clients may believe thattraining is needed when other factors, such as low levels ofemployee engagement or motivation, may be at the root ofperformance gaps. Even if you initially believe that training isnot the answer to the specific performance gap identified, youshould engage with the client to gain an understanding of his/herperceptions and why training is thought to be the answer. Thisinsight will prove useful both as you conduct your needs analysisand as you prepare a final report that provides potential solutionsto your client.

When talking with clients, it is important to understand wherethe client is in terms of un-derstanding what might be neededwithin the organization and to work with where the client is.

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

83chapter 3 Analysis Phase

For example, if a client believes that training is needed, thetraining professional can ask probing questions that include:What’s the basic need or problem that would be addressed bytraining? Why is this need important to the overall organizationalpicture? To what degree is this an ongoing prob-lem? Asking suchquestions helps you to gain a better understanding of what isreally important to the client, be it reducing cost overruns,enhancing customer satisfaction, or reducing accidents. Byunderstanding the fundamental need or goal, you can begin toidentify the kinds of information you’ll need to gather and whatthe ultimate goal of your needs analysis is.

Training vs. HPI —When Clients Don’t Know What They NeedNot allfailures to achieve organizational objectives can be solved bytraining. Lack of knowledge or skill can potentially be remedied bytraining, but many other sources of performance gaps, includ-inginadequate resources (including information resources),inappropriate personnel for the task, or inefficient businessprocesses, will likely not be solved by training. Trainingprofessionals are often called any time there is a perceivedperformance gap and clients may not recognize theinappropri-ateness of training solutions in every case.

In contrast to training, human performance improvement (HPI)focuses on performance, or results, by taking into account the fullrange of organizational factors that affect those results,in-cluding organizational culture, goals, environment, workflowprocesses, knowledge, and skills. 7 Specifically, performance ismaximized when the organization, the individuals within theorgani-zation, and the processes used to perform tasks align. Manyissues that senior leaders perceive as “training problems” arebetter suited for human performance improvement. One goal of theanaly-sis phase is to identify not just symptoms, but root causesof performance shortfalls, going beyond only considering trainingas a solution to identify ways in which all elements of HPI cancontribute to enhanced performance.

Collect Data to Identify Performance, Learning, and LearnerNeedsThe next step involved in a needs analysis involves collectingdata to identify performance, learning, and learner needs. Thissection examines three levels of analysis, the data collectionprocess, analyz-ing data, and using clear standards forevaluation.

Three Levels of AnalysisTraining and development professionalsgenerally look at analysis as three pieces of an important puzzle:organizational, employee, and task. For this reason, analyzing allthree of these factors is extremely important when conducting aneeds analysis.

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

84 TRAINING and DEVELOPMENT

Organizational

Organizational analysis is concerned with identifying bothpresent and future learning needs of the organization. Informationabout organizational needs can be obtained from evaluatingdocuments such as mission and vision statements and also fromasking questions or listening to conversations about where theorganization is headed. Major change initiatives are another sourceof information for organizational analysis: where does theorganization see itself going in the near future and what kinds oflearning needs will those changes entail?

Understanding why there is a perceived need for change, to beaddressed by training or other interventions, is also an importantcomponent of organizational analysis. What kinds of organiza-tionalconstraints are present? How urgent and critical are theperformance areas where change is sought?

Employee

Employee analysis centers on the individual performing the taskand addresses questions such as: How does the person go aboutperforming the task? How well does the person perform variousele-ments of the task? Employee analysis may include examination ofdocuments such as performance appraisals and observations of taskperformance. Effective employee analysis can also focus partic-ularattention on high performing employees to determine what differs intheir approach to the job as compared to others who do not performas well.

Task

Task analysis centers on identifying elements that are essentialto completing specific jobs or tasks. These include specific skillsor knowledge, tools or other physical resources, and propercondi-tions needed to complete the work. A comprehensive taskanalysis requires not only identifying and breaking down thebehaviors and skills needed to perform the task, but also whereemployees’ pres-ent skill sets can be improved. Task performanceconditions, including when and how tasks must be performed and thequantity and quality of performance desired, are also part of taskanalysis.

The Data Collection ProcessUnderstanding the variety oforganizational and individual needs discussed in previous sectionsis a precursor to the next major stage of analysis: collecting datafrom organizational members to understand the full picture ofmisalignment between needs and reality.

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

85chapter 3 Analysis Phase

Understanding Analysis

The actual analysis part of a needs analysis is usuallyconducted by the training project manager in conjunction with heror his client or relevant stakeholders. It’s important tounderstand that this type of assessment must be a group effort tobe effective. In the following few paragraphs, we explore thesubject of assessment, generating research questions, and selectingmethods for data collection.

Subject of Assessment

The training professional must fully understand the goal of datacollection and assessment. You should have a strong sense of theorganization’s needs and which groups or individuals within theorganization are directly affected by current performance gaps.Without this basic knowledge, rele-vant data may not be collectedand the resulting analysis is unlikely to meet the organization’sneeds.

Source of Data

In order to understand fully why an organization isn’t achievingits goals, multiple sources of data must be part of the analysisplan. Simply asking senior executives about their perceptions ofdefi-ciencies does not account for the experiences of those who areactually performing tasks across the organization. Similarly,focusing solely on a single unit within the organization or only onpoorly performing units also provides an incomplete picture of theorganization and where it stands in relation to meeting itsgoals.

An important element of effective data gathering is sampling.Good sampling means that the data can be trusted to represent theviews and experiences of individuals and teams across theor-ganization. Although interviewing every person within a complexorganization or surveying all employees might be possible, it isnot desirable in terms of time and other resource costs (includingslowing down the analysis significantly) or necessary if those whoare included in the data collection represent the entireorganizational membership. A few general rules of sampling shouldbe kept in mind. In general, large samples are preferable to smallones. Even time and resource intensive data gathering techniquessuch as interviews should not rely on conversations with, say, onlyfive of 75 sales professionals. If the group is too large or theresources too limited to include a relatively large sample, thenalternative data collections such as focus groups or surveys, whichcan include the voices and views of larger groups, should beconsidered. Random sampling, wherein each person within theorganization has an equal chance of being selected for input, ispreferable to simply find-ing volunteers or those who areconvenient to interview, such as people who are sitting in a breakroom or whose supervisor is especially supportive of the project(Figure 3.3). Depending on the nature of the organization,stratified sampling might be preferable to random sampling.Stratified sampling makes sure that potentially small subgroups arerepresented within the sample and is par-ticularly useful wheninformation needs to be examined by subgroup or if the researchquestions guiding the analysis have a disproportionate impact orfocus on a subset of employees.

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

86 TRAINING and DEVELOPMENT

Standard of Evaluation

Data sources for assessment of the organization should includeboth objective and subjective sourc-es in order to obtain thefullest picture of how performance impacts the organization (e.g.,pro-ductivity, percentage of orders filled incorrectly) and theindividuals within the organization (e.g., perceptions oforganizational culture).

Objective data sources include those that are quantifiable.Objective information can be de-rived from sources such asperformance data and observations. Objective data are especiallyuseful for diagnosing the extent or frequency of a problem. Forexample, do accidents or quality control issues spike at particulartimes of the week or month? Are customer complaints general orfocused on particular product lines or sales accounts?

Subjective data sources allow for inclusion of perceptions andattitudes into the analysis pro-cess. Subjective information isobtained by surveys, interviews, or focus groups, where individualscan discuss their views on the current state of the organizationand barriers to reaching organiza-tional goals. Subjectiveinformation sources also allow the training professional to get asense of individuals’ thought processes as they perform their jobsor how different people across the organi-zation view the causesand symptoms of performance gaps.

Generate a Research Question

Before developing a plan for collecting data, you need tounderstand what question or questions you want to answer. Questionsmay be reactive or proactive.8 Reactive questions develop inresponse to an identified performance gap within an organization:sales goals have not been met, complaints have increased, oraccidents have injured organizational members. Proactive questionsare based in

FIGURE 3.3 Types of Random Sampling

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

87chapter 3 Analysis Phase

anticipated changes for an organization. For example, newtechnologies might be on the horizon that shift how theorganization interacts with customers or develops products.

Anderson identified several key qualities of good researchquestions.9 First, questions must be worth assessing from theviewpoint of the client. Conducting a thorough analysis thatidentifies key business needs and performance gaps can assist inmaking sure that the question is directly tied to organizationalneeds. Related to worth is clarity; using organizational languagethat is familiar to the client is helpful in making the questionclear so that both the client and training professional know whatto expect and so that organizational members can provide usefulinformation to answer the question. Additionally, questions need tobe answerable or measurable. Without this quality, no interventionwill be able to provide concrete results, making it difficult, ifnot impossible, to evaluate the effectiveness of anyinterventions.

Questions guiding data collection must be thoughtful so thatgood quality data result. Particularly when senior leaders areskeptical about the need for analysis, it is important to generateuseful informa-tion about organizational needs so that sufficientsupport is available for the interventions you propose.

Select Methods for Collecting Data

Although a range of methods for collecting data are available,not all methods are equally appropri-ate or necessary in allsituations. Part of the analysis process is determining whichcombination of data collection methods makes most sense for thecurrent situation.

Anderson argues that data collection methods should balanceutility, feasibility, propriety, and accuracy. This means thattraining professionals need to consider how useful the informationgath-ered from each collection method will be, how time andresource intensive it is to gather each type of information, towhat degree it is appropriate to gather particular types ofinformation, and how well the information collected represents theactual state of performance.

General issues to consider when deciding which types of data tocollect include the nature of the organization and its members,including organizational climate, background and education ofemployees, and specialized skills needed to score or interpret sometypes of data. If an organization has a history of being highlysecretive or not responding to needs expressed in employee surveys,then organizational members may be less likely to respond tosurveys or may be distrustful of inter-view or focus groupquestions. The organizational, employee, and task analysesconducted before this stage are important sources of information sothat you can identify data collection methods that are likely to bemore or less effective.

Surveys

Surveys are typically written sets of questions that areprovided to potential respondents in paper or electronic form.Surveys may include both closed-ended questions, or those that havea single specific response or selection from a pre-determined setof responses, and open-ended questions, which allow

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

88 TRAINING and DEVELOPMENT

the respondent more flexibility in providing an answer. Surveysare specifically designed to obtain data from large numbers ofpersons relatively quickly and inexpensively. Advantages of usingsurveys include the ease of response for participants, ease oftallying results for those administering the survey, and thepresence of quantifiable responses from the data. Disadvantagesinclude the importance of creating clear questions that address thekey issues in which you are interested. Because respondents can’ttypically ask for clarification of survey questions andadministrators can’t ask follow up questions, it is critical forquestions to be thoughtfully constructed and address all relevanttopics of interest. Ul-timately, you need to consider the realpurpose for your survey (see Figure 3.4).

Answer scale and type are also important considerations withsurveys. Open-ended respons-es may provide richer information butalso require some means of coding or grouping responses.Likert-type response scales, such as those which ask respondentsfor ratings on a one to five scale, also have drawbacks. If usingan odd number of response choices, does a midpoint rating indicateno opinion, neutrality, or lack of relevance? It is also easy fortired or distracted respondents to pro-vide a single response toall scale items (e.g., all are excellent or all are poor) withoutgiving much thought to the specific questions being asked. Forsurvey results to be valid, a large sample size is also necessaryand many requests for responses to surveys go ignored. Thus, evenwhen sending a survey to all 200 employees in an organization, youmay obtain only 20 or 30 responses, which is not enough to drawgeneralizable conclusions about the entire group.

FIGURE 3.4 Questions to Ask in Designing a Survey

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

89chapter 3 Analysis Phase

Interviews

Interviews are one-on-one interactions where a trainer ordesignee asks specific questions of a single organizational member.Interviews are particularly useful in fleshing out responsesobtained from a survey or other quantitative data and are typicallyrich in detail. Interviews thus allow an-swers to questions suchas: Why do people perform tasks in these ways? or What are thesources of anxiety or motivation for employees? Unfortunately,interviews are also time consuming for those who administer themand those who respond. For maximum effectiveness, interviewersshould enter the interview with some number of common questions tobe asked in all interviews. Samples for interviews, as withsurveys, must be selected with care. If only highly motivated orhighly dis-gruntled or novice employees are included, the resultingpicture of the organization and its needs is likely to be skewed.It’s also important to note with interviews that knowing thatevents or issues occur frequently is different from understandingthe reasons why those issues occur.

Focus Groups

Focus groups take the idea of face-to-face interaction from theinterview and apply it to settings where there is one facilitatoror interviewer and a group of respondents. Focus groups areespecially useful when trying to develop hypotheses to be tested inlarger groups (e.g., when trying to find out what issues might beof greatest importance and should be addressed in a survey).Nonverbal behaviors of respondents can also be observed in a focusgroup, providing additional information or impetus to ask follow upquestions to a training professional who is aware of thesebehaviors. As with interviews, focus groups can be time andresource intensive. It is generally preferable for one individualto facilitate the group’s discussion of questions and topics andanother, different person to record responses from the group.Another significant concern is that focus groups can be rathereasily dominated by a few highly vocal members of the group, whoseviewpoints may then be per-ceived as representing the whole groupor who may take the group on tangents that are of limitedusefulness.

Observations

Observations occur when a training professional or otherdesignee watches and listens to organiza-tional members as theyperform a specific behavior or job task. Observations are apotentially good source of information for physical tasks asphysical activities can be observed more readily than cognitiveprocesses. Observations can also be used as the basis for creatingflow charts or process maps so that all stages of behavioralperformance are identified. Observing performance within itsnatural environment can also help in identifying environmentalfactors that help or hinder task performance, including conditionsof the workspace or equipment.

One drawback to observations is that observers may not be ableto determine the thought pro-cesses behind behaviors. A specificbehavior might not be part of the standard process for a jobtask,

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

90 TRAINING and DEVELOPMENT

but an individual employee might complete steps in a differentorder or change the process to meet personal needs or because of amisunderstanding of the process. A second drawback is thatindivid-uals sometimes change their behaviors when they are beingwatched. This phenomenon, known as the Hawthorne effect, makes thedata collected via observations less accurate as a representationof how people actually perform tasks.

Tests

For less observable tasks, or cases where specific knowledge ispotentially at the root of a perfor-mance gap, testing is an optionfor collecting data. Testing involves providing a written orapplied evaluation of what a person knows or can do. Its advantagesinclude that it is objective and allows administrators to focus onspecific gaps. As with observations, however, tests do not enablecollec-tion of information about why people do what they do. Testanxiety is also a concern; similarly to the Hawthorne effect inobservations, some individuals do not respond well when they knowthey are being tested.

Performance Data

Performance data are existing or readily collectible informationabout how well specific tasks or processes are performed. Examplesof performance data include quality control reports, injury orcomplaint data, inventories of scrap or damaged materials, andoutput quantities. Performance data can readily show where thereare gaps between desired and actual performance and can help toquantify the extent of such gaps. Performance data by themselves,however, can be influenced by unknown external factors, which mayprovide an inaccurate sense of what actual performance gaps are.For example, quality control problems during a period of time mightbe the result of switching to a different supplier and lowerquality materials instead of a problem with the specific taskperfor-mance within the organization.

Analyze DataAfter data are collected, they must be analyzedeffectively so that accurate conclusions can be drawn aboutexisting areas of deficiency and potential interventions can bedesigned. It is critically import-ant that data analysis isaccurate, as faulty data can lead to lack of confidence in theanalysis and lack of support for the needs assessment.

Using Your Evaluation Standards

When analyzing data, the first step is to put the information ina useful form, if it is not already in useful form. Responses toclosed-ended questions from surveys or observation reports can been-

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

91chapter 3 Analysis Phase

tered into a spreadsheet for ease of interpretation andanalysis. Responses to open-ended questions, including those frominterviews or focus groups, must be analyzed to determine recurringthemes or patterns.

Data should also be placed within a larger context. If benchmarkinformation is available from comparable organizations, bestpractices are identified by professional organizations or otherresearch, or legal requirements provide guidance about performancestandards, these should be included as criteria for interpretingthe data.10 Doing so not only provides a deeper understanding ofthe data col-lected but can also be used in demonstrating toclients the full extent of necessary changes.

It is also important during this phase to avoidmisinterpretation of data. Several common shortcomings of dataanalysis are presented by Anderson:11

•Avoidconfusingperceptionsandreality,asstatementsarenotalwaysthesameasfacts.•Besuretoconsideralternativeexplanationsorinterpretationsofinformation.•Becarefulaboutgeneralizingwhensamplesizesaresmallandthenumberofobservationsor

survey responses are small, as the sample may not represent theentire organization.•Avoidconfusingthefrequencywithwhichaneventorobstacleisreportedandtheimpor-

tance of the event or obstacle.•Bealertforunconsciousbiasonthepartofresearchersoranypersoninvolvedinthedata

collection process.

See Chapter 10 for Statistics for WLP Professionals

Particularly if you have quantitative data from sources such assurveys, tests, or performance re-ports, you will likely wish toanalyze the data to determine if there are differences betweengroups in performance and to see if there are relationships betweenparticular environmental or other factors and performance. Forexample, it may be that second shift and first shift workers havedifferent rates of output or accidents. It might also be that morequality assurance issues occur earlier in the week and, as the weekprogresses, fewer issues occur. Identifying such patterns in thedata may suggest specific sources of problems or models of goodperformance that can be further investigated and shared across theorganization.

Deliver Data Analysis and FeedbackOnce data have been analyzed,the entire needs analysis must be summarized and presented tose-nior executives within the organization. By presenting theelements below, senior executives should have sufficientinformation, provided within the organizational context, toevaluate the proposed action plan. As you develop your report, youneed to consider who your audience is and what types of informationthey will need to support your recommendations and to decidewhether to move

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

92 TRAINING and DEVELOPMENT

ahead with them. The information in the following section can beread in conjunction with Appen-dix A, which is a brief introductionto project management for T&D professionals.

Executive SummaryIt is typically useful to begin your datareport with an executive summary, or a relatively short overview ofthe report. Executive summaries can serve at least two functions.First, they can orient readers to the main issues addressed in thereport, assisting with comprehension and potentially directing themto specific sections. Second, for readers who may not be directlyresponsible for the areas addressed in the analysis but who need toknow something about the report, an executive summary is a means ofconveying the key points without reading the entire report.

Organizational OverviewAn organizational overview addresses thecurrent state of affairs within the organization, includingbusiness needs, where the organization is presently, and the scopeof the current analysis. Existing employee perceptions oforganizational culture, ongoing relationships with vendors or otherexter-nal parties, and recent financial performance are examples offactors that influence the organiza-tion’s current state ofaffairs.

Providing information about business needs and where theorganization currently stands may help to build credibility withthe client and clearly articulating the scope of the analysis willhopeful-ly ensure that the organization and the trainingprofessional have similar expectations about what issues anyproposed actions are meant to address.

Of course, a central component of the organizational overviewincludes the specific results of your analysis and the data youcollected. Information should be provided about how you went aboutgathering data, the sources of information you examined, andnarrative and/or visual presentations of results. Be sure to keepin mind that not all readers will be interested in a highlydetailed version of your results. Understanding your targetaudience is important so that you know how much detail to include.If, as is likely the case, you have individuals within your targetaudience who want little detail and others who want very detailedinformation about your data analyses, it may be best to include aless-detailed presentation within the body of the report and a moredetailed appendix at the end of the report that is available forthose who are interested in seeing the more detailedinformation.

Potential solutions to organizational performance gaps should betied to specific data from the analysis so that it’s clear toclients how the solutions will address the concerns raised by thedata. Both root causes and potential solutions can also beprioritized so that clients can readily assess which solutions arelikely to yield the largest or most immediate benefit to theorganization.

Whereas an organizational overview focuses primarily on factorsrelated to an organization’s success where the organization mightexert at least some control, the next section, the business en-

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

93chapter 3 Analysis Phase

vironment, focuses more on external factors that might be wellbeyond the control of anyone within the organization.

Business EnvironmentA description of the business environment,or factors outside the organization that may influenceorganizational performance, is also an important element ofanalysis. Changes in the range or types of competitors within theindustry, new or existing government regulations or reporting, andthe state of the general economy, which can affect the numbers andtypes of prospective employees available, are all examples ofelements in the business environment.

External constituencies that are specific to the organizationand their needs should also be part of the business environmentanalysis. Current and future status of donors, customer markets,and suppliers can be addressed. Trend information can beparticularly powerful and persuasive in this part of the analysis,as it can provide clarity about patterns of increased costs orturnover among suppliers, potential areas for growth or decline incustomer bases, or changes in the number or fi-nancial capacity ofdonors.

Financial AnalysisPart of your needs assessment report shouldfocus on the financial elements associated with imple-mentingproposed solutions and those associated with taking no action. Animportant element of a financial analysis is to specify how much itcosts the organization to continue with the current state ofaffairs. For example, if 12% of customers are returning merchandisebecause it is defective, the cost of the current performance gapincludes direct costs associated with processing, shipping, andhandling the defective merchandise and its replacements; the costsof personnel time to perform all of the tasks above; and costsassociated with loss of repeat business from frustrated customers.Cost information may be available within the organization orpotentially from industry benchmarks for the typical cost ofparticular job tasks. Analyzing the cost of retaining the statusquo can be power-ful incentive for organizational leaders to worktoward eliminating or lessening unnecessary costs.

Of course, costs of implementing proposed solutions must also beincluded in a financial analysis. Beyond the costs associated withdeveloping and delivering training materials or other solutions,personnel costs associated with attending training, monitoringprogress, and assessing success must also be part of the equation.If a solution requires six months to implement, the costs ofmaintaining the status quo, described above, will also continue foranother six months.

Finally, a financial analysis should address measurable gainsthat will accrue from implemen-tation of proposed actions.Continuing the example above, if customer returns are reduced to2%, what is the resulting cost savings? This part of the analysisincludes providing realistic expectations for what is achievablefrom the suggested solutions. It may be impossible to move to 0%returned

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

94 TRAINING and DEVELOPMENT

items, but your analysis should provide information aboutwhether 3% or 5% is a reasonable target. It may also be the casethat customer returns might be reducible to 7% in the first yearbut are pro-jected to drop to 3% in the second. Whatever theprojected scenarios are, they need to be supported by the analysisdata and included in the financial analysis of implementing theproposed solutions.

Action PlanThe action plan focuses on specific recommendationsfor moving the organization from its present state to desiredlevels of goal attainment. It should include specific activities tobe completed, an explanation of how those actions link to goals,desired performance benchmarks, methods of as-sessing success, anda timeline for completion. As noted above, a detailed budget thatoutlines the cost of activities undertaken should also be included.Responsible parties for each part of the action plan, from designto follow up and data analysis, also need to be outlined in thissection.

Specificity in all elements of the action plan is critical sothat the client has a complete picture of what will be needed toachieve the desired outcomes and to mitigate misunderstandingsabout the scope of the project or the results that are possible.Including responsible parties for each part of the action plan alsoaids in ensuring follow up and reminds the client that support andfeedback for behavioral changes are essential if the proposedsolutions are to yield the desired results.

Transition to Design Once the client decides to move forwardwith some or all of the recommendations within the action plan, thefocus shifts to design. At this point, a formal agreement on goalsto be achieved, scope of activities to be completed, indicators andmethods of measuring success, a list of tangible training products,and a timetable for project completion and follow up should bedeveloped.

SummaryA thorough needs analysis is essential to effectivetraining and organizational development pro-cesses. Althoughclients may not always initially see the benefits of such analysisor may incorrectly identify the organization’s needs, conducting acomplete analysis helps you develop a plan of action that is likelyto be successful in meeting the organization’s needs and inpersuading organizational leaders that proposed solutions willindeed be successful.

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

95chapter 3 Analysis Phase

Chapter Takeaways •Needsanalysisaddressesbusiness,performance,learning,andlearnerneedsandrequiresin-

put from multiple organizational members across theorganization.•Datacollectionmethodsallhavebenefitsanddrawbacks,sothemosteffectiveanalysiswill

tailor the needs to specific organizational questions andtypically include multiple informa-tion sources and methods.

•Notallperformancegapsaretheresultofpoorornon-existenttraining,soitisimportanttoidentifyroot causes of performance gaps before moving forward to propose orimplement potential solutions.

Chapter 3 ExercisesDiscussion Questions

1. Reviewing the list of benefits and drawbacks of specific datacollection methods (survey, in-terview, focus group, performanceassessment, tests), identify organizational characteristics orelements of organizational context that would make using particularmethods highly desir-able or undesirable.

2. Discuss which of the ATD categories of interventions toaddress performance gaps discussed in this chapter might be mostreadily addressed within organizations and which ones are like-lyto be more difficult to address.

Exercises1. You have an initial meeting with a potential clientand she says “Our employees need to do

a better job at working efficiently.” Develop a list ofquestions you might ask to gain a clearer sense of the performanceneeds within the organization. Share your list with a partner whohas completed this exercise and discuss why the questions on yourlist would be helpful in gaining the information you need.

2. With 2–3 classmates, discuss specific strategies forpresenting analysis results that a client may not want to hear foreach of the situations below:•Theclienthasidentifiedpoorperformanceamonglineworkersasthecauseofproduc-

tivity problems but your analysis suggests the problem lies withineffective management in the department and the manager is arelative of the client.

•Youhaveidentifiednewtechnologiesthathavechangedthecompetitiveenvironment,

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

96 TRAINING and DEVELOPMENT

and which have been adopted by competitors, as a significantreason why business needs are not being met but your client hasrepeatedly stated that investing in new technolo-gies is not apriority for the company.

Key TermsBusiness needs Goals or objectives that theorganization wishes to achieve.Employee analysis Identifying howand how proficiently people perform behaviors necessary to

meet organizational needs.Executive summary A brief summary ofthe main topics and conclusions contained in a report.Focus groupData collection method where a group of people are directly askedquestions with a

goal of gaining information about the group’s collectiveexperiences.Instructional systems design (ISD) The practice ofusing consistent and reliable procedures while

creating instructional experiences.Interview Data collectionmethod where a single person is directly asked questions with agoal of

gaining detailed information about the experiences of a singleperson.Learner needs Individual factors that impact learners’ability to enact necessary performance be-

haviors. Learning needs The gap between behaviors that need tooccur for business needs to be met and the

specific behaviors that organizational members are currentlyenacting.Objective data sources Sources of information that can bequantified.Observation Data collection method where a researcher orother observer watches and records

information about task performance in real time.Organizationalanalysis Identifying an organization’s present and future learningneeds.Organizational scan Identifying issues within theorganizational culture, including business pro-

cesses and behavioral expectations, that contribute to effectiveor ineffective performance.Performance data Data sources thatprovide information about aggregate or individual perfor-

mance patterns over time.Performance needs Specific behaviorsthat organizational members need to enact so that business

needs can be met.Subjective data sources Sources of informationthat cannot be quantified.Survey Data collection method thatprovides written questions to a large number of respondents

with a goal of aggregating the responses received.Task analysisIdentifying the behaviors or elements necessary to complete a taskeffectively.Test Performance assessment that evaluates anindividual’s knowledge or skill by simulation, writ-

ten or oral responses, or other means.

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

97chapter 3 Analysis Phase

References1 Rothwell, W. J., & Kazanas, H. C. (2009).Mastering the instructional design process: A systematic ap-

proach (Kindle Locations 812-817). San Francisco, CA: Wiley.2Branson, R. K., Rayner, G. T., Cox, J. L., Furman, J. P., King, F.J., & Hannum, W. H. (1975, August).

Interservice procedures for instructional systems development.(5 vols.) (TRADOC Pam 350-30 & NAVEDTRA 106A). Ft. Monroe, VA:U.S. to Army Training and Doctrine Command, (NTIS No. ADA 019 486through ADA 019 490).

3 Anderson, J. H. (2010). Collecting analysis data. In K. H.Silber & W. R. Foshey (Eds.) Handbook of im-proving performancein the workplace: Volume 1 Instructional design and trainingdelivery (pp. 95-143). San Francisco: Pfeiffer.

4 Jaenke, R. (2013, August). Identify the real reasons behindperformance gaps. T+D, 67, 76.5 Stone, R. D. (2009). Aligningtraining for results: A process and tools that link training tobusiness. San

Francisco: Pfeiffer.6 The ASTD Learning System. (2013).Alexandria, VA: ASTD Press.7 Dent, J., & Anderson, P. (2000).Info-line: Fundamentals of HPI. Issue 9811. Alexandria, VA:ASTD

Press.8 Anderson, (2010), op. cit. 9 Ibid.10 Ibid.11 Ibid.

Chapter 3: Analysis Phase from Training and Development: TheIntersection of Communication and Talent Development by JasonWrench, Danette Jorinson, and Maryalice Citera | 978-1-4652-6585-2| Copyright 2015

Property of Kendall Hunt Publishing

WrenchFMWrenchCh01WrenchCh02WrenchCh03WrenchCh04WrenchCh05WrenchCh06WrenchCh07WrenchCh08WrenchCh09WrenchCh10WrenchCh11WrenchCh12WrenchCh13WrenchCh14WrenchCh15WrenchCh16-AppWrenchCh17-AppBWrenchCh18-Index