| Designing and using a set of reliable and valid | | | | as a "concurrent validation" process - i.e., assessing |
| leadership competencies is critical for organizations | | | | existing leaders on the new competencies and |
| seeking to go from point A to point B. Leadership | | | | comparing those scores against their performance |
| competencies are an ideal profile specifying the | | | | evaluations. The most objective was to do this is |
| types of skills, knowledge, and behavioral traits | | | | through a 360 degree assessment - i.e., obtaining |
| you want your leaders to possess, and therefore, | | | | judgments from subordinates, peers, and |
| exhibit. Leadership competencies can used as part | | | | supervisors. In this way, an objective scores on |
| of a broader talent management system and | | | | each of the competency is obtained without the |
| succession planning, for leadership identification and | | | | inflated scores often seen with self-rated |
| development purposes, and/or as part of a | | | | assessments. After some reliability checks and |
| performance management system - e.g., via a | | | | data cleaning, the aggregated scores are |
| 360 degree feedback. | | | | compared against the same leaders' past |
| Despite the prohibitive cost associated with its | | | | performance scores. If the set of competencies |
| design and use - anywhere from say, $75,000 to | | | | deemed important for an organization are, in fact, |
| $400,000, depending on the size and scope of the | | | | what the organization values, then the correlation |
| project - many organizations fall short of the | | | | between those same competencies and |
| reliability and validity requirements set forth by | | | | performance ratings should be positive. This is |
| such governing bodies as APA (American | | | | called "concurrent validation." |
| Psychological Association) and SIOP (Society for | | | | 4. Validation against future performance - In |
| Industrial-Organizational Psychologists). As a | | | | addition to validating against current performance |
| psychometrician and a consultant, I have found | | | | levels, the same competencies should be |
| the following five steps to be common mistakes | | | | compared against performance at some point in |
| in leadership competency design. | | | | the future - e.g., 9 to 12 months. This technique, |
| 1. Starting from the top - Many believe (HR/OD | | | | known as "predictive validation," further ensures |
| professionals and consultants alike) that there are | | | | that the set of competencies are, in fact, |
| specific leadership competencies that are fixed - | | | | deemed important and that leaders are being |
| meaning that if you find leaders possessing certain | | | | assessed - at least partially - on those |
| qualities (critical thinking skills, strategic orientation, | | | | competencies. Unfortunately, this step is rarely |
| market awareness etc.), you can bring him/her | | | | done in organizations. **Note: Although unlikely, it |
| onboard and he/she will bring out the magic. This | | | | is possible that performance management system |
| is simply not true. Given the large number (in the | | | | is entirely different, and event, opposed to the |
| hundreds) and variations in competencies | | | | set of competencies being assessed. In such a |
| purported to be important, it is highly unlikely that | | | | case, the performance appraisal system needs to |
| one set of competencies apply to all organizations. | | | | be refined to be aligned with the competencies. |
| As with Olympic athletes, while one generally | | | | 5. Setting competency cut-off scores or |
| needs to be "fit," this does not guarantee a gold | | | | categories - Finally, once data have been obtained |
| medal in every event. In other words, like | | | | on the competencies from each leader, there is a |
| athletes, different organization need different sets | | | | need classify individuals according to their scores |
| of competencies in order to optimize their | | | | on each of the competencies. While it may be |
| competitive edge. This means starting from the | | | | tempting to choose an arbitrary category, this |
| top. A solid set of competency, therefore, is | | | | may lead to classifying most, if not all, leaders as |
| modeled after what an organization needs; not | | | | needing improvement. Thus, it is better to allow |
| something offered "off-the-shelf." This step | | | | the data to choose the size of the difference that |
| ensures that you identify precisely those skills you | | | | is meaningful to the population being tested. This |
| feel is "necessary" for your organization to be | | | | approach, known as the "data-driven" method, is |
| successful. | | | | done by using the "standard deviation" (or |
| 2. Validating with existing leaders - Once you have | | | | average difference) of the competency scores. |
| a set of competencies deemed important, many | | | | This approach ensures that the different score |
| organizations fail to validate the competencies | | | | categories are based on the population at hand |
| against incumbents. It is critical to check the | | | | (each organization will differ on what this deviation |
| competencies with existing leaders to obtain their | | | | score is) and is a known "legally defensible" |
| thoughts on how important each competency | | | | strategy. |
| (and related questions) is. This step is important | | | | In sum, there are a number of common mistakes |
| for two reasons: First, it legitimizes the | | | | (mostly by omission) that occur in the |
| competencies in the eyes of those who are being | | | | development of leadership competencies. While |
| assessed - i.e., the leaders themselves; thus, | | | | the above steps may seem overly rigorous, |
| getting them to buy-in on the idea. Second, it | | | | there are ways to incorporate them without |
| satisfies the "face validity" requirement set forth | | | | making the process overly daunting. The result is |
| in the major governing bodies. In other words, if I | | | | a set of competencies that: |
| am design a test to select competent mechanics, | | | | (a) will be viewed by incumbents as legitimate, |
| I need to run the test by real mechanics in order | | | | (b) demonstrates proof that it is measuring what |
| to be sure that the test is measuring what it is | | | | it is supposed to be measuring, and |
| intending to measure. | | | | (c) ultimately allow you to place complete |
| 3. Validating against actual performance - Even if | | | | confidence in the intervention stemming from its |
| organizations are savvy enough to complete the | | | | use. |
| first two steps, many skip over what is known | | | | Copyright 2010 by Stephen B. Jeong, Ph.D. |