Criterion Development: The Unknown Power of Criteria as ...

By Tommy Cunningham,2014-04-29 19:06
12 views 0
Criterion Development: The Unknown Power of Criteria as ...The,the,THE

287 Wieby M.M. Altink, Coert F. Visser and Michiel Castelijns

    Chapter 14

    Criterion Development: The Unknown

    Power of Criteria as Communication Tools


    Adviesbureau Psychotechniek Utrecht B.V., Utrecht, The Netherlands


    Much has been written about the development of criteria: the standards against which personnel are hired and appraised. We know that criteria should reflect the relevant tasks and responsibilities in the job and organization, that they should meet certain psychometric standards and, moreover, that they should not result in unfair discrimination (against races, gender, etc.). Many tools have been developed to derive criteria that fulfil these purposes. However, less attention has been paid to the process by which criteria are formulated. This process offer many opportunities to communicate with organization members about the goals of the organization and the required and desired performance (see also Anderson and Ostr0ff, Chapter 20 in this volume).

    In our opinion, criterion development is not just a manner of applying a psychometric ‘recipe’, as many textbooks still seem to suggest. Criterion development is an activity by which important aspects of behaviors, competencies, and achievements are communicated throughout the organization. Should not all organization members (employees, managers, personnel officers, consultants) work with these important issues that motivate and direct performances within organizations? Why not then pay more attention to their role in the process of criterion development?

    This chapter gives some examples that, we hope, trigger ways of thinking about how to develop criteria and the opportunities to involve organization members in this process. You might achieve the same result that a manager of one of our cliënt organizations described in the following way: ‘Now I understand why I should discuss these measures with the members of my team they simply reflect the goals to which we are heading!”


     International Handbook of Selection and Assessment, Edited by N. Anderson and P. Herriot ? 1977 John Wiley & Sons Ltd.

288 Wieby M.M. Altink, Coert F. Visser and Michiel Castelijns

    The question ‘Why should we develop criteria?’ seems redundant in the case of selection and appraisal procedures. Criteria, or concrete measures of performance, define how a candidate fulfils and in the case of selection- will perform in a certain job. It is a know fact: criteria play a double role. They formulate in a specific manner what individuals have to do, and they also give standards against which we can evaluate whether these goals have been reached. Criterion development follows upon the process of job analysis. First, we state which tasks have to be performed, specify the necessary skills hereto and then we construct specific standards that are necessary to measure, lead, coach and evaluate performance. To give an idea of what criteria are, we show some examples in Figure 14.1. These examples, which we encountered in our practice as external human resource management consultants, show in addition that we sometimes have to deal with behaviors and performances that are not easily described, measured or predicted. Criteria might be formulated on a more abstract or concrete level. They may sometimes concern tasks, competencies, behaviors or capacities and personality requirements. The level of abstraction and the construct to which criteria refer depend on the specific purpose for which criteria are developed and also on several psychometric considerations. However, a central feature of criterion constructs is that they can be transformed into observable skills, behaviors and habits, which can be communicated about with employees to observe, predict and evaluate performance and (future) achievements.



    Many organizations are experiencing drastic changes in their environments. Technological developments make new products possible. As a consequence of the increasing speed of technological developments product cycles are getting shorter and shorter. New markets are emerging as well as new customer needs and demands. Unfortunately, it is no longer sufficient to be competitive in one’s own national market; world class competitiveness is the new aim in order to achieve sustained success. Workers today are generally more highly skilled and better educated than before. They put different demands on organizations. They have less tolerance for hierarchy, they demand high quality of work and the opportunity to develop themselves, to have a meaningful role and to enjoy their work.

     International Handbook of Selection and Assessment, Edited by N. Anderson and P. Herriot ? 1977 John Wiley & Sons Ltd.

289 Wieby M.M. Altink, Coert F. Visser and Michiel Castelijns

Criterion Description

    Productivity Products made/delivered within a certain time span

    Oral communication Messages communicated to others in a clear and understandable way

    Cliënt service Number of complaints by clients; acquired contacts within a certain time span

    People management Motivation/productivity of personnel, creation of ‘team spirit’, skills that relate

    to enhancing individual and team performance.

Figure 14.1 Examples of criterion constructs and descriptions

     Organizations have to be able to adapt to and to cope with the new organizational environments and market demands, which are getting stricter and different. Goals which many organizations set have to do with achieving higher quality, higher speed, lower costs, dependability and adaptiveness (Flood, Gannon and Paauwe, 1995). Finding the right sources of competitive advantage is an important quest. Nowadays, human resources management (HRM) is seen as probably the major source of competitiveness (Pfeffer, 1994; Flood Gannon, and Paauwe, 1995). Human resources are valuable, scarce, difficult to copy and difficult to substitute. Also there are several sources of competitive parity. They imply developments that

    organizations have to keep up with in order to keep up with their competitors. Examples are applying new technologies, flattening organization structures and making processes more simple and flexible.

    Several drastic changes in work are occurring related to the above-mentioned changes. There are changes related to technological developments. The work place of today is getting informated. New tasks emerge in the application of new technology as well. However, there is also a process going on of sub-tasks, tasks, and jobs disappearing. Knowledge-intensive applications like computers and robots replace human work to some extent. Here there is a parallel with Fordism, the mechanization of work. Also emerging are new ways of

    communicating. Of course there is the ‘electronic highway’. People can communicate easily with each other even though they may be far apart. Teleworking is also a new possibility. The need for different ways to management people leads to major changes in organizations too. Human resources are seen as a major factor for achieving company success. Organizations put more emphasis than before on developing a strategic HRM. Making sure that the work force is valuable is a major concern. The development of individuals within organization is too. People demand development opportunities on the one hand; they can be made more valuable for organizations on the other hand. Another way of improving HRM practices is to improve the consistency of different elements, like selecting people, appraising, developing and rewarding them. Line managers are being given the responsibility for these HRM tasks. Topics that get much attention are the flexible work force, broader tasks, more investment in training and development, self-management, and empowerment, management by objectives, ouput-oriented appraisal, flexibility in rewards and raising the internal and external mobility of employees.

     International Handbook of Selection and Assessment, Edited by N. Anderson and P. Herriot ? 1977 John Wiley & Sons Ltd.

290 Wieby M.M. Altink, Coert F. Visser and Michiel Castelijns

    There are also some structural changes in the way work is designed. First, there is a process of revision of labor division and simplification of work processes going on (Visser, Altink and Algera, Chapter 21 in this volume). Secondly, there is a tendency to cluster people into teams. Teams can be more flexible than individuals and are more capable of dealing with complex problems. Many organizations throughout the world are experimenting with self-managing teams (Wellins, Byham and Dixon, 1994). Temporary assignments and project work is another trend. To be very flexible and get fast results people from different disciplines work together with a specific assignment until the desired results have been achieved.

    As a result of all these changes, there are some new demands on people. High priority is put on aspects like knowledge and the permanent development of knowledge. A bigger emphasis

    on intellectual skills and capabilities will be inevitable. Adaptiveness is another criterion, which

    will be relevant due to changing work content and contexts. People have to be willing and capable of broadening their skills. Cooperativeness is a requirement because of the emphasis on

    teamwork. Workers have to be ready to commit and adapt to team goals. On the other hand, people have to be able to work independently. Employees have to be self-managing. They are

    responsible for achieving the results they have committed themselves to. A more intensive co-operation and new ways of communicating places higher demands on communicative skills. To

    be more self-managing, mores knowledge about business processes is required. Orientation toward

    adding value to organizational goals as well as an orientation toward innovation will be

    necessary to meet new market demands.

    These developments clearly make the use of criteria important.


    Because of their importance, work and organization psychologists have paid much attention to the development of criteria throughout the years. Algera and Greuter (1989) discuss aspects that have to do with theoretical and conceptual points of view. For instance, they discuss such issues as ‘Should criteria be defined in terms of behaviors, capacities or performances?’; ‘Should we formulate an overall criterion that reflects all performances, or should we apply

    several criteria that are more or less independent from each other?’ In addition they discuss the meta-analytic research that indicates the relevance of job criteria against single jobs and jobs overall (see also Landy and Rastetgary, 1989; Guion, Chapter 13 in this volume).

     Nevertheless, nowadays a major problem for criterion development is that jobs are no longer clearly ‘visible’, and task activities are rapidly changing. Think, for instance, about work in the IT or health care sectors. Both types of job activities could be recognized clearly in the 1980s. Nowadays these jobs are changing rapidly, and will continue to change in the coming years. From a ‘specialized role’ workers are asked to perform a more ‘broad’ role: they have to service clients and they should have knowledge about many issues beside their own

     International Handbook of Selection and Assessment, Edited by N. Anderson and P. Herriot ? 1977 John Wiley & Sons Ltd.

291 Wieby M.M. Altink, Coert F. Visser and Michiel Castelijns

    specialism. The project work now typical of many jobs is one of the best examples of these developments. Criterion development not only relates to the derivation and description of important job performances, but also to question like: ‘ What is the difference in performance between types of jobs?’; ‘Do these differences relate to the content of work activities, or to the organization in which work is performed or the market and clients that are served?’

     Looking at the knowledge available to tackle the above issues, we think that the work of organizational psychologist, and the work of other authors in the field of criterion development, form a sound basis for a discussion about the status of criteria within organizations. These authors have described major principles that relate to the definition and measurement of criteria. A summary of these is given by the performance modeling approach introduced by Martin Greuter (1989). A design procedure is presented by which psychometric pitfalls can be overcome, and which provides direction to the development of adequate criteria for selection and appraisal. This procedure is described in Figure 14.2. It states that you first have to define the problem that is the goal of criterion development (e.g. in the case of selection, the prediction of future work performance of applicants; in the case of appraisal, evaluation of performance of personnel). The second step directs you to defining requirements, such as predictive validity, practicality, and utility. Then you define relevant criterion variables and you formulate the instruments that enable you to predict / evaluate the performance level on these criterion variables. The steps considering structure, form and parameters (steps 4,5, and 6 in Figure 14.2) relate to the combination of the scores obtained or a more ‘clinical’ model (see also Cascio, 1992).