The 27th Annual Conference took place this year in the Californian City of San Diego with more than 4000 participants. SIOP is the US Society for Industrial and Organizational Psychology (vergleichbar mit der Sektion Wirtschaftspsychologie des BDP). All top psychologists in this area showed up, whether academics or practitioner. A top event was the closing address by famous Dr Albert Bandura, emeritus from Stanford, who has changed the face of psychology. He is known as the originator of social learning theory and the theory of self-efficacy.
Bandwidth of Assessment Design leads to higher Validity
Andrew Speer from Central Michigan University presented an AC study with N=342 participants including a job performance rating as validity criterion. His starting point was the finding which is now known for 30 years since the work of Sacket & Dreher: AC ratings yield to low consistence between the same dimensions over different exercises. He made the additional distinction between correlations between dimensions of similar or dissimilar exercises. For the pair of same dimensions in dissimilar exercises the correlation was as expected somewhat lower than for similar exercises. However, the external validity in predicting job performance had different effects. For pairs of dimensions form similar exercises the validity was r = .23, whereas for the pair from dissimilar exercises the validity rose to r = .30.
This means inconsistent ratings between same dimensions of different exercises not just stand for error. Candidates seem to be better in their jobs if there are able to cope with diverse situations. If we narrow down the AC to a limited set of similar exercises we foster consistence of results within the AC but do not represent the bandwidth of job requirements in practice.
Protection within Internet testing
Testing as a tool for pre-selection is now an established procedure: The US Transportation Security Administration (TSA) has several hundred thousands applicants. If you want to apply as an air hostess with Lufthansa you have to go to an internet procedure prior to talk or see any human being. It was reported at the conference that nowadays 10% of applicants do the testing form their mobile device such as I-Phone, which can be tracked by the program.
A panel discussion at the conference provided an update on security. There are two alternative procedures being discussed: the so called „unproctored“ Internet testing vs. the „protoctered“ testing in a testing center where applicants are physically invited. Participants at the panel reported from rising numbers of cheating at the Internet procedure: Testitems are photographed from the screen, published in Facebook or by unions.
The question is how to manage test security. On a technical level one should not promote test items where you can open a second window with Wikipedia and look up for answers. Similarly, test items with simple yes / no answers should be avoided, they could easily be cheated. Panel practitioner recommended that it should be monitored if the item difficulty goes up which is a sign that items have spread around.
The question is what is more favourable: Internet vs. testing center. Testing center does not prevent cheating. Panel discussants reported from several cases in New York where applicants for admission in senior college were sending somebody else to the test. Beyond cost the panel discussants concluded that an advantage of „unproctored“ Internet testing procedure is that you can attract and test more applicants that allow choosing form a larger pool and get finally the best ones.
New insights in Impression Management in Structured Interviews
Several new experiments were highlighted on the important subject what role plays impression management (IM) in interviews. There are different ways how candidates may try to influence interviewers: Through assertive IM (e.g. claiming accomplishments partly untrue), defensive IM (not mentioning negative things) or non-verbal IM (smiling, frequent eye-contact).
Since then one hat the belief that such influence is more a problem within unstructured interview since there the order of questions is fixed. Julia Levashina from Kent State University however showed that also in structured interview especially non-verbal IM has a strong impact on interviewer ratings.
Candidates in interviews may have more influence than we think
Nicolas Roulin from University Lausanne presented a paper with an astonishing experiment. He conducted 92 Interviews, half of the interviewees had to lie on certain questions and use one out of five tactics of impression management. Via a sophisticated procedure interviewer were asked to code at the videotaped interview on the computer screen the exact cue when they believed the person would have laid. There were four possible outcomes: hits (correct identification of the lie), false alarms, missed and correct rejections. Main outcome of the study was that overall only 35% of the (trained) interviewer correctly found the hits or correct rejections. The rest of their assessments were wrong. The 35% mean interviewers are only slightly better than chance to detect those applicants.
Experienced interviewer with more hits and more false alarms
The next question was highlighted what might predict accuracy in detecting Impression Management. Do experienced interviewer with the background of thousand of interviews like most people in HR have a better experience in detecting what is wrong? Nicolas Roulin from University Lausanne found a different pattern how experienced interview got along. They had more hits and at the same time more false alarms, assuming fakes or lies which were not present. Bottom line is, that experienced interview follow a different pattern than novices in interviewing, however in total they do not have a better accuracy. In the wording of signal detection theory they just lowered down their criterion for detection. As Nicolos Roulin pointed out, experienced interviewer in the study told him, that they thought that applicants may invent little stories anyway and did not believe everything by principle.
This outcome is in line with other new research revealing that experienced interviewer are not more effective in detecting lying (Lievens & Peeters, 2008). Individuals seem to overestimate their capability in this regard.
This astonishing result is in contrast to the self-image of many experienced managers and HR people to detect the truth via their experience: Self-esteem is always linked to real outcomes. Nicolas Roulin and his team tried to figure out what are additional predictors of accuracy in interviewing beyond experience. Interesting results: Male interviewers are better in uncovering lies of applicants whereas female applicants do a better job in disguising and lying when they have to in the interview situation.
Detection of behavioural cues within Interviewing
The TV series „lie to me“ made it popular: Based on the real-life scientific discoveries of Paul Ekman, the hero in the series suggests that by facial micro movements any lie might be detected by deception experts.
Weyhrauch from Kansas State University set up an experimental case study with N=95 interviews to follow this notion. The interviews consisted of eight questions; applicants were instructed to lie at four out the eight questions. In order to enhance their motivation they got a 50$ award for the most convincing lie. Interviews were videotaped and 14 Cues indicative of deception for each response were coded, e.g. not enough details in the answer, logical structure of the answer, spontaneous corrections or lack of eye contact to the interviewer. Again, experienced interviewers were not able to detect these cues. Weyhrauch and his team varied the interviewing method.
There are two interviewing methods popular: Situational interviews (SI) ask applicants what they would do in a certain situation, whereas behaviour description interview (BDI) focuses on what they have done in the past. One would assume that SI might invite applicants to a larger extend to tell stories and lie. However, this could not be validated by the study, no difference between SI and BDI could be found.
The question was brought up, when is Impression Management negative. On the one hand promotion and selection interviews based on wrong data of the applicants lead to harm and costs money. On the other hand honest IM means that the applicant really wants the job and is social competent enough to sell him / herself. From one perspective this is the normal desire to appear likeable, from the other it is lying.
Sources:
Lievens, F. and Peeters, H. (2008) Interviewers‘ Sensitivity to Impression Management Tactics in Structured Interviews. European Journal of Psychological Assessment, 24, 174-180
Leveshina, J., Hartwell, C., Morgeson, F., Campion, M. (2012). Impression Management in Structured Interviews: Review of Research and Meta-Analysis. Presentation at the SIOP Conference 2012, San Diego.
Roulin, N., Leveshina, J., Bangerter, A. (2012). Can Interviewers Detect and Discount Honest and Deceptive Applicant IM? Presentation at the SIOP Conference 2012, San Diego.
Speer, A. B., Christiansen, N.D. (2012). Assessment Center Construct-Criterion Relationship: Situational Bandwidths and Predecting Job Performance.
Weyhrauch, W.S., Culbertson, S., Waples, S. (2012). Behavioural Cues as Indicators of Deception in Structured Employment Interviews. Presentation at the SIOP Conference 2012, San Diego.