Careful now: Understanding the reality of bias testing

Careful now: Understanding the reality of bias testing

Introduction

The growth in interest around our unconscious biases at work has led to more training, coaching and in particular bias testing in the area.  This is a welcome development for those frustrated by the apparent loss of momentum in achieving real  change in organisations.  Unconscious bias is now not only on the lips of diversity and inclusions managers but also on the minds of senior managers looking to enhance organisational performance by reducing conflict, engaging better with customers and by getting the most from staff.  Whilst we may welcome this greater awareness of our often unintentional and subtle biases there are pitfalls awaiting the unwary.  Implementing bias testing requires a different consideration than that normally employed when considering personality or ability testing, in three key areas:

Test Choice

Bias tests come in a range of options in terms of content and delivery.  Most bias tests use a variation on the implicit association test (Greenwald et al., 1998) which bypasses any lack of insight or willingness to disclose by tapping the implicit and unconscious associations people may hold about particular social groups.  IATs come in many forms but the key distinction is between norm referenced and criterion referenced scoring.  Norm scoring will place an individual within the wider population but say nothing about the likelihood of an individual's unconscious associations becoming behaviour ( Blanton and Jaccard, 2005).  Criterion referenced tests are linked to the predilection for both subtle and less subtle behaviours such as bad mouthing, avoiding or joke telling.

Suppliers often offer metrics on the basis of individual differences such as Age, Disability, Ethnic Origin, Faith/Belief, Gender, Nationality, Sexual Orientation and a host of other differences.  Prospective test users should be clear about the purpose of testing and whether it is to address particular organisational issues, or as part of a personal development intervention (or both).  Testing needs to be both relevant and proportionate.  As an organisation, test users might confine testing to areas where they know they may have an issue (e.g. based on staff under-representation, lack of group progression, poor sales to that customer group or particular incidents or complaints).  At a personal level test takers might be afforded wider test access to support their personal development (e.g. the full test range).  They may also have follow up testing ( e.g. a Black Woman test) once  preliminary testing (e.g. a Black Ethnic Origin test and a Gender test) suggests a possible issue for clarification. Knowing that tests are available for such follow up is an important part of test choice.

The engaging nature of IAT tests, the ease of online completion and the apparent simplicity of IAT can lead to test takers rushing into testing without being properly prepared.  Proper briefing and test practice is critical if results are to be trusted, as test retest reliability can fall from around .80 to .50 if test takers are learning on the test.  Allowing direct individual access to testing and feedback, although bypassing many of the problems listed below around data protection and feedback,  also removes the capacity to manage what and how test takers  understand and use the test.  Unsupervised testing like this can undermine the measurement, credibility and organisational importance of bias testing. It can reduce bias testing to entertainment  or,commonly, simply provide  confusing materials for posting on social networking sites

It should be noted that some 'tests'  bill themselves as measuring our personal biases but use more explicit and conscious sorting tasks similar in appearance to the IAT.  They require conscious sorting of stereotypical photographs.  Their use can be a triple whammy, because not only do they have no basis in what  the psychological research tells us, but they may trigger and reinforce stereotypes. In addition they can alienate prospective users and takers of valid and reliable bias tests who recognise their potential to increase bias through stereotype triggers.

Data Protection

Such is the political sensitivity around bias, created often by well meaning but poor advised internal diversity staff, that many test users find that they have to employ some sort of firewall between the organisation and the personal data.  This often requires one or even two levels of external consultants so that there is no question of data being seen by employers.  The length of time personalised data is kept is often minimised to between 7 and 28 days.

Feedback and follow up

One of the strongest criticisms of the Greenwald et al IAT is around feedback.  Norm based feedback tells us little about how a person may behave, and there is no metric-to metric data in terms of what test scores and changes in test scores mean ( Blanton and Jaccard, 2005).  Unsupervised testing where feedback is automatically generated can be misinterpreted by test takes.  For example,  The Greenwald et al IAT may tell test takers that they have a 'Strong Preference for White people over Black people'  which test takers often interpret this as meaning they are 'racist', which may or may not be the case.  This can have profound effects on an individual.

 Individual test takers can also be very sensitive to feedback and the nuances of words can become critical.  Feedback has the potential to trigger hitherto dormant stereotypes, particularly in those with low bias. Research suggests that even using the work 'prejudice' can trigger adverse test taker reactions and increase bias, especially in low bias individuals.  Many bias test users prefer to use the word 'bias' in feedback rather than prejudice but some have gone further and use terms such as 'unintentional people preferences'.

Having an idea as to what the organisation and the test taker will do with test results is as critical as making the right test choice.  Identifying an issue for the individual or organisation is useful, but the test users must be clear about what support there will be in response to results.  Research shows that traditional 'sheep dipping' of individuals in bias management training is often ineffective and may even trigger bias in those who begin with low bias.  Planning for bias testing therefore must include some thought around subsequent interventions, which may have to be personalised to get the best effect.

Conclusions

Bias testing can be seductive and has the potential to make organisations more effective Implementing testing requires full consideration of all of the issues and implications.  Poor thought through or in appropriate bias testing can actually increase organisational bias within the organisation, particularly if test choice, data protection, feedback and intervention are not part of the planning process.