The Dominion Post reports today that the state agencies that spent $1.5 million on psychometric testing last year may have been on shaky legal ground. It might also have added that they're basically standing on some pretty shaky science too.
The core issue in today's story is the use of such testing in restructuring, which has seen employees made redundant not on consideration of their actual work records, but on the basis of their responses to third-party prompts such as "I hate opera singing" and "I like to try new, exotic foods".
An Employment Court decision in April (warning: Word doc) has raised significant questions over the use of testing in such a context. The case wasn't entirely straightforward: Derek Gilbert also claimed his union activity contributed to dismissal, and Chief Judge Colgan did not find that his employer, Transfield, had acted unlawfully. He did decide that Gilbert had been "dismissed unjustifiably" and ordered reinstatement and compensation.
And the judge was scathing about Transfield's use of psychometric testing, finding that its refusal to disclose actual test scores and its lack of access to key elements of the tests themselves ("including questions asked and the actual answers given") was:
... not consistent with the requirements of the Act for information sharing, disclosure, and objective rationality. Not only was this information not available to Mr Gilbert but it was apparently not available to Transfield. That is one illustration of the dubious value of using a psychometric testing tool, designed for recruitment and managerial promotion, to determine which of a number of existing employees should be made redundant. Although the owners of the testing system may have had good reason to keep its ingredients and even results secret, that illustrates the inappropriateness of its use in a process that requires openness and information exchange. Employers proposing to use testing procedures that they do not fully understand, and are not permitted to know about, will have difficulties when challenged by employees such as the plaintiff to justify the consequence of dismissal effected in reliance on the products of such systems.
It would be deeply ironic if managers who sought to minimise their own risk by outsourcing good judgement had in fact exposed their organisations to employment law action.
David Farrar wrote in response to the decision that in serving on interview panels for CEO-type psoitions he had "found the test results to be very accurate, and helpful," but the actual record of such testing is questionable, to say the least. It is often based on dubious binary assumptions and, critically, its test-retest reliability has been shown in several studies to be hopeless. If you retake the test after only a five-week gap, there's around a 50% chance that you will fall into a different personality category compared to the first time you took the test.
And that's when you're not even trying to present yourself differently. Last year, Herald reporter James Russell took a proprietary test twice: once "honestly" and then:
On the second test, I decided I was going to be brilliant at making presentations and loved being the centre of attention (both of which I dread). I also raised my game across the board, although not enough to appear a complete narcissist.
The result? Apart from a bit of mumbling and shuffling around on the spot, the testing firm's expert couldn't tell the fucking difference.
And thus, judgements on people's lives are made on the basis of tests their managers do not understand and which are poorly-supported in science, essentially to relieve those managers of the burden of managing. I don't realistically expect to see the use of these tests cease or even singificantly decline, but I am quite glad to be in a line of work where I am unlikely to ever be presented with one.