Legal Beagle by Graeme Edgeler


New Zealand rockets up the anti-Corruption ratings: a non-Spinoff investigation

New Zealand is again atop the Transparency International list of countries with the lowest perceived corruption.

With an index score of 90 out of a possible 100, the perception of non-corruption has also increased (last year, when New Zealand ranked forth, with a lowly 88). It’s not that the rest of the world is getting worse, New Zealand's apparently getting better. What is New Zealand doing right? I decided to investigate.

The Transparency International Survey of Corruption perceptions takes data from a range of international surveys that, in part, have questions on corruption. There are thirteen in all, including The African Development Bank Governance Ratings, the Economist Intelligence Unit Country Risk Ratings and World Justice Project Rule of Law Index. Corruption is a factor in governance, in economic risk, and in the rule of law, so Transparency International takes the data from the corruption parts of those rankings, and brings that together in one place.

New Zealand features in seven of the thirteen reports (Transparency International will give a country a rating if it features in at least three). Transparency International takes the information it gets from each of those surveys, turns that into a score out of 100, and then averages those scores to determine each country’s score.

So, how did New Zealand’s average score increase so drastically, up from a paltry 88, to a stellar 90?

Let’s look at each of the reports.

In the World Economic Forum Executive Opinion Survey (which surveys business executives), our corruption score fell from 92 to 90. That’s not good.

According to the IMD World Competitiveness Yearbook, a survey of business executives by IMD Business School in Switzerland, New Zealand’s corruption score increased from 93 to 95. That’s the two point increase we need to find, but all it does is cancel out the fall caused by the presumably different business executives that the World Economic Forum used.

In the Global Insight Country Risk Ratings, an assessment by in-country specialists who are part of the consulting firm IHS (now IHS Markit), New Zealand’s corruption score was unchanged: 83 in both 2014 and 2015.

New Zealand was up two points in the Economist Intelligence Unit’s Country Risk Ratings (which uses the experts in the Research Arm of the Economist to construct rankings), going from 88 to 90.

Unfortunately, New Zealand was down in its corruption rating in World Justice Project’s Rule of Law index, dropping from 83 to 79.

And New Zealand was down by five points in its corruption score according to score of risk analysis company Political Risk Services’ International Country Risk Guide, falling from 98 points to 93.

How then did New Zealand’s score increase? If you’ve been counting along, that’s only six reports, and New Zealand featured in seven. In the Bertelsmann Foundation’s Sustainable Governance Indicators report, New Zealand’s corruption score increased from 81 to 99. This is the sole reason for New Zealand’s index score increased.

How did the Bertelsmann Foundation arrive at its conclusions? Transparency International describes it in the following way:

The Sustainable Governance Indicators (SGI) examine governance and policymaking in all OECD and EU member states in order to evaluate each country's need for, and ability to carry out, reform.

The indicators are calculated using quantitative data from international organisations and then supplemented by qualitative assessments from recognised country experts. 

What changed between 2015 and 2016 that accounts for the upgrade? The people who prepare the Bertelsmann Foundation’s report have offered their own insights:

New Zealand is one of the least corrupt countries in the world. Prevention of corruption is strongly safeguarded by such independent institutions as the auditor general and the Office of the Ombudsman. In addition, New Zealand has ratified all relevant international anti-bribery conventions of the OECD and the United Nations. All available indices confirm that New Zealand scores particularly high regarding corruption prevention, including in the private sector.

This synopsis footnotes the Freedom House: Freedom in the World 2015 index. And what did that report have to say about corruption in New Zealand: 

C. Functioning of Government: 12 / 12

New Zealand is one of the least corrupt countries in the world. It was ranked 2 out of 175 countries and territories surveyed in Transparency International’s 2014 Corruption Perceptions Index. However, scandals involving political donations from migrant Chinese businessmen have hurt the government’s image. In May 2014, Minister Maurice Williamson resigned amid allegations of intervention in a domestic violence case involving a Chinese businessman who had made political contributions. In another case, donations were made to the National Party by a Chinese firm, one of whose board members is the husband of the Justice Minister.

So why, did Transparency International raise New Zealand’s anti-corruption score increase in 2016? According to the authors of the reports they relied on (and the authors of the reports they relied on) it’s because New Zealand was highly ranked on Transparency International rankings in 2014.

Now that's funny, and points out a pretty big problem with all of these indexes (although at least the Transparency International index is not quite as ridiculous as the index that suggested that North Carolina is no longer a democracy). but it's not actually the reason.

The Bertelsmann Foundation is interested in sustainable governance, of which corruption is only a part of its assessment. Indeed, the data transparency international uses from their assessment comes from a single question:

To what extent are public officeholders prevented from abusing their position for private interests?

The recognised country experts were told this question addresses:

… how the state and society prevent public servants and politicians from accepting bribes by applying mechanisms to guarantee the integrity of officeholders: auditing of state spending; regulation of party financing; citizen and media access to information; accountability of officeholders (asset declarations, conflict of interest rules, codes of conduct); transparent public procurement systems; effective prosecution of corruption.

Helpfully, the experts tasked with answering this question were warned:

Note: Please be aware that the Corruption Perceptions Index (CPI) of Transparency International uses the data and information given in response to question D4.4 for their assessments. To avoid circularity of assessments, please do not base your evaluation on the CPI.

They were asked to give a score from 1 to 10, with scores explained as following:

  • A score of 9 or 10 signifies that “Legal, political and public integrity mechanisms effectively prevent public officeholders from abusing their positions.”
  • A score of 6-8 would show that “Most integrity mechanisms function effectively and provide disincentives for public officeholders willing to abuse their positions.”

Quantitatively, this appears to be the source of the difference in New Zealand’s Transparency International index score between 2015 and 2016.

So, presumably, that question saw New Zealand get a score of 8 in 2015 (which Transparency International converted to 81), and a 10 in 2016 (converted to 99)?

Actually, no. The Bertelsmann Foundation's assessor gave New Zealand a 10 in 2016, and a 10 in 2015, and a 10 in 2014. It's just that in 2016, this 10 was converted by Transparency International to a 99 on a 100-point scale, and in 2015, that 10 was considered to be worth 81 by Transparency International.

Transparency International explains how this works:

Standardise data sources to a scale of 0-100 where a 0 equals the highest level of perceived corruption and 100 equals the lowest level of perceived corruption. This is done by subtracting the mean of the data set and dividing by the standard deviation and results in z-scores, which are then adjusted to have a mean of approximately 45 and a standard deviation of approximately 20 so that the data set fits the CPI’s 0-100 scale. The mean and standard deviation are taken from the 2012 scores, so that the rescaled scores can be compared over time against the baseline year.

I know a number of people much better at statistics than me occasionally read and comment here, so I won't try to explain it. In the end, I doubt understanding the exact process will help. Although I suspect it gives lie to Transparency International's outline of the results as shocking because so many countries score below 50:

So what explains New Zealand's rise?

In 2015, when New Zealand's Bertelsmann Foundation score of 10 was standardised as an 81, Denmark's score of 10 scaled to 97. Finland and Sweden, whom New Zealand beat on the Bertelsmann Foundation's measure (they received 9s) got scores of 91. The Netherland's score of 7 got scaled to 97 (oops). And Canada's 8 got them 81 like us.

Why did New Zealand rocket up Transparency International's Anti-Corruption ratings in 2016? And why did the Netherland's fall from fifth to ninth? There were data entry errors in Transparency International's 2015 analysis. Fix that mistake, and New Zealand's index score didn't go up from 88 to 90, it went down from 91 to 90. And New Zealand would have been first equal (relying on rounding) in both years.

Assuming, of course, there aren't more errors. And ignoring that these sorts of rankings are stupid anyway.

Update: Several months after this post was published, Transparency International issued an errata for their 2014 and 2015 Indexes. This was accompanied by a press release, and a post on Medium explaining how the error occurred.

19 responses to this post

Post your response…

This topic is closed.