Posts by Andrew Robertson
-
Hard News: Where are all the polls at?, in reply to Mr Mark,
I'm not sure, but you're correct.
However if you have some really clever data science skill, you could draw useful conclusions from even the most unrepresentative data sources.
-
Hard News: Where are all the polls at?, in reply to Paul Campbell,
Despite the name, push polls aren't actually polls, they're a type of campaigning. For that type of campaigning to work in an electorate, you'd need to call 10 times or more the number of people that get called for a random poll.
-
Speaker: ‘Kiwimeter’ is a methodological…, in reply to Hilary Stace,
I've done five or six studies that asked similar sorts of questions. Not once was I asked by the Human Ethics Committee to change their wording.
My submission did give research justification for the statements I included.
-
Polity: Global behemoths and tax, in reply to Don Christie,
I stand corrected then. I know of a company with a clause like that in its shareholder agreement.
-
Polity: Global behemoths and tax, in reply to FletcherB,
I will however point out, that said companies, are generally required (by law ?), to maximise returns for shareholders… which obviously, includes not “spending” money that does not need to be spent… thus, it’s not just a “moral” matter… they are obliged either by legal requirement or at least perceived duty, to minimise their tax exposure/payments by any LEGAL means possible…
Yep. A company's first priority is to their shareholders. This essentially means they want to pay as little tax as possible within the law. If they do anything to lower shareholder value, they can be taken to court by the shareholders. So changing the law might gain NZ more tax revenue, but this would also encourage companies to consider whether having a NZ-based operation (in its current form) is maximising shareholder value. Catch 22 I guess.
-
Speaker: ‘Kiwimeter’ is a methodological…, in reply to mpledger,
It was a dumb question because it never defined special treatment. Person 1 could say Maori don’t deserve special treatment but believe that the Waitangi Tribunal and Whanau Ora are not special treatment but fair treatment. – that Maori should get those things because it balances out the unfairness that happened to them before. But person 2 might think that the Waitangi Tribunal and Whanau Ora are special treatment and think Maori should not have them. So both people would answer “agree” but their perspectives would be radically different … and that defeats the purpose of the question.
This is why we have psychometric measures in the first place. The discipline exists out of recognition that you can’t measure an internal construct with a single attitude statement.
I’m not saying your criticism is wrong or invalid Megan (not at all). And psychometricians do need to think more about context, and how some statements can make people feel. (For example, I’ve heard about a personality test being used in employment that asks ‘Do you prefer a bath or a shower?’. The question may be a valid measure of some personality trait, but it’s inappropriate in that context.)
Many of the criticisms I’ve read are not about Kiwimeter at all. They’re about the way social science measures internal constructs. Seems to me the discussion/debate should be about that, rather than Kiwimeter. It would be a good debate to have.
-
Speaker: ‘Kiwimeter’ is a methodological…, in reply to Angela Hart,
Yes, but the researcher's view is not always given as much weight as some might like, and what what has value to one person may not have value to another.
Also, in a competitive tender situation you have multiple companies telling you all sorts of things, so the final decision will rarely come down to something like cognitive testing alone.
-
Speaker: ‘Kiwimeter’ is a methodological…, in reply to Ian Dalziel,
You’d think they’d be able to see the effects of cognitive dissonance, which is rife in modern society (and politics), and want to exclude its skew to get the clearest results…
When it comes to the big research projects, the final procurement decisions are usually made by a panel of people, one or two of whom may be researchers, and the others will be internal stakeholders at various levels of seniority (usually more senior than researchers).
The researchers may understand the value of cognitive testing, or they may not. However they are just one voice among others (often more senior people who place more value on what the research will help them achieve and the budget they have, than on how it's conducted).
-
Speaker: ‘Kiwimeter’ is a methodological…, in reply to Tze Ming Mok,
I work in the Social Research division of a NZ research company. I will often recommend and offer cognitive testing. You need to pick the right projects/clients though. It can be a really hard sell because it’s quite expensive when it’s done properly (and nearly all of the big research jobs are competitive tenders, where price is a big factor).
Also, from what I’ve heard from clients, cognitive testing is quite often not done properly (they often seem really surprised after they’ve observed me carrying out cognitive testing, or read my cog. testing reports).
-
Speaker: ‘Kiwimeter’ is a methodological…, in reply to Tze Ming Mok,
It’s expensive and slow to do this kind of surveying – it’s ‘oldskool’ in a time of wanting quick online fixes that are ‘good enough’. But I actually think it’s incredibly important – as the British election evidence and our Kiwimeter’ acceptability problem shows – that it’s worth holding up some survey methods as akin to a gold standard, otherwise the baby is out the window with the bathwater.
No argument here. I have designed and run door-to-door probability surveys in NZ, so understand their value. They are becoming rare though. So rare that in a decade or two I doubt any private company in NZ will offer them. It costs a lot to maintain a national face-to-face field force.
The job of the good researcher is changing I think. It will become less about understanding data collection methods, and more about knowing how to sift through the garbage to uncover insights.
As an aside (and in support of your argument), it’s nice to know that the two polls that have come closest to predicting that last two General Elections in NZ both use probability sampling (or try to approximate it).