Hard News by Russell Brown

Read Post

Hard News: Decidedly Undecided

142 Responses

First ←Older Page 1 2 3 4 5 6 Newer→ Last

  • Sacha, in reply to Gavin White,

    Thank you. We had a discussion in an earlier thread about how to simply show error ranges in polls - here's my go at it.

    Ak • Since May 2008 • 19745 posts Report Reply

  • Andrew Robertson,

    As an aside, my perspective on polling is aligned with the 'Total Survey Error Approach.'

    Below is a link to a great textbook on this approach. It is very readable for the non-statistics-minded person.

    Survey Methodology. Groves et al.

    Wellington • Since Apr 2014 • 65 posts Report Reply

  • steve black, in reply to Andrew Robertson,

    Actually we’ve only seen it dip below 800 a couple of times. Those unlikely to vote are still asked the party vote question, but excluded when calculating the party support results.

    I used the numbers on p8 and yes below 800 twice in that series. Excuse me just giving a vague handwaving very rounded range. :-) Was this the right place to get the counts from?

    840 820 834 767 755 813

    So 813 (the most recent number) is those people who expressed a party voting preference (sometimes with a second prompt) AND were likely to vote? And there were other people who were asked and answered, but excluded from the table because they weren’t likely enough to vote?

    Are these numbers unweighted or weighted? Some MR companies, with Heylen in their ancestry, report unweighted bases (sample sizes) with weighted percentages. Excuse me asking and causing more eyes to glaze over.

    And thanks for the reference to that book Andrew. $100 for the eBook version. I’m saving up my coins.

    sunny mt albert • Since Jan 2007 • 116 posts Report Reply

  • TracyMac, in reply to Kevin McCready,

    Love that take - so to speak - on how "the economy" is often such a bullshit term.

    (On another note, can bloggers please consider how medium-grey text looks on off-white backgrounds, particularly with a fine sans-serif typeface? Very difficult to read, esp on small screens. Dark grey is more aesthetically pleasing than stark black, but it still needs to be dark enough to read.

    PAS gets the balance pretty right for texty-text. The lighter grey for other text elements is just on the right side of readability for me.

    Canberra, West Island • Since Nov 2006 • 701 posts Report Reply

  • Andrew Robertson, in reply to steve black,

    Hey Steve

    Yip, you've got the correct numbers, but they happen to be two of the lowest bases we've seen since I can recall.

    Yes 813 is those people who expressed a party voting preference (sometimes with a second prompt) and were likely to vote. And yes there were other people who were asked and answered, but are excluded from the table because they weren't likely enough to vote.

    All the numbers are unweighted, and all the percentages are weighted. It's a MR convention to report unweighted numbers, because weighted numbers can mislead when it comes to determining the reliability of the results.

    Wellington • Since Apr 2014 • 65 posts Report Reply

  • steve black,

    Thanks again Andrew.

    When I reread the caution in your methodology about using it to predict the outcome of the election, I also thought about a certain irony:

    Polling this far out can’t be used to predict the outcome of elections accurately if campaigning works. Especially if it gets disenfranchised young people out there being active and voting.

    Which might raise some interesting questions about whether campaigning works and actually shifts voter behaviour (versus say, awareness).

    sunny mt albert • Since Jan 2007 • 116 posts Report Reply

  • Andrew Robertson, in reply to steve black,

    Which might raise some interesting questions about whether campaigning works and actually shifts voter behaviour (versus say, awareness).

    Yeah - interesting question. Polling is actually only about 2-3% of the work I do (well, maybe a bit more in an Election Year). Quite a bit of what I do involves measuring the effectiveness of social marketing/behaviour change campaigns. Behaviour change is such a gradual process (unless it's legislated!), and it's near impossible to determine causality.

    I see the election campaigns more as brand campaigns, and the internal polls a bit like brand monitors - the parties are trying to find a 'key message' that aligns with their brand and increases credibility among potential voters.

    Wellington • Since Apr 2014 • 65 posts Report Reply

  • Andrew Robertson, in reply to steve black,

    Just to add to that - 'brand Key' is an interesting one.

    I think it works for National at this election because the PM has committed to running and seems fairly healthy. If I was advising a client about their brand though, I'd say don't let it be tied to something that may be temporary.

    Wellington • Since Apr 2014 • 65 posts Report Reply

  • tussock, in reply to Andrew Robertson,

    Actually, we've had people tell us that we shouldn't even report to whole numbers - but provide only ranges. If we did that though, we couldn't convert to seats and it would be hard to tell how close things are under MMP.

    Of course you can convert it to a range of seats. National getting between 58 and 66 seats is what your poll data is saying. A little math would tell you the odds they're polling above 60 seats. Taking the median and suggesting they get 62 seats exactly is simply not supported by the data.

    You could even point out that National get at least 58 seats, Labour and Green at least 52, minor parties 4, and there's six seats could go to either block is the real story of the poll. The uncertainty of the cross-bench point is the bit that actually matters. Highlight it.

    Maybe some more people will GOTV to try to help it one way or the other. The one curse of polling is so many people end up thinking their vote doesn't matter. That it's all decided already, when the uncertainty is always there, right on the cross benches. Well, usually it is.


    Even got a house graphic for you: Striped seats in the middle with colours covering the uncertainty range of each party. Labour's got 5 half-seats, Nats 6 Half-seats, Greens 3 half-seats. Look at 'em there, just waiting for your lazy ass to get out and vote.


    Edit: Then put up a graphic with how many seats the non-voters could swing. Because it's what, 30 seats? More?

    Since Nov 2006 • 611 posts Report Reply

  • Mr Mark,

    Well, as the bloke who kicked the whole thing off with this (admittedly very brief) analysis of the Left-leaning (or,at the very least, anti-National Government) proclivities of the Undecideds and those unlikely to vote... http://sub-z-p.blogspot.co.nz/2014/06/fairfax-ipsos-poll-february-2014.html, I'll be replying to Andrew ( and Thomas and KiwiPollGuy ) in an up-coming post (couple of weeks time at most). Not that I'll be spending too much time arguing about the correlation stats (I'm sure Puddleglum can argue his or her own corner on that score - indeed, I think they've already responded briefly to Thomas in their comments section). I'm more interested in the wider implications.

    In the mean time, here are the Fairfax-Ipsos Preferred PM breakdowns by Party Support...http://sub-z-p.blogspot.co.nz/

    Wellington • Since Dec 2009 • 128 posts Report Reply

  • Mr Mark,

    I see my first link didn't work.

    Let's try it again...http://sub-z-p.blogspot.co.nz/2014/06/fairfax-ipsos-poll-february-2014.html (If it doesn't work this time, can I just say Bugger ! - thank you).

    Wellington • Since Dec 2009 • 128 posts Report Reply

  • Andrew Robertson, in reply to tussock,

    Tussock – it’s not that simple.

    The results for each party are non-independent, so if the number of seats for one party changes, this changes the number of seats other parties will get (irrepective of what their range is). Parties with a range crossing the 5% threshold would then add to this complexity and would shift entire ranges altogether. Then to add even more to the complexity, the total number of seats will shift depending on whether parties have ranges that might create an overhang. The number of seat combinations would be ridiculous, and impossible to report.

    So no, a little bit of maths won’t do it unfortunately.

    Wellington • Since Apr 2014 • 65 posts Report Reply

  • Ian Dalziel,

    How would you ever know if the people 'polled' are telling the truth...
    it does seem to be a rare commodity in the wild!

    There are many reasons an individual might give a response contrary to their true intentions, or even just to appear as if they had intentions...

    Christchurch • Since Dec 2006 • 7953 posts Report Reply

  • Andrew Robertson, in reply to Ian Dalziel,

    How would you ever know if the people ‘polled’ are telling the truth…

    You don’t, and some people may lie. It would have to be a lot of people though to really make much of a difference in a random sample of 1,000. People are much less obsessed with politics than those who comment on politically-orientated blogs. Not that many people will have a motivation to lie.

    In each poll I do chuckle a wee bit at some of the responses we get which help to illustrate people’s (dis)interest in politics. For example – I’d vote for that fellow Key – can’t remember what party he’s in and I’d vote for Winston First and What’s the blue party? and The one that supports farmers.

    Wellington • Since Apr 2014 • 65 posts Report Reply

  • steve black, in reply to Mr Mark,

    Well, as the bloke who kicked the whole thing off with this (admittedly very brief) analysis of the Left-leaning (or,at the very least, anti-National Government) proclivities of the Undecideds and those unlikely to vote…

    Alas, we really need the raw data to make stronger inferences. And in fact we really need data from a panel (or longitudinal study) to know how individuals move in or out of being decided, wanting a change, etc. There are too many different causal models which fit the correlations observed from outside the system (I know them as "ecological correlations" but the name depends on what discipline you come from). We can't rule out some models based on the evidence we have, so we will be left with multiple interpretations.

    sunny mt albert • Since Jan 2007 • 116 posts Report Reply

  • steve black, in reply to Andrew Robertson,

    People are much less obsessed with politics than those who comment on politically-orientated blogs.

    Oh please say it is not so... People who don't care and just get on with life? :-)

    Glad you handled that question. I saw it and left the room quickly. :-)

    sunny mt albert • Since Jan 2007 • 116 posts Report Reply

  • Richard Aston, in reply to steve black,

    Oh please say it is not so… People who don’t care and just get on with life? :-)

    Yes my friend we swim in a sea of political apathy. I am shocked when I meet people who are not sure who the blue party is or just don't vote.

    I read the American Idol show gathered as many votes as the congressional election.

    Northland • Since Nov 2006 • 510 posts Report Reply

  • Jim Cathcart,

    Some people might be interested in this blog post on communicating statistical uncertainty, which came to me through Linkedin this morning.
    https://gss.civilservice.gov.uk/blog/2014/06/uncertainty/

    Since Nov 2006 • 228 posts Report Reply

  • Mr Mark, in reply to steve black,

    Part 1


    "There are too many different causal models which fit the correlations observed from outside the system. We can't rule out some models based on the evidence we have, so we will be left with multiple interpretations"

    I think that's more a criticism of Puddleglum's analysis on The Political Scientist (essentially along the lines of Thomas Lumley's and KiwiPollGuy's critique). I'll leave Puddleglum to respond to those arguments.

    I haven't focussed on correlation analysis at all. And I haven't been particularly interested in swings in and out of the Undecided category (although the evidence is clear that there has, indeed, been a swing from Labour into Undecided territory over the last 6 months).

    I've simply argued that, throughout the last 18 months of Fairfax-Ipsos Polls, we have a striking contrast between the climate of opinion suggested by (1) the Party Support results (based solely, as they are, on the Decided and those likely to vote) and (2) the 'Do you favour a Change of Government' measurement (based on the entire sample). A striking contrast all but ignored by Fairfax journos and the wider MSM (and, therefore, hidden from the wider public).

    Wellington • Since Dec 2009 • 128 posts Report Reply

  • Mr Mark, in reply to steve black,

    Part 2

    (My Lap-top suddenly can't handle posting longer comments so, unfortunately, I'll have to break it up into different parts. This is Part Two: The Sequel)

    If you take October 2013 ( Fairfax-Ipsos ) as an illustrative example, those favouring a Change of Government outnumbered respondents happy with the status-quo by a clear 7 point margin (50/43). One can only infer that the broad mood was for a change of government. And yet in the same poll, the Nats led Labour by a hefty 16 points and the Right led the Left by a clear 7 points in the one measurement that receives all the MSM attention and associated ( National is soaring/Labour is Dog-Tucker ) hyperbole. Look at the comments on Puddleglum's site, for instance, and you'll see people are shocked to discover that National is not, in fact, currently favoured by well over half the Country. That, indeed, despite the 57% rating in the latest (obviously rogue) Fairfax-Ipsos , National's support among the entire sample (read: a rough approximation of all adult New Zealanders) is in fact slightly lower than it was in 2012 (when they were scoring less than 50%).

    Wellington • Since Dec 2009 • 128 posts Report Reply

  • Kumara Republic, in reply to Andrew Robertson,

    People are much less obsessed with politics than those who comment on politically-orientated blogs.

    Yes, it's a sad state of affairs when a perfect storm of factors - including narrowing attention spans and tabloidised political commentary - conspire to produce low-info voting. Civics education has been touted as a partial solution, but in America, where the term 'low info voter' originated, civics ed is integral to the system but voter turnout is even lower than ours.

    The southernmost capital … • Since Nov 2006 • 5446 posts Report Reply

  • Mr Mark, in reply to steve black,

    Part 3

    I really can't see multiple interpretations of this. Except to suggest that possibly - just possibly - a small minority of those favouring change wanted a National-led Government, but one involving NZ First in some capacity (whether in a formal Coalition or other arrangement). But you'd have to guess that (assuming they exist at all) they'd comprise a pretty small sub-group of the Yes, Change Government response.

    Yep, I'd certainly like the question to be a little more explicit - along the lines of asking the entire sample: "Do you favour a Labour-led or National-led Government ?" (Or a more neutrally-expressed version of this). And, in an ideal world, all of the five public polling companies would ask it - not just Fairfax-Ipsos .

    Wellington • Since Dec 2009 • 128 posts Report Reply

  • Mr Mark, in reply to steve black,

    Part 4 - The Finale

    .......But I just don't think there are multiple interpretations or overly-complex nuances here (particularly in the context of the liberties - bordering at times on gross misinterpretation - taken with Party Support data by journos in their Front Page and Op-Ed poll analyses).

    Need to acknowledge the massive disconnect between a relentless media narrative of overwhelming support for the Key Government and what the Change of Government data is actually telling us. The thing is: the MSM don't do nuance when it comes to interpreting the Party Support results, so I really don't think we should be treating the Mood for a Change of Government measure as some great mystery that we need to be exceedingly cautious about interpreting.

    Wellington • Since Dec 2009 • 128 posts Report Reply

  • Mr Mark,

    I do, of course, acknowledge Andrew's argument about the whole rationale for conducting political opinion polls (the brief being to determine as precisely as possible the outcome of an election held at the time of the poll - and therefore the need to exclude likely non-voters and the Undecided), but there are much wider implications involving public perceptions of the general political mood of the Country, which in turn of course, are inextricably intertwined with turnout.

    Wellington • Since Dec 2009 • 128 posts Report Reply

  • steve black, in reply to Mr Mark,

    I’ve simply argued that, throughout the last 18 months of Fairfax-Ipsos Polls, we have a striking contrast between the climate of opinion suggested by (1) the Party Support results (based solely, as they are, on the Decided and those likely to vote) and (2) the ‘Do you favour a Change of Government’ measurement (based on the entire sample). A striking contrast all but ignored by Fairfax journos and the wider MSM (and, therefore, hidden from the wider public).

    You have outed those reporting polls on failing to tell the whole story. Good work. It reminds me how Tony Abbott (in opposition) was consistently rated lower as preferred Prime Minister than either Julia Gillard or Kevin Rudd. But his rating just wasn’t reported. All the focus was on a “shocking new low” for Gillard or Rudd.

    However,

    My alternative model #1 is that 8% of those polled answered “I favour a change of Government” AND say they will give National their party vote. That’s because they want a change of the rag tag band of small parties supporting National.

    My alternative model #2 is that 9% of those polled answered “I favour a change of Government” AND say they will give National their party vote. That’s because they want a change but not to a Labour led government.

    My alternative model #3 is that 3% of those polled answered “I favour a change of Government” AND say they will give National their party vote. That’s because they are perfectly able to hold apparently contradictory thoughts in their head.

    My alternative model #4 is that 10% of those polled answered “I favour a change of Government” AND say they will give National their party vote. That’s because they wish there was some alternative but don’t see one.

    My alternative model #5 is that 6% of those polled answered “I favour a change of Government” AND say they will give National their party vote. That’s because they don’t like National but they want to make sure the Greens have no influence in Parliament.

    I simply maintain that you can’t exclude my models on the grounds that they don’t fit the data. And I can’t choose among them either. We haven’t got the data at the detailed level needed. At this point further arguments about which is the better model stop being evidence based. But the different models (yours included) make different predictions about how the elections might go, and what the effect of different “agreements to cooperate” by different parties might be during the campaign.

    note: I haven’t seen the exact wording of the questions the Ipsos poll is using and there may be 11 more models I could come up with if I looked at the wording and question order.

    sunny mt albert • Since Jan 2007 • 116 posts Report Reply

First ←Older Page 1 2 3 4 5 6 Newer→ Last

Post your response…

Please sign in using your Public Address credentials…

Login

You may also create an account or retrieve your password.