Hard News by Russell Brown

Read Post

Hard News: Gower Speaks

206 Responses

First ←Older Page 1 2 3 4 5 6 9 Newer→ Last

  • Ianmac, in reply to bob daktari,

    I couldn’t care less for the continual news reportage of polls…. seems only to satisfy the math geeks who seem 100% in agreement that they are (seriously) flawed and cannot a soundbite make

    That may be so but a huge part of the audience get the message intended.
    This or that Party is rising in the Polls. They must be OK.
    This or that Party is down in the Polls. They must be rubbish.
    That's all I need to know because I have more important stuff to do.

    Bleneim • Since Aug 2008 • 135 posts Report Reply

  • steve black, in reply to James Green,

    Hi James,

    Thanks for that. A link lets everybody else see it too, not just me with my shelf of reference books. Much obliged.

    @ Peter Calder: yes you are quite right, but it is more subtle (yeah, whatever dude maths geek stuff coming) because at very low or high percentages (<1 and >95) the confidence isn’t symmetrical about the point estimate. Which is why the proper confidence limits calculated from the binomial distribution never dip below zero or go over 100%. And it’s also the reason that you shouldn’t ever write United Future 0.1% +/- 0.2%. The properly estimated Confidence Limits won’t be positive and negative in the same amount. So you can’t just have a simple +/- with a single number.

    When somebody writes “party support is at 0.1% +/- 0.2% this is almost always a sign that they are either using an approximation to the true distribution which shouldn’t be used near the extremes (0% and 100%), or not sufficiently statistical literate.

    @ JonathanN: Bless you for putting things straight using R and the proper calculations. Am I seeing something not quite right in your confidence limits for NZF? If their “support” is estimated at 4.9% your confidence limits are outside the point estimate: 5.3% – 9.1%. Or have I missed something? I stopped after finding this, so I haven’t pasted your code into my R or gone back and checked any of the others.

    sunny mt albert • Since Jan 2007 • 116 posts Report Reply

  • Pete George, in reply to steve black,

    steve, JonathanN has used the Colmar results which had 7% for NZ First, not the 4.9% they got with Reid Research.

    Dunedin • Since Dec 2011 • 139 posts Report Reply

  • steve black, in reply to bob daktari,

    I couldn’t care less for the continual news reportage of polls…. seems only to satisfy the math geeks who seem 100% in agreement that they are (seriously) flawed and cannot a soundbite make

    It's not the polls which are flawed. They are what they are. It is the analysis and reporting which is seriously bogus. They could all save a lot of money and just follow iPredict or use a random number generator to create their narrative.

    sunny mt albert • Since Jan 2007 • 116 posts Report Reply

  • steve black, in reply to Pete George,

    Ah. Thanks. Now I can focus on the interesting part...

    sunny mt albert • Since Jan 2007 • 116 posts Report Reply

  • Russell Brown,

    And meanwhile, Roy Morgan:

    Labour/ Greens (45%) regain lead over National (43%) for first time since January

    Quite a lot of nil or marginal change, but National dipping 2.5% and NZ First rising 2% and the Conservatives 1% might be significant.

    Labour rose but Lab/Green vote fell slightly.

    Internet Party in at 0.5% too, as is Mana. This might yet be a thing.

    Auckland • Since Nov 2006 • 22848 posts Report Reply

  • Rob Stowell,

    Definitely not a done deal. Long way to go and bound to be the odd unexpected wobble along the way.

    Whakaraupo • Since Nov 2006 • 2110 posts Report Reply

  • David Hood,

    Here is a very, very rough election result simulator (written in R) just using random binomial draws based on the support in a poll.
    https://www.dropbox.com/s/ylbsp9ggm1y9lls/oneVoteTwoVoteRedVoteBlueVote.R
    I'll put up a proper copy on github tomorrow, but my computer this evening isn't really set up for that.
    With the Colmar poll, the single most likely outcome (49% ish ish ish) was that it was up to NZ First provided the Conservatives were not gifted a seat. If the Conservatives have a seat the single most likely result is a clear National victory (54% ish ish ish).
    But this is so speculative it falls into "for the purposes of entetainment".
    Also, while I know data, I am sooo not a statistician.

    Dunedin • Since May 2007 • 1445 posts Report Reply

  • David Hood,

    And with the Roy Morgan anything but NZ First being a decider is pretty unlikely (baring a Conservative seat, and even if the Conservative's have one NZ First deciding is still more likely than anything else by a bit).

    Minor correction to the above post- with the Colmar results, in my very bad simulator, NZ First is about as likely to be the decider as National is to be clearly ahead.

    Dunedin • Since May 2007 • 1445 posts Report Reply

  • Andrew Robertson, in reply to steve black,

    When somebody writes “party support is at 0.1% +/- 0.2% this is almost always a sign that they are either using an approximation to the true distribution which shouldn’t be used near the extremes (0% and 100%), or not sufficiently statistical literate.

    Also, there is a time factor. The methodological and statistical detail you could present for a single poll is enormous. I’ve delivered 60-100 page reports to clients, just describing fieldwork and sampling success, data treatment, and error estimates (no actual interpretation of results). Putting something like this together can take a couple of weeks.

    In the short time available for exporting, verifying, weighting, analysing, and writing up poll results, judgement calls simply must be made. I’m not making excuses here. While confidence intervals for parties polling below 1% could be interesting to a small group of people, they are not relevant to most, and have little bearing on what can be reliably inferred from a poll. I simply wouldn’t devote time to this unless there was a very good reason why it’s more important than, and should take the place of, other things to be reported on.

    Wellington • Since Apr 2014 • 65 posts Report Reply

  • Ianmac, in reply to Russell Brown,

    Fancy Audrey publishing the Roy Morgan Poll results. And fancy the Herald allowing her to since it conflicts rather with their own poll.
    http://www.nzherald.co.nz/politics/news/article.cfm?c_id=280&objectid=11231762

    Bleneim • Since Aug 2008 • 135 posts Report Reply

  • JonathanM,

    David: Nice! A good start, but I don't think we want to simulate a random number of seats in parliament given the proportion of the vote :). Rather, we want to simulate a random election, given the uncertainty from the poll results.

    I've taken the liberty of updating it to operate within a Bayesian framework:

    http://pastebin.com/35XJmDXG

    Based on Colmar Brunton alone, there's a 73% chance of National victory, with 27% chance of Winston deciding. With a bunch of assumptions ofcourse, such as electorates for Act, Maori + UF and no electorate for conservatives.

    The nice thing about doing it in a Bayesian context is you can incorporate any uncertainty distribution you want (e.g. you could use realisations from Peter Green's bias-corrected poll of polls).

    Since Jul 2012 • 64 posts Report Reply

  • steve black, in reply to Andrew Robertson,

    All true. I should have added “or some other factor” and lack of time could well be it. I know that doing this sort of polling is rather like corporate advertising for Market Research companies. You do it to get your company name out there, not because it makes you any money.

    Parties polling below 1% are only going to be interesting if they are going to get a seat in Parliament. But in that case then it is going to be an electorate seat, which as far as I can tell is entirely outside the scope of this sort of polling which is reported in public. So one wonders why bother to report anything about them at the “party support” level and clutter up the graph? Why bother to read meaningful ups and downs of “party support” in the tea leaves for the “rats and mice” parties?

    That would leave space to properly report the refused and undecided responses which are very important and are probably more deserving of a line on the graph. That would make a change from the same old illegible lines across the bottom for those "rats and mice".

    sunny mt albert • Since Jan 2007 • 116 posts Report Reply

  • BenWilson, in reply to JonathanM,

    I would love it if more stats reporting was Bayesian. It's the only part of stats that really makes sense to me, because it answers the question you really want answered - the chances of things happening. Finding some mean/CI and framing some godawful backwards hypothesis test, and then reporting the answers off an arbitrary "very high, high, good, fair..etc" just makes me switch off. When someone says high, I always want to know how high.

    It also has the advantage that you can apply conditioning really easily to work out other interesting questions.

    Auckland • Since Nov 2006 • 10657 posts Report Reply

  • WH,

    I found the chart Sacha prepared quite helpful.

    I'd have thought there'd be at least some interest in a well-thought out election polling blog. It would involve analysing the data provided by the main news organisations, providing a rolling average, commenting on broad trends, critiquing poor coverage and providing insight on best practice in layman's terms.

    It might even be able to positively influence the way news organisations present polling data. In any case, Labour and the Greens have a real interest in making sure the implications of current polling figures are clearly understood.

    Since Nov 2006 • 797 posts Report Reply

  • Andrew Robertson, in reply to steve black,

    That’s a really good point.

    I suspect (know) that it would (seriously) annoy quite a few people to not have ‘their party’ in the graph – I get calls/emails from people expressing such for parties that are grouped in ‘other’. They feel as though we are disadvantaging their party by not reporting the result.

    There’s also an issue with putting the undecideds, which have a different base, in the same chart as party support – the percentages won’t stack up!

    You may laugh – but 'why don't the %s add to 100!?' is one of the most common concerns or queries I get from people when I report/present survey results :)

    Maybe a separate graph or table would be a solution.

    Wellington • Since Apr 2014 • 65 posts Report Reply

  • Pete George,

    I don't think sub 1% parties should be discounted. It makes a difference whether they are wobbling from 0-0.5 or wavering from 0.5 to 1. Small parties can move late in an election and it could make quite a difference whether Mana, ACT or UF look possible for picking up a list seat off the party vote.

    Gavin White says in his latest post:
    "Here's another angle on this - again assuming that they (Internet Party) don't win an electorate, has a party ever increased its vote from below 1% at the beginning of election year to over 5% by election day? In our first poll of 2002, United Future polled 0.3%, and they were below 1% in every poll that year until the election campaign, where they polled 6.7% in the actual election."
    http://sayit.co.nz/blog/what-chance-do-internet-party-and-conservatives-really-have

    And noting the appearance of the Internet Party (under 1%) and tracking their progress is of interest.

    Dunedin • Since Dec 2011 • 139 posts Report Reply

  • JonathanM, in reply to BenWilson,

    Finding some mean/CI and framing some godawful backwards hypothesis test...

    I think as long as you emphasise and discuss the mean (or effect size) and the CI rather than the P-value you're doing fine, but agreed in many contexts the Bayes approach tends to be more natural.

    It'd be great if the Bayes technique was taught more to undergraduates, but that may be challenging given the mathematical sophistication required - I'm not sure how much you could get away with skipping that like you can with frequentist stats.

    Since Jul 2012 • 64 posts Report Reply

  • Stephen R, in reply to WH,

    I'd have thought there'd be at least some interest in a well-thought out election polling blog.

    We could call it FiveThirtyEight.co.nz ...

    Wellington • Since Jul 2009 • 259 posts Report Reply

  • David Hood, in reply to JonathanM,

    Jonathan, Great (and a classic demonstration that posting something bad on the internet is a great way to get something better).

    Since I was going to add it to github, but you have done a lot of work on making it actually work, I was going to mark the project MIT licence (a basic attribution for using the code in other things) or Public Domain, do you have an opinion?

    Dunedin • Since May 2007 • 1445 posts Report Reply

  • Matthew Poole, in reply to Pete George,

    I don’t think sub 1% parties should be discounted.

    They only get lumped into "other" when they don't have an electorate seat and they're < 1%. ALCP, Libz, etc, they're never going to get enough votes to matter. UF, Act, Mana, they only break the rule because they have sitting MPs so every votes for them does count.

    Auckland • Since Mar 2007 • 4097 posts Report Reply

  • WH,

    We could call it FiveThirtyEight.co.nz ...

    I'm an electoral-vote.com user. It got me through the 2008 election and I like the simplicity of it.

    I'd be really interested to know more about how the parties go about trying to persuade voters to change their minds.

    Since Nov 2006 • 797 posts Report Reply

  • JonathanM,

    David: Sure, MIT license is fine - more than happy with folk to use it and improve it. :)

    Since Jul 2012 • 64 posts Report Reply

  • Kumara Republic, in reply to Matthew Poole,

    ALCP, Libz, etc, they’re never going to get enough votes to matter.

    The Libz definitely won't. They disbanded and threw in their lot with ACT (or what's left of it).

    The southernmost capital … • Since Nov 2006 • 5441 posts Report Reply

  • Kumara Republic, in reply to Stephen R,

    We could call it FiveThirtyEight.co.nz …

    Or should that be OneTwentyOne in the NZ context?

    The southernmost capital … • Since Nov 2006 • 5441 posts Report Reply

First ←Older Page 1 2 3 4 5 6 9 Newer→ Last

Post your response…

Please sign in using your Public Address credentials…

Login

You may also create an account or retrieve your password.