Thank you for producing and sharing the code David and Jonathan. I'm installing R now on my home laptop (already got it at work and am familiar with R although hardly fluent in it) so that I can play out some scenarios on it.
Good offer Russell. What would be good is a person (or preferable persons) available to post (here or a specialist site?) an expert analysis of every poll. Encourage anyone to promote this information as a reference alongside any poll comments/posts. Get the information out there for anyone who's interested and promote it to journalists to improve their working knowledge of poll analysis.
One thing I'm targeting is promoting and linking to quality information in any forum I see discussion on a topic where I know a source of detail or expert opinion.
A strength of a maturing traditional/social media is the interchange of information and promotion of informed discussion.
I just tried running a simulation using the numbers from the latest bias adjusted tracking poll rather than the latest poll. This gave it as 45.9% chance of National and NZF deciding 54.1%.
I think unless something major happens between now and the election once again Winston will be deciding where the country goes - and I think the left will be disappointed if they expect him to support them. While I would want him to go with Labour; I really don't think he will. If it were just Labour then it would be a realistic possibility but he won't get along with the Greens and he doesn't want to be the third party in the coalition.
The only way I can see a left-leaning government at the moment (this is assuming no significant shift in support) would be with a Labour-NZF coalition that excludes the Greens but somehow they buy their support on supply and confidence. This would lead once again to the Greens being shafted by Labour (see both 2002 and 2005). And with some parties wouldn't be possible but with Labour saying 'If you don't agree to it then it's three more years of National and anything is better than nothing.' - they may cave and roll over. It's still a very unlikely outcome compared to a National-NZF coalition.
I'll try to translate that:
Thank you Russell. I want someone to post an expert analysis of every poll, and for other people to refer to it. And for journalists to understand poll analysis. Discussion is good.
If anyone thinks 'PeteGeorgeSays' is a valuable service with guaranteed funding, do let me know. #righto
I was thinking about this overnight.
FiveThirtyEight's prediction that the Republicans will take the Senate in 2014 really shows how emphasising certain elements of the status quo can put you in a conservative, even defensive, mindset.
Given that positive change is a definitional goal of progressive politics, persuasion has to be at the core of what our parties do. Maybe we could crowdsource the Labour/Greens election strategy.
In terms of better data presentation and analysis I just couldn't resist re-posting this which just came my way.
I liked your graph with ranges Sacha. You get my vote (and assistance if you want it). I'm way behind in my R and nice graphs skills, but I've always meant to polish up.
There was (and is still until the PR is in) a bug in my code, which meant that parties under the threshold without an electorate weren't ignored. As this most likely means the Conservatives get counted when they shouldn't, the result is quite important.
With Colmar Brunton the fixed result is 47% Nats, 53% Winston and a small chance (about 0.05%) of either labour-led or hung.
Fixed code available here:
I suspect things will be quite different with the house-effect adjusted poll of polls - I'll see if I can get a routine written up that samples a bit better from that.
While I would want him to go with Labour; I really don’t think he will. If it were just Labour then it would be a realistic possibility but he won’t get along with the Greens and he doesn’t want to be the third party in the coalition.
We'll see on that. He's a real wildcard. The two blocs will have to haggle with him about what he gets. It might depend on who can swallow the biggest Winston bribe.
But we're quite a ways out from the election. Support's going to move around some more. It's still anyone's game.
a small chance (about 0.05%) of either labour-led
Labour-led meaning "not requiring (but possibly including) Winston" here? I ask because "Winston decides" surely includes an unknown chance of Labour led government?
I ask because “Winston decides” surely includes an unknown chance of Labour led government?
I'd like to think that with NZ Firsts state assets policy alone, that they can find room to compromise with Labour and Greens to change the govt and or a mix with Mana and Internet if necessary. The point being, all these opposition Parties need to have a vested interest in getting along (that's MMP) and an understanding that compromise is inevitable if we want change. I'm going to persevere with emails to all of them even if I never get asked in any polls, I can sure as hell tell all opposition parties I want them to compromise to get rid of this current lot. hrrumph!
I ask because “Winston decides” surely includes an unknown chance of Labour led government?
It is either a Labour or National Government- it is defined by the difference between the two blocks (sans Winston) is not greater than the seats NZF have. It makes no judgement about what Winton would do- he could go with National or Labour or allow a minority Government. So yes, that percentage does include a Labour led government with NZF support (it just makes no judgement as to how likely that is within the 53%).
It’s hard for me to read the titles of the bars which say “w/o NZF” on my iMac with 21.5” screen. But I appreciate you are suffering from somebody’s “law of graphical displays” which says there are never enough pixels…
And on the subject of graphs…how did you put your graph inline like that? Using attach? And does it have to be a png format? Will a jpg do? (just testing things out and I see a jpg will do)
Is there some sort of documentation I’ve missed which tells us a bit more? Like how to get my post text both above and below the attachment?
From the Horizon poll:
If there is a coalition government after the next general election, which main party would you prefer to lead it?
Answers from those who gave their party vote to NZF in 2011 (n=230): Labour 73%, National 27%
Incidentally, for anyone think "I should learn R" it is one of the major languages used in free online courses about data, for instance at Coursera (the Massive Open Online Courses site). Generally you would need 8 to 10 hours a week for a lot of the courses (slightly more if you haven't done this style of course before and are out of practice with learning)
The John Hopkins ones all use R (though the Toolbox one covers things like sharing code on Github) and have a Learn for Free button. They also are intended to run every month, so one wouldn't have to join the intake for the R Programming course starting tomorrow for example.
The text is on the graph as part of the image- in the latest update on
it is the barplot with the setting main="" adding the text above the graph, the setting sub="" adding text below the graph
I actually did some making graphs in R examples for something else a few weeks ago (though it didn't include main and sub) over here
it just makes no judgement as to how likely that is within the 53%
That would be an awesome conditional probability to be able to get. Then we could work out the overall probability of National-led govt.
Just for fun since I'm swotting this stuff right now:
P(N) = P(N|W)P(W) + P(N|~W)P(~W) (Bayes)
where N="National led govt", W="Winston holds balance", and ~ means "complement of", and | means "given" and
P(~W) = 1-P(W) = 0.47
So if P(N)=0.5, P(N|W)=(0.5-0.22)/0.53=0.53
Thus, if the original probabilities are correct, Winston only needs to be more than 53% likely to choose National, for National to have a better than even chance of winning the election. But even if Winston has a 90% chance of going with them (so P(N|W)=0.9), National still only has a 0.53*0.9+0.22=0.70 = 70% chance of winning the election, and even if he's absolutely certain to go with them, the best chance they have is 75%.
Ben, while Winston deciding is an outcome, it is not a government, so if you were trying to work out the government, I think it would look more like this.
Which under the Colmar results put national anywhere between just under 47 and just under 100 depending on how Winston breaks.
It is a probability that can shift pretty easy if the sides equalise though, as they did in the Roy Morgan
@BenWilson: According to the output from Peter Green's poll of polls, P(W) = 1. Nats + Winston are then around 10 seats clear.
In the scenario analysis that I've done from the poll of polls, it indicates that the Nats have a 23% chance of going it alone* without Winston (even when he gets in), whereas Labour's only shot is with Winston, and even then they only make it over the line 66% of the time.
*alone here means Nats + Act + UF + Maori.
I'm not sure if there has been much entertaining as to whether Maori will go one way or the other. If they switched to going with the left then the numbers above reduce to 1% chance of Right alone and 98% chance of Left+Winston. Still no possibility of Left alone though!
and even if he's absolutely certain to go with them, the best chance they have is 75%.
I think you've gone wrong somewhere in that the labour clear win chance was 0.005. If Winston is absolutely certain to go with National then when you combine the National clear win and the Winston decides portions together you should get around 100 - 0.005 (Labour win) - hung parliament (??). Whatever the exact numbers are if you decide that NZF is 100% going with National then National would be well over 75% likely to win the next election.
OK, so P(N|~W) is not 0.47 in those figures. It's 0.995. Which makes bmk correct.
Presumably these calculated percentages are based on the idea "if the election was held now". The only uncertainty is whether the polls have accurately captured what the population wants, and we get all our combined probabilities from that?
I ask because it obviously isn't going to be held now, so extrapolating the uncertainty of an electoral outcome some way into the future just as the uncertain in current polling data strikes me as quite bogus. That's like saying that the only uncertainty of the price of the NZD/USD at the time of the election is the uncertainty we have in it right now. In other words, we know exactly what it will be (which we obviously don't).
What I'm saying here is that the actual uncertainty calculations we're making about the polling data might be applicable right before the election, but at this point they're not. The polls will go up and down as they have over the last 6 months. The true variance in all the outcomes much be considerably more, the further out from the event we actually are.
Makes me think that modeling it as a random walk might make more sense, when making the long range predictions. I most certainly can't believe that the odds of a National government are currently 99.5%. If we had these polls the day before the election, then I'd believe it more.
You are right that the polls are only predicting if an election were held tomorrow - not the actual election. They show the current situation rather than what will happen on the actual election.
That being said the current polls don't give National a 99.5% chance of governing - the 99.5% figure only comes if you factor NZF as being 100% likely to go with National. If you say NZF is 50/50 between Labour and National then you get a likelihood of around 75% for National governing - which is curiously enough about where iPredict puts them.
And I think that's a fair reflection of the current reality. But the election is still a long way away and those numbers are bound to move between now and then.
Do the Greens steal votes from Labour? Does NZ First steal votes from National?
If we look back at the polls (rather than use the output from the simulator from poll of polls, as we know they’re independent with the exception of National due to the way it predicts) then we can assess the dependence between the parties.
Also, prompted by David’s comment on github about how we might show the lack of independence between the number of seats of left and right, I had a little play, the result of which is here: