Posts by steve black
Last ←Newer Page 1 2 3 4 5 Older→ First
-
Hard News: Decidedly Undecided, in reply to Thomas Lumley,
Much more succinct than my raving, Thomas. Thank you.
-
Hard News: Decidedly Undecided, in reply to Ianmac,
Not an “anomaly” sensu stricto . It might be better to say that the results may just be within the expected range of variation given the survey methodology. When you take more than a simple random sample model of errors into account your more complete error model will widen the confidence limits. Market research companies only take the first error term from simple random sampling into account, and leave out all the other factors. Including possibly, a 50% refusal rate to take the survey (the survey non response rate). Never mind the non response (undecided or refuse) on specific questions. What about the vast number you failed to contact or didn’t want to take your survey? If those people are different in any relevant way from a true representative sample then your simple error estimates are optimistic.
Anybody from a research company want to tell us what your overall response rate is please? That is vital to know. As important as the percentage of undecided voters.
I like your null hypothesis. My alternative suggestion is that the “swings” from week to week (within a poll) or between polls, provide a better measure of true uncertainty in the system. The system in this case is what is happening in voterland plus how we conduct and analyze our surveys.
In other words, rather than explain the “swings” treat them all as examples of how much true variation is there. At the moment the pollsters and commentariat tend to “explain” changes based on thin air – because you don’t know why people change their voting ideas unless you have collected detailed data at the individual level regarding just that. Making up stories about why changes happen is just like the business commentators who make up stories about why share values went up or down yesterday. It isn’t based on real data, i.e. asking people why they bought or sold at the price they did.
In both cases (election polling and share market reporting) commentators simply look for a recent event which might be top of mind and “explain” changes using that. Maybe the stories are true, maybe not. But we don’t know.
Russell, my wife called me in to see your show but it was just winding up. I am very glad it is still available to view.
-
A friend (who shall remain nameless for obvious reasons) is a professional evaluator. She noted that as soon as the Nats took office from Labour, Government contracts for evaluation work pretty much dried up. One presumes the work was moved from professional evaluators into accounting firms, management consultants, and the economic and actuarial disciplines. Unless all evaluation of massive programme changes was simply thrown out the window to save money. Why bother when neoliberal economic theory proves you are right? Who needs actual evidence from other disciplines? What do "soft subjects" like public health and epidemiology possibly have to contribute? ;-)
What about also appealing to the Government Auditor on the grounds that if transparency is not there on evaluation criteria nobody can possibly know if the programme changes worked or not? Saved taxpayer money or cost taxpayer money?
-
Hard News: The Language of Climate, in reply to bmk,
My other big problem is the way reporters feel like they are being balanced by doing an article on climate change. They interview a scientist saying it’s happening; so for ‘balance’ they go to a scientist who says it’s not happening. Then they finish the clip. This leaves the impression that the science is divided.
I've puzzled over how we could "assist" the broadcast media to come up with a way to indicate the strength of numbers behind the two scientists they interview when they do the old "in favor" vs "opposed" one per view notion of balance. Maybe they could always present a graph showing the relative sizes of the two groups behind them as they speak if there is video. I though of tweaking the mic volume to represent the relative number of mainstream scientists (100,000) to the skeptics (100)* but then the skeptics would be completely inaudible. Nice, but probably not a workable solution just yet...
* just making up numbers of course, it might be much more extreme.
-
Back to the original theme about the language of climate. One thing I've noticed is that when Climate Skeptics talk about it they often say "Global Warming" and when other people (especially scientists) talk about it they often say "Climate Change". Anybody else notice that?
I blame the MSM for simplifying things down in headlines to "Global Warming" and I blame naysayer politicians and skeptics for using the opening which the MSM gave them. Talk of "Global Warming" is often a prelude to mentioning things like "but we've had the coldest winter ever" or "but the average global temperature isn't really going up".
I try and keep the focus on variability. If the average temperature stays the same, but we have higher high temperatures by 50% and lower low temperatures by 50%, we will get fried and frozen by turns. All with no change to the average. Ditto for wet and dry periods. We'll be desiccated and flooded by turns. It is a statistics thing, but as we know the MSM often have a bit of a problem with the relevant concepts like variability.
-
Hard News: Gower Speaks, in reply to JonathanM,
6. All of the above + other stuff
Just a partially formed thought:
Could “other stuff” include a different model which postulates that people move in or out of main party support (lumped together – although not in voter’s minds!) versus minor party support (lumped together) depending on what they currently perceive as the “strategic choice” to achieve what they want out of the election? For example, some (what %?) of major party supporters might move to be minor party supporters when polls “tell them” that the Nats can govern alone thus it is "safe" to go minor.
I haven’t worked through all the consequences of such a model yet. I’ll rely on younger faster brains to work out of the model might or might not be interesting.
-
Hard News: Gower Speaks, in reply to JonathanM,
The average house-effect for the Greens from Peter Green’s poll of polls is 0.46%, so not as high as other parties, but around 1/2 of a seat. Roy Morgan is at 1.2% and Fairfax Media Ipsos at -0.2%
I might have missed discussion of this, but are there any references you (or others) can point me to which discuss these house effects in more detail? I'm getting curious.
For example, are they consistent for a given research house over some years? How consistent? Could some sort of model incorporating randomness account for them? Presuming not...on to:
Has anybody put forward some info or ideas on what methodological aspects might give rise to house effects (aka bias in the technical statistical sense) of this magnitude and consistency? Are we looking at questionnaire design? interviewer training? sampling methodology? interview methodology? number of callbacks? weighting scheme? Curiouser and curiouser, cried Alice.
-
Hard News: Gower Speaks, in reply to bmk,
In reality what might be happening is Labour is taking 2% off National while Greens are taking 2% off Labour. Unless you had a panel you regularly poll and track the changes there would be no way of knowing.
Bingo. I’d call it a longitudinal study, market research companies call them panels. Basically you have to track the same individuals to see who moves in and out of being undecided vs party support for a particular party. Otherwise we’re just interpreting “ecological correlations” (that’s what I call them, but they will have other names in other disciplines) which is a good example of why you hear “correlation doesn’t imply causation”. Don’t get me wrong, you can use all sorts of techniques for prediction quite successfully, but that doesn’t get you strong evidence for the “why” questions we talk about in our narratives.
I seem to remember that in the studies of the 1999 and 2002 elections Vowels, et al. followed a panel (among other data sources). I’ve just gone and grabbed them off my shelf. I’ll have to have re-read them and get back up to speed.
On another note, I always bridle upon hearing "in a democracy people get the government they deserve". Well, I don't think so. I usually end up getting the government other people deserve. I deserve better. And under current scenarios I may end up getting the government Winston deserves.
-
Hard News: Gower Speaks, in reply to David Hood,
It’s hard for me to read the titles of the bars which say “w/o NZF” on my iMac with 21.5” screen. But I appreciate you are suffering from somebody’s “law of graphical displays” which says there are never enough pixels…
And on the subject of graphs…how did you put your graph inline like that? Using attach? And does it have to be a png format? Will a jpg do? (just testing things out and I see a jpg will do)
Is there some sort of documentation I’ve missed which tells us a bit more? Like how to get my post text both above and below the attachment?
-
In terms of better data presentation and analysis I just couldn't resist re-posting this which just came my way.
http://www.statschat.org.nz/2014/04/05/oh-what-a-difference-a-bit-of-formatting-makes/
I liked your graph with ranges Sacha. You get my vote (and assistance if you want it). I'm way behind in my R and nice graphs skills, but I've always meant to polish up.
