Hard News by Russell Brown

Read Post

Hard News: Briefing, blaming, backing down

191 Responses

First ←Older Page 1 2 3 4 5 6 8 Newer→ Last

  • Hilary Stace,

    Giovanni has just posted on Roughan's column.
    http://bat-bean-beam.blogspot.co.nz/2012/06/un-educated.html

    Wgtn • Since Jun 2008 • 3226 posts Report Reply

  • Bart Janssen, in reply to James,

    These numbers are being quoted as if they’re regression coefficients

    What's worse is these numbers are being quoted as if they are real. I personally think the meta analysis is likely to be shown to be a crock of shite. There are some serious red flags in the list that suggest the analysis has created something odd.

    Auckland • Since Nov 2006 • 4460 posts Report Reply

  • dc_red, in reply to Russell Brown,

    Can I just say that it’s telling and admirable the way you’ve all gone for the actual policy issue, rather than pontificating about who won and lost?

    The Union for Pontificators might have something to say about us taking Duncan Garner's job. That man's passion for declaring winners and losers - usually based on unstated criteria of his own invention - knows no bounds.

    Oil Patch, Alberta • Since Nov 2006 • 706 posts Report Reply

  • Ian Dalziel, in reply to Kumara Republic,

    the roads to nowhere...

    Now if they can at least do an audit on the Roads of National Significance, they’ll go some way to earning back some respect

    Well maybe not an audit, just a dose of reality...

    Mr Brownlee said of the briefing paper that it indicated the Government was "progressing a very aggressive infrastructure problem"* but the agency had simply sounded a cautionary note about completion times.
    "The Government's expectation is that NZTA [the agency] will expedite these projects as they have been requested and required to do, so I can't see at this point any reason why we would expect a slippage in the early part of the programme."

    *this could be the catch phrase of a whole new Road Sanity campaign
    - bring on the "Ghost Jobs" too...

    Christchurch • Since Dec 2006 • 7950 posts Report Reply

  • Paul Williams, in reply to Tristan,

    So class size is number 40 but feedback, direct instruction and early intervention are in the top 10 you know all the things a small class size gives you the the opportunity to do… FFS you couldn’t make this up…

    Is it that clear? I'm not arguing that the key factors aren't inter-related, and your point makes intuitive sense, but if the list was compiled following a study of classroom effects from a wide variety of different sized classes then it doesn't follow that the size of the class determines whether or the top ten factors occur.

    Sydney • Since Nov 2006 • 2273 posts Report Reply

  • Ross Mason,

    Here are a couple of paragraphs from the conclusion of Hattie's paper.

    When addressing the issue of reducing class size, it seems important to investigate the underlying motivations for teachers and parents. For example, the synthesis of meta-analyses indicated that the presence of disruptive students (even one of them) in a class has the effect of decreasing achievement by 0.79—which is enormous. When I ask teachers if they would choose between a class size of 15 when I choose the students, or a reduction of 5 from their current class and they choose the students, they nearly always prefer the latter. For many teachers, it is the presence of a few disruptive students that often lead them to desiring smaller classes. There is a question also about the optimal class size; although there seems to be some “magic” in the literature and among policy makers around a class size of 15. When asked in a survey as to the optimal class size, New Zealand secondary teachers claimed that 16 was optimal for Year 13 (the final year of school), 19 for Year 12, 21 for Year 11, and 23 for Years 9–10 (which is not that different from the actual class sizes in NZ secondary schools)

    .....

    The argument in this paper is that those teaching practices that are conducive to successful learning are more likely to occur in smaller rather than larger classes, and these practices do not actually occur more in smaller classes because teachers have been prepared to, and indeed do, work with larger classes using more transmission practices and therefore they are not so equipped to adopt the more effective practices when they are given smaller classes. (A related argument is that research reviews and/or meta-analyses need not be based on finding a single effect-size which is somehow expected to speak for itself, but that previous research findings can be developed within a theoretically informed model of how the effects may be improved.)

    It is conjectured that a way forward in this class size literature is to investigate the classroom and curricula attributes in various levels of class size, the nature of teaching excellence at each of these levels, and the necessity to dramatically alter the concept of teaching excellence when class sizes are changed. Without changing the teaching and ensuring rigor in the curriculum delivery then the effects of this most expensive policy is likely to be close to zero. Maybe identifying teachers who can adopt the nature of teaching that is excellent for smaller classes is the first step, as such teachers could make major and positive effects on student learning: whether this is from 50 to 30, or from 30 to 15. These teachers could work within current schools, although issues of fairness and workload for teachers and students would need to be addressed, and the contingencies for all teachers would need to move from working conditions alone to also embrace positive student learning outcomes in a very public defensible manner.

    One of the dilemmas for policy makers, however, is that educational research such as outlined in this paper has had little effect on the lobbying by parents, teacher groups, and politicians for the reduction of class sizes (Achilles, Krieger, Finn, & Sharp, 2003). Further, it is common practice for private “elite” schools to advertise smaller classes as a bonus and feature of their schools, with clear market research showing that this is an attractive feature. Teachers and parents are more convinced that if the working conditions for teaching and learning appear optimal then it is “logical” that benefits must follow, and they seem less convinced of the evidence of the effects of these working conditions, often becoming dismissive of researchers who show evidence to the contrary. If a legislature decides to pour the millions (and cumulatively over a short time, billions) of dollars into reducing class sizes, there are two strategies that may add value to the learning outcomes well beyond the expected 0.13 standard deviation increase in learning outcomes (achievement or non-achievement). First, it is important to identify those teachers who employ excellent teaching methods optimal for smaller classes as outlined above, and provide them with smaller-sized classes (perhaps with many different cohorts of students per week). This introduces the concept of “specialist teachers of small classes.” At the same time it is important to acknowledge the excellence of teachers of larger-sized classes, and it likely that there are major issues of “fairness” that will need to be addressed. Second, it is important to identify 3–5 major innovations (e.g., reducing class sizes, reducing class sizes for teachers who teach in a manner as noted above, providing a day per week free for teacher planning and marking, introduction of reciprocal teaching and maximal feedback, etc. See Darling-Hammond and Miles (1998), and Odden (1990) for further possibilities), and set up a comparative research design to assess the effects of these innovations on student learning. Random assignment (as in Project STAR), comparative benefits (as outlined by Levin (1988) and Jamison et al. (1974) noted above), and the use of effect-sizes relative to costs and teacher workload, etc., could be powerful elements, particularly if the study is conducted as a public debate about the benefits of these innovations. Such a debate would inform parents and politicians of the relative effectiveness of the various innovations and it is extremely likely that they could be convinced by the results, thereby moving from seeking different conditions of learning to desiring optimal “effects” on their children. Perhaps the major way forward for any innovation that is as costly as reducing class-sizes is to move the debate from asking does it positively influence student achievement (as Table 2 above showed that almost everything can pass this test) to asking does it positively influence student achievement more than other interventions? The contingencies need to move from focusing not only on working conditions for teachers and students to also focusing outcomes deriving from these working conditions (Hanushek, 2005).

    It is obvious that this paper has been influencial in the govts argument. I can't help thinking that if they had left it at enhancing the quality of teaching and teachers (and Hattie's argument of getting teachers to change their teaching of small classes) without having to find the treasury quid pro quo of "finding savings to justify it" then they would have had the buy in of all parties.

    Upper Hutt • Since Jun 2007 • 1590 posts Report Reply

  • linger, in reply to Bart Janssen,

    the analysis has created something odd.

    Not really; it has created something predictable – the largest effects are identified for the factors that are most consistently associated with better learning outcomes (assuming for the moment that those were measured appropriately; that’s another can o’ worms). We should expect that the more specific factors, which have a more direct causal connection, will be more consistently associated with outcomes than class size per se. To that extent, the results are not “odd” at all.
    But (and this is where the policy response was so deeply flawed) it does not follow that class size is unimportant, since many of the more specific factors listed are only easily applicable in smaller classes.

    The basic methodological problem is that, given the likely connections between factors (which are recognised by Hattie), a simple regression model should not have been used; a structural equation model would be more informative (though considerably more complicated). Hattie's interpretation of the results does not entirely ignore the problem, but it does suggest he is not fully aware of the fundamental limitation of his statistical model.

    Tokyo • Since Apr 2007 • 1942 posts Report Reply

  • David Chittenden, in reply to Keir Leslie,

    Which is a highly political viewpoint.

    I think it's more a case of where ideology and the tools of analysis considered acceptable by Treasury (or even mandated in legislation, public service guides etc) and politics overlap, rather than Treasury being political. (Maybe I'm giving them too much credit.) Examples of tools of analysis are our beloved cost-benefit analysis and the % rate used in return on investment calculations. I worked for a bit on climate change and the minister was always pushing for low discount rates ('social discount rates', say 3% or less) because climate change is such a long term issue. Using rates of 7% (or whatever it was) makes absolutely no sense and has a huge impact on the supposed benefit of a policy, but Treasury loved to hold the rule book on that one.

    Fundamentally, the structural set up of the New Zealand civil service is fucked, and has been since the '84 Labour government reformed it. It will need to be fixed, at one point or another.

    Absolutely agree (see my earlier post), but here the biggest culprit is the State Services Commission. They have the mandate for improvement but seem to see absolutely nothing wrong with it and unfortunately that goes for most public service workers as well in my experience. That said, there are always a number of agitators working within the system pushing for change - and that goes for within Treasury as well. Unfortunately, most get frustrated and don't stay long but there are some who stick around and don't just get sucked into the system.

    The role Treasury has had here is deeply concerning. It raises real questions about Makhlouf's job.

    I hate to say it but I think this is completely normal Treasury behaviour (again see my earlier post). Change is definitely needed but it's systemic change. Removing Makhlouf alone would achieve nothing.

    Since May 2011 • 31 posts Report Reply

  • Cecelia, in reply to Ross Mason,

    That's really interesting. It's much more subtle than "class size has been proven" blah, blah, blah.Interesting that no one has taken up the "disruptive student" issue - that is a major factor in our lower decile schools. A relative of mine is sending his very bright daughter to a Catholic school because they fear "drugs" in the local intermediate. I'm sure there are drugs in every school but the disruptive student factor will worry me when my grandchildren go to school. I went to a public school myself, taught in public schools and sent my children to one but now I'm not so sure. What has changed?

    Hibiscus Coast • Since Apr 2008 • 559 posts Report Reply

  • David Chittenden, in reply to Rich of Observationz,

    There are economists who don't go along with the Treasury orthodoxy. In fact, NZ has a particularly hard-core right-wing monoculture amongst economists, compared to other nations. It would make sense for the next left-wing government here to repeople Treasury from overseas.

    Thanks for that - a good read. On austerity, have you seen these guys? Be Outraged - There are alternatives! (pdf)

    Yeah I agree. We bought the neo-liberal agenda unbelievably well in NZ and it permeates even us who are against it. I spent some time in Sweden and Norway and after finding out how Norwegians would build hugely expensive undersea tunnels to islands with 50 inhabitants, even I was shaking my rational economic head. (But they do have lots of oil money and that's how they want to spend it.)

    I think there are many good people in NZ as well, but yes, some fresh blood would be good. We need to reform the mandated tools of policy analysis (see another post of mine) and there are some amazing, but often complex, tools out there. We need to seriously train public servants to use them and we need to make departments responsible for other department's outcomes, as well as their own. This would really help to get beyond the unbelievably stupid situation we have now where we can either have an economy or an environment. (And there are other examples ...)

    Since May 2011 • 31 posts Report Reply

  • merc, in reply to David Chittenden,

    This would really help to get beyond the unbelievably stupid situation we have now where we can either have an economy or an environment. (And there are other examples ...)

    The setting of false dichotomies, http://en.wikipedia.org/wiki/False_dilemma - mirrors Calvinism, http://en.wikipedia.org/wiki/Calvinism - there is that streak among us.

    Since Dec 2006 • 2471 posts Report Reply

  • Kumara Republic, in reply to David Chittenden,

    We need to seriously train public servants to use them and we need to make departments responsible for other department’s outcomes, as well as their own. This would really help to get beyond the unbelievably stupid situation we have now where we can either have an economy or an environment. (And there are other examples …)

    I’m tempted to think a half-decent policy would be to drop the guilty parties in a Sao Paulo favela for a few weeks, to give them a crash course in what disparity means.

    It’d either be the catalyst for a Damascene conversion, or it’ll reinforce their inner Randy Weaver.

    The southernmost capital … • Since Nov 2006 • 5441 posts Report Reply

  • Martin Lindberg, in reply to merc,

    false dichotomies ...mirrors Calvinism, http://en.wikipedia.org/wiki/Calvinism

    What aspect of Calvinism you are referring to?

    Stockholm • Since Jul 2009 • 802 posts Report Reply

  • Russell Brown,

    So a few people have kindly offered donations towards the cost of this week's upgrade of Public Address. I was going to give that a swerve, but have had some news that changed my mind.

    If you're so moved -- and don't feel obliged to -- you can throw a little money in the hat here:

    http://pubadr.es/7942

    Auckland • Since Nov 2006 • 22848 posts Report Reply

  • David Chittenden, in reply to Kumara Republic,

    It’d either be the catalyst for a Damascene conversion, or it’ll reinforce their inner Randy Weaver.

    My thoughts exactly, but not as artfully articulated!

    Since May 2011 • 31 posts Report Reply

  • merc, in reply to Martin Lindberg,

    Predominantly the 5 points and how they are based upon being in favour or out with regards to God's favour.
    It's an oblique and esoteric point, and one that I find difficult to discuss, particularly in the South Island.
    With regard to education systems it may be pertinent in part due to the configuration of our schools historically along religious lines.
    Postnote, I have been to a favela in Sao Paulo, there is nothing to prepare a Kiwi for the sheer numbers of people living, effectively under cardboard, ruled over by swords, machine guns and then, deadly inflation.

    Since Dec 2006 • 2471 posts Report Reply

  • Hilary Stace,

    Wrote this letter to the ed on 30 May (not surprisingly not published)

    The National government has an admirable aim of lifting student achievement, and has come up with three solutions: national standards, performance pay and increasing class sizes (in exchange for more professional development). They have also plunged into the intractable world of 'wicked' problems.

    As part of my public policy PhD I studied 'wicked' problems. These are issues that are complex and resistant to straightforward solutions. There is often little agreement about what the problem really is, or how to approach it, and attempts at resolving one area often lead to unintended negative consequences elsewhere.

    My research suggested one vital approach in tackling wicked problems: policy people and politicians need to build good relationships with all parties involved, including those who have 'lived experience' of the issues. The wisdom of those who are the targets of the policies is invaluable in formulating solutions that will work in the real world.

    However, in developing and implementing these policies the National Party has shut out those with front line experience - the teaching profession - and neither have they engaged with students themselves.

    So I would not be surprised if these attempts at lifting student achievement fail.

    Wgtn • Since Jun 2008 • 3226 posts Report Reply

  • David Chittenden, in reply to Hilary Stace,

    As part of my public policy PhD I studied 'wicked' problems. These are issues that are complex and resistant to straightforward solutions. There is often little agreement about what the problem really is, or how to approach it, and attempts at resolving one area often lead to unintended negative consequences elsewhere.

    I'm always excited when other people get this stuff. :)

    It was one of the reasons I enjoyed (most of the time) working in the public service - you get to work on really challenging stuff. Unfortunately though, what I found most challenging was getting the system to understand this and letting you have a decent crack at it.

    I'd also add that with our very interconnected world these days most policy problems are 'wicked' problems.

    Since May 2011 • 31 posts Report Reply

  • Bart Janssen, in reply to linger,

    We should expect that the more specific factors, which have a more direct causal connection, will be more consistently associated with outcomes than class size per se. To that extent, the results are not “odd” at all.

    But the only way you would get such a large difference between class size itself and some of the factors directly linked to class size is if decreasing class size had some negative component as well. Especially if multiple factors dependent on class size have a positive effect, as in this case. I agree there will be aspects related to class size that should have a higher impact but not to the degree seen in the analysis.

    This suggests to me that the data is a merge of multiple studies where most studies measured class size but only a few studies measured some of the other factors. In order to present all those factors in one table the data would need to be processed. Without very careful data processing such differences could (and my guess in this case have) distort the findings.

    But I really don't know. I'm not an expert in education nor is my stats good enough to tackle meta analysis. However I have seen numerous meta analyses fall apart under scrutiny after they were published making me very nervous of such approaches. Personally I'd prefer to treat the individual studies as independent. It takes more effort to assess the conclusions of independent studies and more time but in my experience it is well spent.

    As the excerpt that Ross posted shows Hattie is not unaware of some of the issues indicating eg that the low impact of class size may be due to failure to alter teaching practices to take advantage of smaller classes.

    Auckland • Since Nov 2006 • 4460 posts Report Reply

  • Sacha, in reply to Ross Mason,

    The contingencies need to move from focusing not only on working conditions for teachers and students to also focusing [on] outcomes deriving from these working conditions

    what does that mean?

    Ak • Since May 2008 • 19740 posts Report Reply

  • Scott Chris, in reply to Ross Mason,

    It is obvious that this paper has been influencial in the govts argument.

    Maybe. Not sure what the government's argument was.

    Still, what Hattie seems to be saying is that there is no sense in putting the cart before the horse. For smaller class sizes to have a substantially positive effect on learning outcomes, first teaching practice has to be modified. Hattie's suggestion of an incremental approach to this transition phase makes sense to me.

    Auckland • Since Feb 2012 • 167 posts Report Reply

  • Scott Chris, in reply to Sacha,

    what does that mean?

    It means it's all very well having smaller class sizes but if they make no difference to learning outcomes then they are a waste of money.

    Auckland • Since Feb 2012 • 167 posts Report Reply

  • Hilary Stace, in reply to David Chittenden,

    Yes lots of wicked problems around these days, and people usually roll their eyes when you bring up the, now old hat, stuff about 'wicked' problems. But I like the analysis as it really challenges hierarchies, power relationships and encourages risky new ways of doing things. I particularly like that it provides an opportunity for those with lived experience to be brought into the process and have that expertise respected and valued.

    For example, with the student achievement thing, the real experts are the students themselves so why not get some engagement process run by and for those in the so called tail, and everybody else just listens, writes or acts on what they suggest.

    Wgtn • Since Jun 2008 • 3226 posts Report Reply

  • slarty,

    May I just say how bloody awesome this discussion is? Imagine if our parliament were so erudite, intelligent and plain polite.

    I particularly like the Wicked Problems. Dworkin humanised.

    Since Nov 2006 • 290 posts Report Reply

  • Jane Pearson, in reply to slarty,

    +1

    Since Feb 2010 • 28 posts Report Reply

First ←Older Page 1 2 3 4 5 6 8 Newer→ Last

Post your response…

Please sign in using your Public Address credentials…

Login

You may also create an account or retrieve your password.