Saturday, September 1, 2012

The Insanity of the Marzano Evaluation System...

You have to laugh at the state of education in Florida (and to some extent, the US).  While NAEP scores continue to rise throughout the country for 30+ years, you still have people screaming for the latest fad to save us.  In the case of Florida, they bought into Marzano's "causal" framework.

Let's start with the most obvious problem:  it's ridiculously convoluted.  If I were asked to create a parody of an evaluation system, I couldn't possibly make something more hilarious than Marzano.  Even they state it "works best" with a year of planning and training.  What the hell kind of evaluation system requires a year of EITHER, yet alone both?  (Answer:  A very bad one.)  If you are going to invest this kind of time (and the resources) into a program, you better have extremely high expectations for it.  Sadly, pretty much everyone with a modicum of intelligence knows that this is not going to change anything (yet alone significantly for the better).  It shouldn't be necessary to have to tell ostensibly well-educated people this, but a simpler (and thus more easily better understood) system will work better than a complicated one.

How many "elements" (items) are in it?  The answer is 60, in four "domains" (edu-speak for "areas"; you can't charge $23/book for using simple language!).  The evaluation system would be better as a bulleted list of "suggestions"--it would have saved millions of dollars.  In that role (a list of "good suggestions"), Marzano is just fine.

But, surely it will be massively successful; after all, Marzano's system has been evaluated by none other than Marzano himself--just check out Research Base and Validation Studies on the Marzano Evaluation Model, April 2011.

To quote this (presumably) non-peer reviewed work, "[The] Marzano Evaluation Model is based on a number of previous, related works that include: What Works in Schools (Marzano, 2003), Classroom Instruction that Works (Marzano, Pickering, & Pollock, 2001), Classroom Management that Works (Marzano, Pickering, & Marzano, 2003), Classroom Assessment and Grading that Work (Marzano, 2006), The Art and Science of Teaching (Marzano, 2007), Effective Supervision: Supporting the Art and Science of Teaching (Marzano, Frontier, & Livingston, 2011)."

That's right--you know Marzano's framework will work because it's based on Marzano's work!  (Also, Phillip Morris would like you to know that smoking increases your libido, cures cancer, and prevents male-pattern baldness.)  Check out the section entitled The Research Base from Which the Model Was Developed:

"Each of the works (cited above) from which the model was developed report substantial research on the elements they address. For example, The Art and Science of Teaching includes over 25 tables reporting the research on the various elements of Domain 1."

I don't believe I've ever seen the number of tables in a book cited as evidence of the book's quality, i.e., "You know it's good because there's a lot of it."   Apparently taking lots of (often old and often poor-quality) semi-related data and mashing it together makes good research.  Only in education research is this even remotely possible.

10 comments:

  1. "The evaluation system would be better as a bulleted list of "suggestions"--it would have saved millions of dollars. In that role (a list of "good suggestions"), Marzano is just fine."

    This is exactly right. Many of the mandated garbage in the education system fits this description (LFS immediately comes to mind).

    ReplyDelete
    Replies
    1. It's sad, especially with recent budget shortfalls, that the politicians continue to push this worthless--and expensive and time-consuming--garbage onto us.

      Had to look up LFS; fortunately, my district (or at least my school) has been spared this bunkum for now.

      Delete
  2. I keep hearing "only 2% of teachers are innovating/highly effective" according to Marzano. Sounds like it was more like an investment on the part of those who were scared about paying any merit pay.

    ReplyDelete
    Replies
    1. I only found one reference to the "2%" and that was in his "Race to the Top" "whitepaper" (read: Paper that looks academic, but really isn't any more so than this blog...in fact, given that it's Marzano, I'd say my blog is probably more credible simply on the basis that any positive number is greater than zero). In it they state,

      "Marzano (2009) has noted that a teacher who is classified as “most effective” (i.e., at the 98th percentile in terms of his or her pedagogical skill)
      will be expected to produce student achievement that
      is 54 percentile points higher than the achievement
      produced by a teacher who is classified as “least
      effective” (i.e., at the 2nd percentile in terms of his or her pedagogical skill)"

      A 54 percentage point increase? That's just laughably unrealistic. One of these days I'm going to have to slog through some more of Marzano's pseudo-research (aka "meta analysis") to try to figure out just how incompetently the numbers were crunched. In the mean time, I'll leave it to Dr. Wexler's analysis:

      http://www.amazon.com/review/RYB2CUDKIISEP

      PS Notice how surprisingly few reviews Marzano's books have?

      Delete
    2. "Sounds like it was more like an investment on the part of those who were scared about paying any merit pay. "

      And yes, that's definitely a benefit for administration/politicians: Not only will this "improve" the schools, but it all-but-guarantees not having to pay out a lot of money in the process.

      I'm willing to bet Dr. Marzano's proctologist helped him find that 2% figure.

      Delete
  3. A bit more of why I don't trust anything from "experts" such as Marzano:

    http://ventingmycynicism.blogspot.com/2012/10/marzanos-database.html

    ReplyDelete
  4. Thank goodness, I found someone else who would dare to question the master, the guru, the end all of education. While he is making millions establishing his brand, public school educators have been thrown under the bus, and he is at the wheel, driving forward and backwards impressing every school administrator in ear shot of the screams of agony wrenching out from under the bus! I will listen to him when he comes into my school and spends one entire year implementing personally his "strategies" of effective classroom management, etc. Who evaluates his work and dares to test it to prove it more than a quick fix, pie in the sky phenomena. Let's throw Hunter in the mix with it, shall we?

    ReplyDelete
    Replies
    1. School administration (particularly at the district and state levels) are proof positive that there's a sucker born every minute...

      And good luck getting the "esteemed" Marzano to spend time in a real classroom and not one picked out for photo ops. ;)

      Delete
  5. I'm so glad that I found this site! My school here in Queensland Australia is all fired-up about this model, while I thought what a HUGE waste of human resources. Our Principal is a true disciple and flew to a workshop in the U.S.A. to really get soaked in Mazano strategies. We just this afternoon had an hour long meeting on one point; a design question that otherwise would have been a two minute casual discussion. I thought I was the only one on earth that thought this is all a great temple built of cardboard and wishing it would all blow away. I have taught for 25 years and have never seen such a complicated, response-funneled, mind-directed, time-consuming waste of time.

    ReplyDelete
  6. Mary Wilburn March 6 2014 - This too will pass but unfortunately not untl it has driven all of the good teachers out of education. Marzano is such a bunch of convoluted bunk leavng little to no room for individuality in teaching styles.......how boring for our students. Really....60 things we have to do every lesson to be good teachers. Why not. Make it 65 tell treachers how to all smile....walk....sit.....stand....and take a #@*$ exactly the same way so the can get a check in the right collumn. Young administrators are all hyped on Marzano talk like cult members. It is a pathetic situation.

    ReplyDelete