Evaluation v Dry Rot

Excuse a rant today, but further evidence of dry rot in the higher education enterprise has wafted my way. A dear friend told me that while serving as a reviewer of applications for a prestigious (as in well-known) national award she was struck by a lack accumen in providing evidence of program success. One unnamed institution said they increased success by about 67%. All well and good, but that was a whopping 3 students to 5 students out of 100 served. Not good.

I’ve watched multi-million dollar and small-scale initiatives stumble more than once over what otherwise would be common sense. Figuring out how things work (or not) isn’t rocket science despite the theoretical posturing in the evaluation literature. As practitioners, we can’t ignore the fundamentals: what types of students change given what types of influences? If we can answer that, its easier to get to ways to increase the good we found.  And, it all starts with conversations that occur before a project is implemented. Unfortunately, in my work, institutions want to move right to remedies without crossing the evaluation threshold.  Dry rot.

Who tolerates this state of affairs? I doubt the straightjacket that is imposed on most graduate students in education about research rigor is always helpful for their future employers and students they serve. Few programs seem to care about preparing their graduates to make a case for the success or failure of a given program or project. A too-eager attachment to the so-called “scientific method” reduces most students to sort of zombie state  (see my post below about Randomized Controls) and undercuts other valid evidence.  The energy required to reach the “Gold Standard,” or assigning students to treatment and control groups, serves to fuzz up the context in which an intervention operates. As my colleague says, “not only do you need the results but you also need the context.”

Evaluate up!

Despite, or maybe because of, the rant above Rick Voorhees is enjoying the end of January in spectacular Glenwood Springs, Colorado, USA. He’ll celebrate Ground Hog Day in Charlotte at the Developmental Education Initiative and Achieving the Dream national meetings.