An Accurate Critique of Educational Research

Warning: this link sends you to a blog with a lot of really big words.

Is education research a form of alchemy? | ALT Online Newsletter.

via Is education research a form of alchemy? | ALT Online Newsletter.

I recently went to visit with someone regarding a non-education issue. He “prescribed” me two books to read. One was a “touchy-feely” book. The other was a more…scholarly…publication. While the first book was an easy read, the second contained words that this Ph. D.-holding blogger had to look up on the internet. At times, I found myself looking up the definitions of four words on a single page.

This article is like that second book. It is not really designed for the average reader, and I wonder Aaron Sloman’s purpose was to communicate with anyone or just to say something that has been bothering him on scholarly record. Still, once all of the words have been distilled to more standard vernacular (!), Dr. Sloman’s words make great sense. Here is my take on a fairly insightful article:

Most of the education “research” out there is bunk. It’s based on correlations as definitive answers to problems demanding greater explanation. The best example of any of this involves test scores from various schools, or even employment data from various colleges. Such correlational results don’t explain the true “why” in the structures and outcomes. We know that XX% of graduates from Henley College are gainfully employed; we don’t know whether that’s because of name recognition, connections, knowledge base, or flat-out luck. We may know that 4th grade test scores rose once a specific technique was put into practice. What we don’t know if that’s the real reason and why it may have made a difference. We also fail to account for any other options that could have been better.

Alchemists spent a great deal of time, money, and energy determining correlations. What they ended up with was a pile of data that was very shallow. In education research institutions throughout the United States, this type of investigation is accepted as the norm, sometimes even as a gold standard. But this type of research has resulted, at best, in a standstill for education. Deeper theories need to be formulated and tested to bring about the systematic change in education that America allegedly seeks.

Dr. Sloman continues the chemistry analogy:

Accelerated progress in chemistry came from developing a deep explanatory theory about the hidden structure of matter and the processes such structure could support (atoms, subatomic particles, valence, constraints on chemical reactions, etc.). Thus deep research requires (among other things) the ability to invent powerful explanatory mechanisms, often referring to unobservables.

Exactly. Perhaps the key to constructive “research” in education practice needs to begin with theories worth testing. The next Benjamin Bloom is out there right now, but s/he may be slogging through piles of data (test scores, graduation rates) to find relationships. Howard Gardner’s Theory of Multiple Intelligences has yet to be tested on any meaningful scale. That’s unfortunate. Here is one theory that holds tremendous promise for our children. Testing that theory would take serious time, money, and support. Unfortunately, the dividends may prove disruptive to the current system and financially minor.

The situation gets worse when you throw politics and commerce into the picture. Any correlation between anything and test scores gets picked up by a think tank or publishing house as “proof” that a certain system works. With the billions and billions spent “reforming” education, perhaps we could have found a few key elements of learning theory that would have created meaningful change in our educational system.

The problem is dual: (a) there is too much money to be made in the current system, and (b) it would take a substantial resource shift to make education research as effective and meaningful as the research the scientific community used–and continues to use–because it’s still working.