But Am I Doing a Good Job Teaching?

As an instructional designer, I have a lot of conversations with professors. They come to me for help planning instruction. Maybe they want to use a new technology, and this requires some planning––and some reflection.

“What do you want the students to learn?” is the chief question. But there are other questions:

• What do you want the students to learn?
• What will you see that lets you know they’ve learned it?
• What are your resources and limitations?
• What works well about your current practice––and what would you like to change?

The most troublesome question sometimes is: “How will you measure your own success?”

In learning circles, this is often called “evaluation.” You did something new: Did it work?

One obstacle to evaluating changes to teaching is: we’re actually not very good at evaluating teaching effectiveness in the first place. There are competing models for how to do that. But on the practical level, we just don’t have a clear pathway to evaluating our own teaching effectiveness.

There are some simple reasons. Typically, the same person teaches and gives the grades. So it’s not like there’s some separate yardstick. Perhaps the professor thinks “they did worse this year,” or “they did better.” But it’s hard to measure.

Often when I ask professors “How will you measure your success?” they will say “I’ll compare this year’s grades to last years’s.”

This may not work, as students vary from year to year. Instructors write new tests. Our tests are seldom validated, etc., etc.

Yes, you can test your students using a validated instrument, if your field has one. But most don’t.

There is, however, a different way to measure success, and it has to do with Bloom’s trichotomy: competency, proficiency, and mastery.

A clear example can be found in musical performances, such as singing.

• If you can’t sing a a certain number of pitches, durations and volumes accurately and consistently, you’re not a competent singer.
• If you can sing loud and soft, high and low, in various timbres, then you have the some of the proficiencies.
• But you’re only an artist when you can decide how to sing to achieve a specific effect. You can aim everything in a specific direction. Your skills are coordinated, integrated, and goal-directed.

Instead of comparing this year’s grades to last year’s, this other pathway asks the instructor to define competence, proficiency, and mastery, and to set goals for each.

This isn’t easy. The challenge is: those who teach are generally at the level of mastery. And at mastery, those lower levels have gelled into something the mastery. Like bricks in a wall that’s part of a building, the basic information of the field has become part of a larger structure.

• The expert can recognize large patterns quickly and accurately, work with big chunks of knowledge, and move rapidly back and forth between details and the bigger picture.

When I was a professor, I found defining these levels very difficult, and my success was mixed.

When I taught Introduction to Film, I could get most of the students to mastery: they produced competent analyses integrating different film terms and purposes, and at the same time, they pursued their own interests.

• To be competent at film analysis, you must be able to use the vocabulary correctly. If you can’t reliably spot a close-up or a zoom, you can’t do film analysis.
• It’s not enough to be able to say “that’s a close-up”; you need to be able to say what it does, what purpose it serves. You must be able to explain or align details with purposes and patterns. The close-up brings us into the character’s mind, or plays such-and-such role in the narrative, or makes a formal pattern, etc. Without explanation, description is inert.
• Finally, you’ve achieved mastery at film analysis when all your descriptions and explanations can be coordinated and integrated, you can say something larger in conversation with other scholars, and you can form and answer questions that suit your own needs and interests.

(I suspect these three levels must be similar for some very different disciplines, if only because concepts, functions and explanations, and debates are three disparate levels of increasing, integrated complexity.)

When I taught screenwriting, I had a great deal of difficulty getting the students to be competent because I myself could not define it clearly enough. Eventually, by comparing the best student work with the rest, I realized that for screenwriting competence means telling an involving story through actions. I knew this implicitly, but not explicitly. I had to pursue a conscious process to make this tacit knowledge explicit.

At first, my students couldn’t recognize an action, nor write one. Once I found a way to define sentences that present an action, the students could start to (a) write actions, (b) make them interesting, and (c) tell a story using them. (It’s like representing depth in a painting: if you can’t make blobs of color of different shapes and values, and you can’t organize them in specific ways, you can’t paint a landscape.)

Suddenly, when I could guaranty competency, I could spell out the proficiencies and help the students practice and integrate them towards a goal. Eureka!

When I taught Hollywood film history, I could get the students to competence, but I had no specific way of getting them to proficiency or mastery.

• Everyone learned the basic facts and how they fit together.
• But some students could give an analysis, and others stayed mired in facts. It’s hard to pursue your own interests when you can’t get beyond received knowledge.

Only later did I realize I could define the three levels as follows.

• Competence in classical Hollywood cinema means a few things:
  • describing a film in terms of why it was produced within the studio system: “This is a B-picture at Warner Brothers, adapted from a crime novel, using contract players”;
  • describing visual and narrative details of films and other related visual artifacts;
  • recalling relevant facts about the institutions of the period.
• Proficiencies include relating these historical and formal facts such that a film text (for instance) can become a symptom of various dimensions of Hollywood cinema as a social activity: political, cultural, and economic forces, just to name a few.
• Mastery means collaborating with other scholars to test theories and hypotheses, and pursuing one’s own interests.

So: facts and concepts, interpretation and explanation, debate and hypothesis-testing.

It’s not just that competency, proficiency, and mastery allow us to test better: they allow us to teach better. We know where we’re trying to get our students, and they know where they’re going.

And if we do this, we can then measure our own effectiveness much better––because we know what we’re trying to do.

But the process of knowing what you know is generally not quick.

Grades are a blunt instrument. If we translated them into competency, proficiency, we’d better communicate our expectations to our students.

In my experience, Bloom’s trichotomy proves useful, because it helps us to know better what we know but can’t say––and so to say it.

––Edward R. O’Neill, Ph.D.