Tests are a chance to learn

Test early and test often.

That the message of a new paper in Science on learning by Karpicke and Roediger. This is an important paper for educators, as it claims to show that you don't learn as much just studying for a test as you do actually doing tests. That is, testing is more effective than studying.

To test this, they taught students a little Swahili. Students had to remember that "kaa" meant "crab," for example. They would see the Swahili word and its English definition on a computer screen for a few seconds, during which they would have to try committing it to memory. This was the "study" condition.

In the "test" condition, students were shown a Swahili word on a computer screen and had to type in the English word.

In every case, the students would do a round of studying, then a round of testing. In some conditions, the researchers set thing up so that students were only tested for words that they previously missed. This makes intuitive sense -- if you know something, why study it again?

The researchers varied how many pairs of words students got in each bout. Sometimes, they would get 40; sometimes, they would get less than 10. This does mean that the total number of trials -- i.e., individual pairs of words -- varied.

After eight rounds of studying and testing, students had this task cold. They were perfect. 100% recall in all experimental conditions.

The researchers asked each student how well they thought they would do on the final test; students reckoned they get about half the words. Then the students went away for a week, and came back for a final test.

If the real learning is going on during studying, you would expect to see a strong correlation with the number of study trials and student performance.

The effects are huge.

Students who were tested a lot over everything recall about 32 of the words (80%) -- more than double those who just studied, who recalled about 14 words. There's not even any overlap in the conditions. That is, the worst person who was tested on everything still did better than the best person who only studied.

That, my friends, is what my stats professor, John Vokey, called significant by the I.O.T., short for inter-ocular test. It's so bloody obvious it hits you right between the eyes.

At first, I thought this could be a simple case of distributed learning. It's been shown many times that studying a little over a long period of time is much more effective than studying a lot all at once. But this shows something different. It shows that encoding information over and over again isn't very helpful on its own, because there's another element going on: retrieval.

To use a wacky Zen metaphor, it doesn't matter how much you deposit into your bank account if your ATM card is busted and you can't withdraw any cash.

It is astonishing that an effect this powerful hasn't been recognized. And this clearly has strong implications for how we should teach students. This could well be a reason why clickers work. Why "just in time" quizzes work. Not just because students are getting more study or more exposure to information, but because they are being tested more, and they get better at retrieving information.

If I weren't constrained by already telling my students how my classes would work this semester, I would be retooling my classes right now to include even more tests.

Reference

Karpicke, J.D., Roediger, H.L. (2008). The Critical Importance of Retrieval for Learning. Science, 319(5865), 966-968. DOI: 10.1126/science.1152408