The Cult of Pedagogy Podcast, Episode 79 Transcript
Jennifer Gonzalez, host
We’ve talked about “retrieval practice” two other times on this podcast: First in episode 21, when I interviewed Peter Brown, one of the authors of the book Make it Stick. We talked about what cognitive psychology says we should be doing to help our students store the things they learn more permanently into long-term memory. One of those techniques was called retrieval practice.
The second time was when I interviewed cognitive psychologists Megan Smith and Yana Weinstein for episode 58, Six Powerful Learning Strategies You Must Share with Students. Retrieval practice was one of those six strategies.
In both cases, I noticed that although we were discussing collections of techniques, retrieval practice was at the core of both groups. Because it’s such a powerful practice, I would think that by now it would have really taken off, that I’d hear teachers talking about it and see the term crop up in lots of education spaces, but I don’t see that happening. So I wanted to give it one more push by carving out a designated spot for it in my podcast archives and on my website. My goal is to help retrieval practice become a regular part of classroom practice and our common vocabulary as teachers.
If this is the first time you’re hearing the phrase “retrieval practice,” I’ll give you a simple definition: It’s the act of trying to recall information without having it in front of you. So if, for example, you’re studying the systems of the human body—skeletal, muscular, circulatory, and so on—you could do retrieval practice by attempting to list those systems without looking at the list; you’d just try to recall them from memory. Then when you’ve listed all you can remember, you’d open up your book or notes and check to see if you got them right.
You might be thinking, This is nothing new. And in some ways, you’d be right: The whole concept of flashcards is built on retrieval practice, and flashcards have been around forever. What’s new is the research: In recent years, cognitive psychologists have been comparing retrieval practice with other methods of studying material—things like giving review lectures, study guides, and re-reading texts. And what they’re finding is that nothing is as powerful for cementing long-term learning as retrieval practice.
One of those studies was conducted in 2006 by my guest, Dr. Pooja Agarwal, and her colleagues. They looked at students in a middle school social studies course. Over a year and a half, while the teacher continued teaching as normal, students were regularly given no-stakes quizzes (meaning they wouldn’t count against their grades) on the material. These quizzes only covered about one-third of what was being taught at any given time. The teacher left the room for every quiz, so she had no idea what material was included in the quizzes. Here were the results: On exams given at the end of every unit, students scored a full grade level higher on the material that had been included in the quizzes than on any of the other material. The other concepts had been taught and reviewed by the teacher as they normally would; the only difference is that some things also appeared on the no-stakes quizzes, and those were the things that students retained more fully when tested on the end-of-unit exam. The very act of being quizzed actually helped students learn better.
Here’s what this means for teachers: When we teach something once, then want to do something else to help students learn it better, instead of just reviewing the content, we’re much better off giving students something like a quiz instead. In other words, if we do more asking students to pull concepts out of their brains, rather than continually trying to put concepts in, students will actually learn those concepts better.
Quizzing and flashcards aren’t the only way to use retrieval practice in the classroom, and in this episode, I interview Pooja Agarwal about specific techniques you can start using right away, plus some tweaks that will make these techniques even more effective. To learn more after you listen, visit Dr. Agarwal’s website, retrievalpractice.org, where you can dig more deeply into the research on retrieval practice and download a free retrieval practice guide.
Before we start, I’d like to thank Raymond Geddes for sponsoring this episode. For over 90 years, this third-generation family-owned business has been making fun and affordable school supplies, stationery and toys that students love.
Whether you’re looking for merchandise for your student-run store, new fundraising ideas, or awesome rewards for your classroom treasure chest, Raymond Geddes has what you need from school supplies, fidgets, scented pens, non-candy Halloween treats and much more!
Visit cultofpedagogy.com/geddes and use code COP20 when you check out to get free shipping and 20% off your first order.
I also want to thank you for the reviews you’ve left for this podcast on iTunes. Every new review gives new listeners more reasons to come check out the show. So if you like what I’m doing here and you think other teachers would like it, too, head over to iTunes and leave a review. Thanks.
So before I play the interview, I want to say one more thing about retrieval practice. In other episodes of this podcast, we’ve talked about more hands-on, inquiry-driven approaches to learning that don’t prioritize retaining specific content—things like Genius Hour and project-based learning—but within any of those frameworks, students still have to learn and retain discrete information. Take, for example, a student who has decided to learn Arabic as his focus for Genius Hour, or students who are studying soil pH as part of a gardening project. In both cases, they will need to study and remember concrete information in order to make progress, so even if your instruction isn’t set up in a traditional way, understanding how retrieval practice works can help you help your students learn better.
Now here’s my interview with Pooja Agarwal.
GONZALEZ: Pooja, welcome to the podcast.
AGARWAL: Thank you so much. It’s my pleasure.
GONZALEZ: We’re going to talk all about retrieval practice today, but before we get into it, if you could just tell us a little bit about who you are and the work you’re doing around this topic and why this is something that’s really interesting to you.
AGARWAL: I would like to think of myself as almost a combination of cognitive scientist with a K-12 teaching background with all sorts of other things thrown in. All of my interest in this world of cognitive science and retrieval practice really started when I was in college and I was majoring in elementary education and getting my certificate. Visually in my head I was going to one side of campus for all of my methods courses and my practica and learning about K-12 education and elementary ed, and then as a double major in psychology, I would literally walk across to the other side of campus and in the psychology building at Washington University in St. Louis I would be taking psychology classes on how humans and students learn and remember. It was just so weird to me that the two different buildings and the two different sides of campus, and my two different majors were talking about the same thing, but in very different ways without even really talking about each other. And so it was one of those sort of ah-ha moments for me, where I wanted to bring them together and I wanted to figure out how they work and how we can especially put in some research-based practices into classroom learning.
GONZALEZ: Got it. And so did you end up spending any time in the classroom, or did you just start to go into the cognitive science route of things?
AGARWAL: I did student teach fourth and fifth grade, and I’ve subbed here and there, but I did go into research. At the same time, I’ve made it a focus of my cognitive research to be in K-12 schools for 15 years or more than 10 years, and so almost all of my research has been in K-12 classrooms, which is a bit unusual, and so I’ve really enjoyed that part of being in schools and doing research with fire alarms and snow days and absences and students who are sick and substitute teachers. And so I feel like I have a really fun experience and perspective of K-12.
GONZALEZ: Got it. And so it’s become so important to you that you now have a whole website devoted just to retrieval practice. It’s called retrievalpractice.org. Tell us a little bit about what teachers will find when they go there. What do you have over there?
AGARWAL: On retrievalpractice.org, I have a combination of research, so downloadable articles from academic journals or guides or books written by researchers. I also have resources, so a number of different ways to think about implementing strategies in the classroom, resources like a YouTube channel, and then tips really based on what other teachers have done in their classrooms or how you can do this on a really practical level, so how you can implement retrieval practice in a minute or less. So the website really provides this combination of research, resources, and tips, and I’m particularly excited about making that applicable for K-12.
GONZALEZ: Nice. I realize I kind of got ahead of myself, because I think the next question I was going to ask you is since you have been studying cognitive science and how it applies to learning … well, okay. I have two different questions. One would be I want to know what made you sort of narrow your focus down to retrieval practice and then also what is retrieval practice? Because at this point, people are listening and they’re going, “Wait a second. They keep saying “retrieval practice,” and they haven’t defined it.” So why don’t we define it, and then let’s just talk about why it’s so powerful in such a way that it made you say, “I’m just going to focus right on that.”
AGARWAL: Great. Yeah, that’s a good idea. Retrieval or retrieval practice is this idea of bringing information to mind or having students really pulling out what they know from their heads or from their minds. And so an example I like to give is to ask, for instance, Jenn, what you had for breakfast yesterday.
GONZALEZ: Right.
AGARWAL: Do you remember what you had?
GONZALEZ: I do, because I have the exact same thing every day. I do, I have spinach and mushrooms and eggs and oatmeal.
AGARWAL: Wow. That’s much nicer than my coffee.
GONZALEZ: So, yeah. But I get what you’re saying. For a lot of people it would require a little bit more thought, yeah.
AGARWAL: Right. So kind of thinking back and trying to remember … A more educational example I suppose is, for instance, thinking about—or maybe Jenn, do you know—what year King Tut became a pharaoh?
GONZALEZ: Oh my gosh. Do you know, I used to know all about King Tut. When I was in fifth-grade my teacher was obsessed. I have no idea. It was a BC, right?
AGARWAL: Yep, yep.
GONZALEZ: 500 BC? Am I anywhere near?
AGARWAL: Uh, sort of. Do you remember how old he was?
GONZALEZ: Oh, he was a kid, wasn’t he? Wasn’t he like 10 or something?
AGARWAL: Yeah, he was 9 years old.
GONZALEZ: Okay.
AGARWAL: Isn’t that so neat?
GONZALEZ: Yeah.
AGARWAL: So that’s, I think, another good example of retrieval. It’s sort of this abstract concept, but we do it every day, and we almost take it for granted when we’re thinking about where we put our keys or someone’s name when we meet them. Maybe we don’t often think about King Tut or breakfast, but retrieval is that idea of thinking back and pulling information out. And something that’s so neat, I really think is so neat about research from cognitive scientists and research my colleagues and I have done, is that that simple act of thinking back and retrieving information so dramatically improves learning. As I mentioned in the research I’ve done that’s with Roddy Roediger, Mark McDaniel and others mentioned in the book Make it Stick that I know you’ve reviewed, in that research we’ve done some simple retrieval practice, and we can talk about strategies, that not only improves learning, but we saw benefits of retrieval practice to the end of the school year, and we saw benefits double for some students, and we saw benefits for special education students. It’s just this really neat robust, reliable thing that we take advantage of and don’t think about very often.
GONZALEZ: I want to make sure that we are really clear. What you’re talking about is sort of the act of quizzing yourself on information without having it in front of you. What teachers will typically do is they will use that as an assessment strategy and give kids a grade on it. What you’re saying is retrieval practice is not that. It is actually asking somebody to remember something as a learning strategy, not as an assessment tool for measuring.
AGARWAL: Yes, 100 percent. Thank you for mentioning that. There is, I think, this interesting interplay between retrieval and classrooms and retrieval for us outside of classrooms. Outside of classrooms, thinking about breakfast or King Tut, is not super anxiety prone, but as soon as we walk into classrooms, retrieval is so attached to the idea of assessment. Because it’s such this powerful way to learn, the more we can flip the notion of retrieval as a negative and flip that in the classroom to retrieval as a positive, as a learning strategy, makes a huge difference.
GONZALEZ: Right. And so here’s, I think, a great example. I am now going to remember that King Tut was 9, because we had that conversation. It wasn’t a test, but you asked me to try to remember it. I didn’t Google it, I just tried to remember, and that actual effort of trying to remember is actually strengthening those neural pathways, and then when you gave me what the actual answer was after I guessed, it’s now more solid in my brain, and that worked with all different kinds of content. That effort to try to retrieve it is what the dynamic is here.
AGARWAL: Yes. And different content and students and age groups ranging from elementary students to medical education to older adults. It’s really this, I don’t know, it’s this cool thing that just seems to work for all students all the time in all content areas.
GONZALEZ: And it sticks longer than just the assessment that students eventually take later. If they’re asked about this information months and months later, they still have a better retention of it because of doing the retrieval practice.
AGARWAL: Yes. We’ve especially found that in K-12 and in medical education, yeah. It’s pretty phenomenal.
GONZALEZ: So our goal, my whole reason to want to have you on this podcast is because I don’t believe teachers are doing this enough. I don’t think they realize the power of it, and I really want to have teachers finish listening to this podcast and say, “Okay. I want to start doing more retrieval practice in my class, and I know how to do it.”
AGARWAL: Yeah.
GONZALEZ: So what we’re going to do is we’re going to talk about some of the ways that they can incorporate this into their instruction.
AGARWAL: One of the ways, again, that I think educators already use retrieval and a really great strategy is think-pair-share.
GONZALEZ: Okay.
AGARWAL: All right. And so as opposed to a teacher perhaps reviewing everything from a previous lesson about Ancient Egypt is to ask students to think about one thing from that lesson or one thing about King Tut in a way that that student has to retrieve that information.
GONZALEZ: Okay.
AGARWAL: And then to work in pairs and then to share it with the class all involves this element of retrieval in a way that review doesn’t. I like the distinction between thinking about putting information into students’ heads versus pulling information out of students’ heads.
GONZALEZ: Right. And it’s the trying to pull it out, and a lot of teachers tend to default to just, “Let’s review what we did yesterday,” and that puts more information back in, and so the act of trying to pull it out … So one strategy would be to just say, “What is one thing you learned yesterday?” And this is where all of the kids, on their own first, would be thinking about an answer or writing down an answer?
AGARWAL: Yes. There isn’t a whole lot of research off the top of my head on this, but I do feel it’s important for students to think individually and retrieve individually as much as possible, only because if you jump right into pairs, then we all know as educators that some students are retrieving and some students may not be.
GONZALEZ: Right, right. Yep. And so having them do it individually, I used to find that too. I would like to have students just write something down real quickly, because it forced them to actually think of something to put down. And so, yeah, I think that’s another nice component of that individual retrieval. So you would say, “What’s one thing you learned yesterday?” And then they would share that with a partner?
AGARWAL: They could share it with a partner, you could have a class discussion, all which plays into this idea of feedback and how important getting feedback is, and especially as we know from the pair and the share, learning from each other, maybe other students learned or remembered things that we did not. But I love that retrieval can be even more simple in that there’s plenty of research. If a student just writes down two things or one thing they learned, and then moving on, that’s okay. You’ll still get benefits of retrieval without spending five minutes in a classroom discussion.
GONZALEZ: So this really literally could be a 60-second thing that happens at the beginning of class?
AGARWAL: Yes. Beginning, bell work, exit tickets, middle of the lesson, anytime.
GONZALEZ: Okay. And that’s really important that teachers understand that this is not something really time consuming to add to your plans. This is really something that you could work in. Now I’m looking at some of our notes, and you’re talking about the difference between, say, these are two different things. One would be if I just taught something, and then we’re packing up at the end of class and I ask everybody to get out an index card and write down one thing that they learned today. There’s some value in that, but why is it powerful in a different way to ask them tomorrow what they just learned yesterday?
AGARWAL: Oh great. There is so much fun research on what we call spaced practice or spacing. And again, I think this is a fairly intuitive strategy, but building it in more, asking a few days later what a student may have learned a few days earlier includes this spacing, and it makes that retrieval more challenging. So, for instance, I could have asked you … well, maybe you’re not the best example, but I could have asked you what you had for breakfast today …
GONZALEZ: Right, right.
AGARWAL: … instead of asking you what you had for breakfast yesterday. And so for many people, that almost slight extra struggle boosts the benefit of retrieval even more. It’s almost a more extra challenge to pull information out that’s a good thing.
GONZALEZ: Yeah. I think I remember in something else I read somewhere that the difference between those two things is that the thing you just learned or the breakfast I just had, that’s still kind of in short-term memory, so it’s almost just a reflex to pull it back up again. But waiting a little bit longer with the spaced practice, you’re actually trying to pull it out of long-term memory, which is more difficult. The phrase “desirable difficulties,” I see that all over the place, that actual struggle is making the learning more powerful.
AGARWAL: Yes, exactly.
GONZALEZ: Okay. So we’ve got the quick think-pair-share is a way of having students do this. What are some other ways that teachers could build in more retrieval practice into their instruction?
AGARWAL: One way, kind of tied in but doesn’t have to be think-pair-share, quite the same way is to use clickers or you had mentioned index cards. There’s also a good resource called Plickers where students can click in their response and you can have a discussion or give quick feedback and move on. Of course clickers then may take a little bit more preparation than coming up with questions on the fly.
GONZALEZ: Right.
AGARWAL: But they do seem to have fantastic benefits. So most of the research my colleagues and I did in middle school and high school classrooms were all with clickers.
GONZALEZ: Okay.
AGARWAL: Either with multiple choice, or the cool ones where high school students can text in a response. And those encourage, again, that individual retrieval for students and it has that really good benefit, especially in terms of data, that can be used as formative information for the teacher. There I would definitely step back and still re-emphasize that the activity is for learning.
GONZALEZ: Right.
AGARWAL: It’s not for assessment.
GONZALEZ: Right.
AGARWAL: But it just happens to be a side benefit.
GONZALEZ: Right. And this is another… I know, I think it’s Annie Murphy Paul, I’m not sure if you’re familiar with her, she writes a lot about low-stakes testing, and she gets so much grief from people, because there’s so much pushback on the other kind of testing that we do way too much in school, which is the standardized testing and trying to pull data and everything. We’re talking about testing as an instructional strategy, and so these mini quizzes. I mean, a teacher could really sort of map out a whole unit of instruction and plan within there these really brief little quizzes that aren’t going to go in any grade book anywhere. They’re just there to get the kids to start reflecting or retrieving some of that information.
AGARWAL: Even on the retrievalpractice.org website, I don’t use the word “test” at all with the exception of maybe one place. And for me it goes back to really focusing on retrieval as a positive and getting away from that notion that retrieval is a negative.
GONZALEZ: Got it. I don’t know if your research or anybody else has ever researched whether or not teachers are actually using the phrase “retrieval practice” with students, teaching them that and saying, “This is what we’re doing”? And I would wonder if there’s any positive impact of that, of students getting what’s going on, you know? Understanding how their brains are actually working better when they’re doing this?
AGARWAL: Yeah, that’s a great question. I’m not sure about research with teachers. I know anecdotally that many teachers find it beneficial to use the phrase “retrieval” or “retrieval practice” to emphasize that positive aspect. One thing we have done is again with my colleagues at a middle school and a high school outside of St. Louis, we surveyed more than 1,000 students, which is, I think, pretty neat and asked them about their test anxiety, because another concern with all of this retrieval is maybe students are going to feel or maybe it’s going to increase students’ anxiety, and we find the opposite. So we found that by the end of the school year, more than 90 percent of students say that clickers help them learn, and more than 75 percent say that clickers make them less nervous or less anxious.
GONZALEZ: Okay. Good. I want to also make sure that we differentiate between retrieval practice and excessive test prep. I’ve seen in schools where students are plugged into these programs where they’re just constantly answering multiple-choice questions on reading passages, and to me, that’s just trying to get them prepared to do standardized tests, and it’s not asking them to do retrieval on something that they’re actively learning right now or recently. It reminds me of The Matrix, just plug all of them into learning systems. But you’re actually talking about content that they’re learning in school right now.
AGARWAL: Yes.
GONZALEZ: Okay.
AGARWAL: Yes. Content they’re learning in school, in classrooms. When it comes to test prep, and I think particularly for my friends who are medical students they’re using retrieval as a learning strategy, but that still places that negative emphasis. So, “I’m only using this test prep program because of the high-stakes test later,” which is not the kind of retrieval that we should be encouraging with our students in our classrooms.
GONZALEZ: Okay. So we’ve talked about think-pair-share, and then using sort of mini quizzes with clickers or Plickers or even just index cards or things like that. And I can also, for people listening, provide a link to the Plickers website. I’ve talked about it, I think, on this site before. It’s a nice, cheap way of using technology without spending really any money, because it’s a teacher using their phone. So anyway, we’ll explain that later. And then tell us a little bit about the brain dump.
AGARWAL: Yeah. I really like this activity that I like to call a brain dump. And there’s again research we’ve done on the benefits of this type of retrieval where you could simply again, in one minute or less or a few minutes, is just ask students to brain dump—to retrieve or pull out everything they know about that American history lesson.
GONZALEZ: Right.
AGARWAL: Or everything they know about King Tut. And a lot of students. we find, especially in elementary students, are just so surprised at the amount of knowledge they have.
GONZALEZ: Yeah.
AGARWAL: Because they may not be retrieving it until they do this brain dump, and all of a sudden, it’s so great to see students filling up a page or three pages or more with all of this information that’s been in their heads and then to brain dump it.
GONZALEZ: I could see that especially in something like social studies or even if you’re studying a novel or something in a literature class to just … The first step is just the student doing their own brain dump, but then what happens next after they do that?
AGARWAL: Again, the first step would be to not grade it or at least to grade it very minimally. So when I do this in my own classrooms, my brain dumps or my mini retrievals are only about 2 percent of students’ grades, and I really emphasize it’s only 2 percent. So with the brain dumps, again, you could use that as a think-pair-share. A colleague of mine, Patrice Bain in her sixth-grade social studies classroom, she has students do a brain dump at the beginning of a lesson, at the beginning of a unit, beginning even of the whole semester. So what do you know about Ancient Egypt? Students will do a brain dump, and they may only have a few sentences or half a page, and then to do the same brain dump at the end of the unit. And even the simple act of giving students their original brain dump compared to their last brain dump is such a nice way to reinforce the emphasis of learning. And so it’s not even that those brain dumps have to be followed by think-pair-share, which would be a good strategy, but they don’t have to be. It can just be a simple confidence boost in a retrieval for students to compare their earlier retrieval to their later retrieval.
GONZALEZ: Got it, got it. I would think just in a lesson like that too, if you are going to be doing the sharing, to see what somebody else came up with, I’m imagining the kids looking at other people and being like, “Oh my gosh, I forgot about that,” and all of a sudden it’s another layer of … even though you as an individual did not retrieve that information, it’s kind of another way of giving feedback on it.
AGARWAL: Yeah. And to build … oh, there are so many things to build community with it so students benefit from each other. If it’s more of a conversation, I find that sometimes we want to give peer feedback or we have peer grading activities. But if it’s a simple, “Hey, this is what I got,” and “This is what I did on my brain dump,” is that, as you mention, an amazing beneficial feedback opportunity.
GONZALEZ: Right. You’d also mentioned to me in some of our correspondence that when you do the brain dumps or other retrieval practices, that it’s fair game, that you ask about topics from any point during the semester. Talk a little bit about that.
AGARWAL: That’s built into what we chatted about earlier with spacing or spaced practice. So sort of moving things into long-term memory or even more longer term memory and making that retrieval all the more challenging. That not only boosts learning, but it also helps students not cram, in a sense that my retrievals aren’t just a midterm and a final, or my retrievals aren’t for tests. They’re little mini quizzes or little mini things that happen each week, but in terms of past lessons, I’m also very mindful of what I ask students to retrieve. So I may say, “Hey, a few lessons ago we talked about the concept of correlations, so hopefully you remember what that is. Maybe now, just give an example of a correlation.” So I hardly ever ask students to define a correlation. That’s not the sort of retrieval I like to include or foster in my classroom. But even just having students muck around a bit in trying to draw an example, and from that, that sort of retrieval, again, is so low stakes that students become comfortable in my classes with, “Wow. I actually remembered that stuff. That’s really cool.”
GONZALEZ: Right.
AGARWAL: Or, “Gee, I didn’t remember that, but now we can have a think-pair-share, and help remind me of other examples.”
GONZALEZ: Yeah. Well that brings up a question for me. So if you’re asking students to recall something and they can’t, are you saying that the 60 seconds or so that they were trying to figure it out followed by the answer, so to speak, that that has value in and of itself?
AGARWAL: Yes, it does. So retrieval, even if students can’t necessarily retrieve information they’ve learned, or students retrieve but they are making errors, or they were retrieving in accurate information, there’s lots of research that that original retrieval benefit is valuable, and then providing feedback, after that correct information or the lack of information, is especially critical. So we did this in middle school classrooms where we had students retrieve, and we compared retrieval with no feedback vs retrieval with feedback. And even that simple amount almost corrects students’ misinformation with a simple amount of feedback. In other words, it’s okay if students make mistakes, because we all know we learn from mistakes and doing that is part of the retrieval, and then getting feedback is just as important.
GONZALEZ: Got it, got it. So we’ve talked about simple think-pair-shares. We’ve talked about doing actual short little quizzes, brain dumps. And then let’s also talk about flashcards for a minute.
AGARWAL: Sure.
GONZALEZ: I guess as an instructional strategy and … I’ve relied so heavily on flashcards as a college student, and nobody ever taught me how to do it, but I feel like some people just wouldn’t naturally get how to do it, and so yeah. Tell me what you think about flashcards.
AGARWAL: Flashcards, again, are an intuitive way that we use retrieval, and we talked a bit about the negative aspects of test prep or using flashcards to study for class. Students use them, and as you said students aren’t often taught how to use them effectively. There’s some great research by my colleague Jeff Karpicke about how students often will, as I like to say, cheat themselves. We’ve maybe all had the opportunity where we have a word on the front and we have a phrase or an example on the back, and you look at the front and say, “Oh yeah, I know that.” And you flip it over, and you seem to think to yourself that you’re really confident that you know the answer. And so with flashcards, one common error that students make is literally cheating themselves. They’re not retrieving. There’s just this element of confidence that “I know it.” With Jeff’s research, he’s shown that students also drop their flashcards too early.
GONZALEZ: Okay.
AGARWAL: So in other words, if you retrieve the correct answer once or if a student does, there’s a tendency to then drop that card out of their deck.
GONZALEZ: Okay.
AGARWAL: “I retrieved it once. It’s correct. I know it, so I drop it out.” And based on his research, it’s really optimal to successfully retrieve something around three times. There’s also research by other cognitive scientists Katherine Rosen and John Vendlinski who show that there’s sort of this optimal three-time retrieval. So students shouldn’t just drop their flashcard as soon as they know it, but to really keep it in there. And then I would say the third strategy is to shuffle a deck. So instead of students going through their flashcard deck in the same order and then going through it again and again in the same order is to shuffle that deck up each time, which again challenges retrieval, helps pull things out, and is much better than just going through it in the same order every time.
GONZALEZ: Right. Go back for a second to the first thing about them cheating themselves, because you’re talking about them looking at the one side, and then is it that they’re flipping to the other side too quickly before they actually are answering the question, so to speak?
AGARWAL: Yeah, too quickly or almost out of what we call familiarity, so looking at that front side and thinking just a simple, “I’m familiar with it,” or “I know it,” as opposed to using your flashcards and kind of similar to what we were talking about with think-pair-share or in the classroom is if a student looks at a flashcard on the front is to actually say out loud and retrieve out loud what it is, what the answer is, or retrieve on paper as opposed to just looking at that front and feeling that we know it and flipping it over.
GONZALEZ: Yeah. And that’s such a nuanced difference. You’re talking about possibly a matter of two seconds, but it’s so important, because … And I know I’ve talked about this on this podcast before, but I would remember in college I would be sitting there with my flashcards, and I would see people in the same study hall with their textbooks just open, and they would be studying and I would be studying, but I would be thinking, “They’re just looking at the words.” I knew, without knowing this research, I just sort of knew I couldn’t remember any of that information if I wasn’t sort of covering it up in some way and testing myself on it and to see if I could actually remember it. I think too with the flashcards, one of the mistakes I see people make is if it’s just a vocabulary test they’re studying for, for example, there are words on one side and definitions on the other, and I’ll see people look at the word side and try to remember the whole definition in their head, which I think is not necessarily a bad exercise, but they’re not going to get tested that way. So I would always think, look at the definition and see if you can remember what word that is describing. You can recall the word, then flip it and check instead of trying to remember word for word … Anyway. But I think that what you were saying before, I just wanted to make sure that was clear, that when students are cheating themselves, it’s because they’re just looking and they’re thinking, “Yes, I’m familiar with that,” and they turn it over without actually physically trying to retrieve the information on the other side.
AGARWAL: Exactly. And related to your observations in college, we also know that when students read textbooks, they tend to reread them and reread them and reread them, and so they’re not retrieving.
I’m going to take a quick break to tell you about our other sponsor, Kids Discover Online. They’re doing some really cool stuff to drive Inquiry-Based Learning. Kids Discover Online is a platform that offers a huge library of award-winning non-fiction science and social studies materials and units for elementary and middle school learners.
Each unit is a full lesson on subjects like the Constitution, Ecology, and Ancient China. They also have this cool feature called Discover Mode, a visual concept map that shows how units like Ancient Egypt and Simple Machines are connected. It’s really visual and interactive, and fun to play around with. Kids Discover Online also offers multiple reading levels, an assessment builder, and single sign-on with Google and Clever.
You can get started for free! Just go cultofpedagogy.com/kidsdiscover and check it out.
Okay, let’s get back to the interview.
AGARWAL: Really helpful as a study strategy is also a brain dump of some sense. So if a student reads a textbook chapter, then closes it, does a brain dump, and then opens it.
GONZALEZ: And then that’s when they’re getting their feedback, right, to check and see if they remember things accurately.
AGARWAL: Exactly. And that’s when they’re also getting their retrieval as opposed to taking notes while reading.
GONZALEZ: Right.
AGARWAL: And so I’ve done research comparing retrieval on open book retrieval versus closed book retrieval, and having that book closed is far more beneficial. You’re not cheating yourself again.
GONZALEZ: Right. So if somebody is a biology teacher in high school, they could be teaching their students these principles and … I feel like if they taught their students this stuff in September, they would see so much more success for the rest of the year, if they set aside a couple of class periods to show students, when you’re at home studying, so many teachers will send their students home and say, “Study for the test.” Kids have no idea how to do that, and most of them will just open up their notes and look at them and open up their textbook or go online and look at the information instead of doing something with it, and if we could just start showing our students, “Okay, look at this spread. You got a textbook open. I want you to look at it for a while, read it if you want to, whatever. Shut your book. Get out a piece of paper, try to write down everything you remember, and then go back and check.” You’re going to already have leapt ahead of where you would have been, right? Am I getting that right?
AGARWAL: Yes. And in terms of the leaping ahead, one observation I like to make with students is that when they leap ahead, they’re actually being more efficient with their time, so their retrieval is not just effective as we’ve talked about, but especially with students, students are learning more, so they don’t have to spend as much time studying.
GONZALEZ: Nice. That’s good.
AGARWAL: And that makes a really big difference in terms of getting buy-in for students.
GONZALEZ: So teachers could be using, so this is one of those things that’s actually a student strategy and a teacher strategy.
AGARWAL: Yes.
GONZALEZ: Teachers could be using these strategies with their students but also saying to them, “You can do this at home.”
AGARWAL: Yes.
GONZALEZ: I think we should be making flashcards with kids at school, give them five minutes to work on it, and then say, “Go do more of this at home and you’re going to ace this test on Friday,” or whatever it is.
AGARWAL: Yep, yep.
GONZALEZ: Are there any tweaks that teachers could be making to the quizzing or the think-pair-share or the brain dumps that would make these strategies even more powerful?
AGARWAL: One of the tweaks I like to talk about, and I think is a valuable question is retrieval just beneficial for vocabulary words? Or is retrieval just beneficial for remembering this one aspect in American history? And it was a question I was really interested in, I guess, again because of its classroom applicability, and in my dissertation I looked at higher-order thinking, can you use retrieval to help students increase their higher-order thinking? So on a sort of Bloom’s Taxonomy and analysis or a creation or a synthesis, how can we use retrieval to really push and increase student higher-order learning? So in terms of a tweak, I found in my dissertation with college students and K-12 was providing a mix of retrieval questions.
GONZALEZ: Okay.
AGARWAL: So in other words, if we want students to be thinking on a higher-order level, then we should make sure our retrieval questions and activities are also complex and higher-order.
GONZALEZ: Okay.
AGARWAL: As opposed to just hoping and expecting, “Gee, well, if I have students retrieve a vocabulary word, they should be able to apply this in a higher-order example or a higher-order type of material.” What I found is that no, you really ideally provide a mix of the two, of fact-based knowledge and higher-order knowledge during retrieval if that’s the type of learning you want to see in the future.
GONZALEZ: Right. I’m listening to a lot of this, and as a language arts teacher and a lot of my focus in the classroom was teaching students how to write, and so when I hear about a strategy like retrieval practice, I think, “Well, this wouldn’t apply to me, because I didn’t teach a whole lot of discrete, retrievable knowledge. A lot of it was just very skill-based.” So what do we say to teachers that teach that type of a thing, where it’s just really skill-driven, or maybe even somebody who’s like a drama teacher, and they’re trying to teach their students how to … well, actually I know exactly how it works with drama, because students have to memorize their scripts. But I just want to have a little conversation about that, about certain subject areas. I’m always saying, “Well, I’m not science, I’m not social studies, so this just isn’t going to work for me.”
AGARWAL: I find writing and math, those two content areas, to be particularly fascinating because of a lot of content areas I feel that students are really retrieving. They have to write essays and practice skills or they have to answer questions and calculations. So a lot of retrieval is already built in. With writing, one thing I’ve learned in my own classes is instead of saying, “Well here are the good things and the bad things that this essay did or this artist did or this video filmmaker did,” I recently asked my students, “Well, what are one or two things you liked about what was done? And what are one or two things you didn’t like that were done?” And for me, even with writing, it’s going back to not just reviewing information but asking students, helping students retrieve or connecting something with a previous lesson or a previous essay. So there’s a lot of parallels, and I do feel retrieval is already built in, which my guess is why there isn’t yet much research on retrieval in writing or in math.
GONZALEZ: Yeah.
AGARWAL: But I think it’s still valuable, and I think it’s built in.
GONZALEZ: You’ve got my brain working now, because I’m thinking I would teach a lot, and all of us teach, with a lot of mentor texts and mentor sentences, where you’re sort of looking at something that is already well-written, and then trying to kind of imitate some aspect of the way that they did that, so I guess I could see reading a short story or something like that, putting it away and asking students about their impressions and what was really strong in this, and what did you really like, and they would have to try to remember, how did the author actually achieve that effect? How did they make me feel this way? And then go back to the text and try to find exactly where they did that, and then you’ve got the retrieval going on, but then you’ve got a reason to be searching for a specific aspect of the craft.
AGARWAL: Yeah. Or even then going from closing the text or the essay, thinking about, or writing about, or think-pair-share with what they liked or disliked, and then maybe a next step with retrieval is writing a sample paragraph.
GONZALEZ: Right.
AGARWAL: So here’s what you did like or disliked, now write a paragraph focused on what you liked.
GONZALEZ: Yeah.
AGARWAL: And then going back to see if they’re all sort of on par with each other.
GONZALEZ: Right. Yeah, and you know, actually there’s a lot of emphasis now too on finding textual evidence for things. Sometimes you’ll read something and you’ll have a certain opinion based on something you read, and you don’t exactly know where you got it, so maybe not having that text available right away is a way to try to remember, “There was this one part where they were saying this,” and then you actually get the opportunity to go back and look for it. I guess that sort of is similar to retrieval practice?
AGARWAL: I think so, yeah. Or even maybe a future lesson is to look back and retrieve back and say, “We read this essay and we already talked about how it’s a good model or textual evidence that it provides. Tell me or retrieve one of those things that we learned or talked about,” so it’s still about a broader concept with textual evidence, but related to retrieval again.
GONZALEZ: Got it. Okay. So I got you. So the lesson could be something a little bit less with one specific correct answer, but asking the students even to just try to explain in their own words, “What did we just learn yesterday?” They’re still retrieving that skill or technique.
AGARWAL: Yes.
GONZALEZ: Okay.
AGARWAL: As opposed to just already moving on.
GONZALEZ: Right.
AGARWAL: We know that students forget, and they forget very quickly, and so instead of just moving on or reviewing, is going back to that idea of how can I take review that I do as a teacher and turn it into a retrieval opportunity?
GONZALEZ: Got it, got it. Oh, I love that. Man, now I’m seeing all kinds of applications for language arts. Because yeah, sometimes we teach our kids to do something, and then if we never really go back to that skill or talk about it or anything, it’s just sort of like, “Oh yeah, we did that one thing.” It’s like when I ask my kids at the end of every day, “What did you do today?” It’s like, “I don’t know. Nothing. Whatever.” Yeah. Okay. There was one other thing in the notes that I’m seeing that we didn’t talk about much yet was about the difference between elaborative feedback and correct answers in terms of feedback being an important piece of retrieval practice.
AGARWAL: So feedback is an important piece of retrieval practice, and there’s a lot of lab research and K-12 research and medical education research showing that there’s sort of this one-two punch when you include retrieval followed by feedback. Part of that isn’t just so students know if they got their answer correct or incorrect, but it adjusts students’ metacognition. So thinking about their own thinking, being better at estimating or judging what they know and what they don’t know. So sort of going back to the idea of feeling really familiar and cheating yourself with a flashcard. With feedback, then students can better understand, “Wow, I really thought that that was the answer, but I got it wrong.” An example we like to give is remembering or thinking about what is the capital of Australia? I don’t know if you know what the capital of Australia is.
GONZALEZ: I don’t know. I bet it’s not Sydney or Melbourne though, because that’s what people would say, and then they would be wrong, right?
AGARWAL: Exactly. So we often feel confident that it’s Sydney or Melbourne, but it’s actually Canberra.
GONZALEZ: You know …
AGARWAL: Yes?
GONZALEZ: No, no. I’m probably saying what most people say, because I kind of had that too, and I was like, I don’t even know what that is.
AGARWAL: Well I know it, because I’ve looked it up, but that’s cheating, right? It’s a good example, especially for our students where they can be so confident in something but they can be off base. And so providing feedback for that type of metacognition, understanding, “Do I really know it or do I kind of know it?” That’s an important aspect of telling students whether they were correct or incorrect or giving them elaborative feedback. The difference between correct answer or sort of right/wrong feedback and elaborative feedback you can imagine is more of the explanation, here’s why something is correct, here’s why something is incorrect. A lot of laboratory research has shown that the elaborative feedback is much more beneficial for learning.
GONZALEZ: Okay.
AGARWAL: Which seems pretty intuitive, right?
GONZALEZ: Yeah.
AGARWAL: It’s helpful for students to know why they got something wrong as opposed to just “you’re wrong” or “you’re right.”
GONZALEZ: Right.
AGARWAL: There’s still, I would say, a mixed bag of research in part because when we looked at elaborative versus correct answer feedback in an eighth-grade science classroom, especially over the long term, so by the end of the semester, feedback was still very beneficial after retrieval, but we didn’t see a difference, really, between elaborative feedback and correct answer feedback.
GONZALEZ: Got it.
AGARWAL: And of course laboratory research is not only with college students with well-controlled materials. Laboratory studies also have a very short time span, so usually they only looked at learning after one week.
GONZALEZ: Right.
AGARWAL: And what I really like about our research in eighth grade and in K-12 is we look at learning over a semester over a year. And so I do feel that elaborative feedback has its benefits, but as an educator, if those benefits are somewhat similar, especially in the long term, I’d recommend doing what’s practical.
GONZALEZ: Right. And you can get a lot more feedback, I guess, if you’re only doing correct answer feedback, you can give a lot more of it.
AGARWAL: Right, yeah.
GONZALEZ: Okay. Man, this has been great. I thought I already knew retrieval practice, and I feel like I’ve learned stuff just in this hour of talking to you, so I’m so excited. I hope this is really helping encourage more teachers to use retrieval practice. Is there anything that we skipped over that you wanted to talk about before we kind of get into your contact information?
AGARWAL: Oh, one thing I would encourage is to just start small. I think a lot of these strategies might sound like a big deal, and we have talked about earlier strategies that can be very simple, like asking a question and then having students do think-pair-share, writing down their response. I really like to encourage us to think about, how can we use this tomorrow?
GONZALEZ: Yeah.
AGARWAL: So what is one way we can use retrieval in our classrooms tomorrow without having to come up with quizzes, without having to suddenly buy clickers, without having to think about feedback? What is one way we can switch retrieval from a negative to a positive? What is one way we can flip from reviewing information to students retrieving information and really pulling that information out? How can we start small? What can we do tomorrow?
GONZALEZ: Good. I think that probably last little bit helped a few more teachers resolve to give this a shot tomorrow. I hope they do.
AGARWAL: Yeah, me too.
GONZALEZ: So where can people find you online?
AGARWAL: So a lot of the information we talked about is on retrievalpractice.org. That has resources, including books, websites, other podcasts that you can access to learn more. I also have research included, so academic journal articles, links to articles written by cognitive scientists and tips, information and strategies that teachers have used in their own classroom that are easy to adapt in any classroom or content area. So that’s all on retrievalpractice.org. We have a special website just for Cult of Pedagogy, which is retrievalpractice.org/cult, and there you can find some special extra information about subscribing for email updates and getting kind of a summary of what we’ve talked about and the very key research, resources, and tips I recommend.
GONZALEZ: Fantastic.
AGARWAL: And I’m also on Twitter @retrievelearn, and you can learn more about me and my nice jumble of background in cognitive science in K-12 at PoojaAgarwal.com, on Twitter @PoojaAgarwal.
GONZALEZ: Fantastic. Thank you so much for spending so much time helping us unpack this and get it into more classrooms.
AGARWAL: Oh, thank you so much, and I’m so excited to share all of this, I really am.
For links to all the resources mentioned in this episode, including Dr. Agarwal’s free guide to retrieval practice, visit cultofpedagogy.com/pod and click on Episode 79. To get weekly updates on all my newest blog posts, podcast episodes, and products, sign up for my mailing list at cultofpedagogy.com/subscribe. Thanks so much for listening, and have a great day.
This podcast is a proud member of the Education Podcast Network: Podcasts for educators, podcasts by educators. To learn more, visit edupodcastnetwork.com.