My name is Beth Ingram. I’m the Provost here at North Dakota State University. Please join me in welcoming professor Eric Mazur. I never asked myself the question how am I going to teach? Which is kind of strange right because when you do something new in your career that should be the first question you ask yourself. The question did not come up in my mind. It was perfectly clear what I was going to do. I was going to do to my students what my teachers had done to me to lecture this is a picture of me as an assistant professor teaching at Harvard. It’s a very old picture the picture was taken BC before computers you see them I’m using an overhead projector. Now in my own defense I think we all tend to do that right we try to project our own experiences onto the world around us and I naively thought that I’d learn physics sitting in a room like this listening to my professors teach physics and I’m sure that they too when they started teaching made that same assumption all the way back to this guy here who is the German King giving a guest lecture at the University of Bologna in 1125. Apart from the way we dress it’s the same. In fact you know to remind ourselves that our traditions are you know dating back all the way to the Middle Ages we still wear these robes at graduation but other than that it looks the same. I was asked to teach physics to pre-medical students none of my colleagues wanted to teach the course because pre-medical students are not very kind to physicists. But not so for me I you know I got a very high evaluations 4.5 4.8 on a five-point scale and on top of that my students did quite well on what I consider difficult exams. So very quickly I started to believe that I was the world’s best physics teacher. Now that turned out to be complete illusion nothing could have been further from the truth but very pleasant. So it went on quite a while. Now if you look at education around the globe that scene that you see on the screen behind me is repeated all over the globe right when we see it everywhere. In fact learning spaces I guess this is more performing space that a learning space so if I’m sure that most of the spaces on campus here are built this way. So I would like to ask you to describe the process since you know that seems to be happening in most education spaces that is illustrated on the screen behind me. What is it that is actually happening there. Projecting. Talking at okay. Sitting. Listening. Did I hear somebody say sleeping there? Do I have to remind you that these are my students? This is me there on the screen but you know now that you mentioned sleeping the French writer Albert Camus is claimed to have said once, “some people talk in their sleep, lecturers talk while other people are sleeping.” Now notice that most of the words that we’ve heard I think in fact all of them pertain either to them or to me. Talking, I’m talking they’re not talking. Listening, they’re listening I’m not listening. Pontificating, I might be pontificating. They’re certainly not pontificating. We’re both there at the same time. So is there a way of capturing the process between them and me? What is it that is happening? They’re telling but that would be again me, right? Sharing, that’s interesting but sharing I see sharing as a two-way process. I share something with you, you share something with me. These are pretty passive. One-way knowledge transfer. I like the transfer, I like the one-way but I question the knowledge. Is knowledge something that you can actually transfer? Think about that. Is knowledge something that you can transfer in a room like this? I would argue no because knowledge is something that needs to be constructed in the brain of the learner. But I like the one way and the transfer. Information, lectures focus on the transfer of information and you know what my students had actually rubbed that in my face very early my teaching career and instead of paying attention I’ve gotten upset. See I just told you that when I started teaching I never asked myself “How am I going to teach?” but there was a question that did come up in my mind. What was that? Not how but what? Exactly, so I went to a colleague who had taught the course before and I asked him that question he said “Oh in this course we used we’ve used in the past the book by Halliday and Resnick” those of you who may have had a physics course a while back may know that book. It has been a classic for probably 60 years and I was told, which surprised me a little bit having been educated in Europe before I came over here as a postdoc, that students would buy the textbook. I mean in Europe, you know, why would you buy the textbook if the professor’s presenting the content of the textbook to you. You might as well save the money. But anyway he told me be sure that the bookstore has enough copies. So a month before the course started I went over to our bookstore the Harvard Coop. I went to the person responsible for buying textbooks and I said be sure that by September 15 you have 150 copies of Halliday & Resnick in stock. And as I walk back from Harvard Square to my office I thought, “wait a minute if the students have Halliday & Resnick and I have the same book then what do I do in a classroom?” Now I started to get nervous you know so I knocked on my colleagues door before going to my office and asked him that question and he said, “Oh Eric, don’t worry there are lots of different physics textbooks” and he showed me a shelf full of books that had collected over the course of his career. And I started looking and very quickly I found the perfect book. It was perfect for two reasons: one it was different from Halliday & Resnick so at least I was not just regurgitating the contents of the book that they’d bought. But that was not the important reason, the important reason was the book was out of print. So for every class I would prepare lecture notes which I would put on the overhead projector or put on the board behind me and because I knew that my notes were different from the textbook I thought it would be good for the students to have a copy of my notes. Remember this was BC so it’s definitely BI, before the internet, there was no way of posting them so I had to actually hand out photocopies. So at the end of each class as they walked out through the doors in the back just like here. They could pick up a copied set of the lecture notes a photocopied set. Now why do you think that I hand them out at the end of class and at the beginning of class? Otherwise they would not pay attention or they would not stay but isn’t that already admitting that there’s a problem? I mean why force the students to get the information out of my mouth if they could get it from my lecture note? That that question never that idea that thought never crossed my mind but you know what happened? What happened was that at the end of that first semester about half a dozen students wrote on their end of semester questionnaire in the comment section, “Professor Mazur is lecturing straight from his lecture notes.” Hello, I mean what was I supposed to do develop another set of lecture notes to lecture from that was different from lecture notes I handed out to them? I mean, these ungrateful students. But you know they had a point. I was lecturing from my lecture notes and if they would have looked at that textbook they would have seen that the book wasn’t that different from the lecture notes. Now this scene here behind me is repeated all over the world. I’m sure that if we were to stop this talk right now and together walk around campus and pop in a few classrooms you might see a room that’s very similar. So that begs the following question “Is education just the transfer of information?” I mean it’s not a crazy question, right, because that’s what I would say probably around the world 99.9% of instructors do. If you believe that education is just the transfer of information press 1. If you think information is more than just the transfer of information press 2. Okay so let’s see where we stand in on this issue. I don’t know what C is. Well, ignore that one but we have an overwhelming majority that says “No education is more than just a transfer of information”. What 7% so a dozen or so of you said yes. I have a warning for you. If you’re a teacher and you clicked one you’re in trouble. You’re about to lose your job because let’s face it. Let’s imagine for a moment that education were just a transfer of information. I know most of us don’t agree with that but let’s just imagine for a moment. Education is just a transfer information If that were the case, what would be the logical thing to do? Put it online exactly. In fact it’s already happening. What would we lose if we took all of our courses and all of our classes and put them record them in different languages and put them online? Of course we’d lose our jobs but but but, in addition, to our jobs what is it that we would lose? Interaction, did I hear that? Interaction but how much interaction is there really? See, I’m trying to interact with you here it’s not easy it’s not easy because the space is simply not conducive to to interacting. You came in here thinking I’m going to sit down and listen to Erik Mazur present and that puts you in the same passive mode you would be in a movie theater or in a concert or anywhere else. You know I make a habit of observing of observing colleagues on campus teach different classes and when it’s a lecture class it’s always the same scene you see. We have some pretty good lecturers on campus you know, engaging, speaking style, dynamic, and they talk and then after 10 or so minutes they stop. And they say “does anybody have a question?” The students look down. They don’t want to make eye contact and if the instructor waits long enough it’s always the same person in the front row who asks questions. Most people do not want to interact. So I would say we may lose a little bit of interaction but precious little because most students do not interact. I think actually if we were to put all of our courses online we’d actually gain something. Because see one of the problem with a lecture is that there is very little opportunity to think. All right let’s say that we not not talking about education. I wasn’t talking education but I was talking about physics and I say something that confuses you. “Hmm I’ve got to think about this.” Your mind starts wandering and as it starts wandering you’re no longer listening. Right, you either listen or you engage cognitively in a meaningful way with the material and you think. Have you ever had in one of your classes a student raise his or her hand and say “Professor, could you please be quiet for five minutes I need to think.” It’s never happened in my courses and I’d be surprised at if it happened in yours but you have to admit if you actually want to think that’s what you should do. Now online you could do that. Right, you could hit the pause button and think “hmm, I’ve got to think about this.” Not that it happens but you could. Right, so my point here is this, if education is just the transfer of information we’re doomed. We’re doomed. Because there are better ways of transferring information than orally in a lecture. Luckily, I think most of us agree that there’s more to education than just a transfer information so we won’t lose our jobs. So now let me talk about, let me turn to the 93% of you said no. What more than just the transfer of information is education? Social engagement. Motivation, I would argue I could get motivated from an online lecture. I’ve seen some TED talks that are extremely engaging and motivating. So that could still happen. Yeah, critical thinking and debating. Reciprocity, we’re learning when we’re teaching. Very good. Personal connection, although again I would argue you can to some degree do that online I’ve seen again some TED talks that are extremely personal and personally connecting. I’m thinking of this one woman who got a stroke and described sort of the whole experience of a stroke well it’s riveting and and and and connecting. The lecture can get feedback from the from the audience and gauge what’s going on. Combining. God I got you fired up now. Combining ideas to create new knowledge. Two more. Social interactions. Synthesizing knowledge so I think that last point is good point to pause for a moment because that’s more or less where I wanted to go. You see it’s not enough for the learner to open his or her skull get the information in and close the skull and then hope to hang on to the information long enough to be able to regurgitate on the exam. As a learner you have to do something with that information you have to extract from that information the knowledge, the mental models if you want, that permits you to apply whatever knowledge is embedded in that information in a new context. Now, I’ve often asked myself where did that happen for me? Ask yourself that question too, where did it happen for you? Where did the things click or stick? Where did you have the aha moment? “Ohhh, I get it. Now I get it.” Did that happen while you were sitting in a room like this listening to your professors? I see quite a few people shake their heads no. No, it probably happen outside of the classroom not in the classroom. But that was a crucial part of us becoming content experts and you know we were dedicated to doing that otherwise we would never become professors most of our students however taking, especially in large introductory classes, are not taking the course because they want to become an expert in that field because somebody told them to go and take that course. Anyway it took a long time to find out that it was a complete illusion that I wasn’t the world’s best physics teacher that was probably one of the worst. I read about a test that tests students conceptual understanding of force using just words and the author of the test claimed that it didn’t make much difference whether you tested the students at the beginning of the semester or at the end of the semester they would do equally poorly you know most of my students at Harvard have taken AP course and gotten five. So I barely even talked about force in my course. I sort of assumed they already knew about it and I learned it in high school. So I read that and I thought “no way not my students”. See after all this author had been at Arizona State University and it mostly tested students in the Southwest you know California, Arizona so I thought maybe there’s some kind of a disease that’s raging in the southwest. Up, up in the Northeast at Harvard its definitely going to be better but one thing I’ve learned is you know you don’t just make statements you show things. So I thought I am going to show that in my class students ace this test. It’s too late to do a pre test at the beginning of the semester. We’re about two weeks before the exam and but I thought I have to show it. So I gave this test to my students it took no more than two minutes for my life to change forever because they’d just started or one student sort of slowly raised her hand and I walked up to her and she looked at me and she said “Professor Mazur, how am I supposed to answer these questions? According to what you taught us or according to the way I usually think about these things?” I had no idea how to answer that question and by the time the test had been completed it was clear that I had a major a problem. Students had no clue what the concept of force was. So, you know, I started to think about education in a different way it’s it’s not just about the transfer of information. The transfer of information is a necessary first step but not sufficient. The learner has to have an opportunity to assimilate or (inaudible) would have probably called it accommodate the information. In the traditional approach to teaching we put all of the emphasis in class on that first step and then we let the students take the responsibility for that second part. If you ask yourself, “which of those two steps listed on the screen is the easy one and which is the hard one?” I’d be surprised, I’m not going to do it because I’m going to run out of time but, if we were voting on it I think we’d all agree it’s a second one the hard one. So it’s kind of ironic that we put all of the emphasis on the easy part and then leave the hard part to the students on their own we should really focus on the hard part. So that’s when I came up in 1990 with the idea of you know flipping that around and giving the students the responsibility for the transfer of information so that in class I could essentially focus on the assimilation of the information. Right now I want to pose the question, if you’ve been able to successfully give students the responsibility for the information transfer, then what do you do in the class? Well the answer to that question is nothing new. Teach by questioning instead of telling. Who’s the first one to actually have said that? Socrates 2,000 years ago and here we’re in the 21st century and still most classes especially in STEM fields, I think you know humanities are probably doing better, are still mostly taught by telling rather than questioning. Now, it’s not that I was sitting there thinking at home, “Socrates teach by questioning.” No it was sort of a serendipitous discovery. You see, after giving that test to my students, not only was I shocked they were equally worried. Right, there was two weeks before the final exam they couldn’t understand why they’d done so poorly and they were worried. So they asked me for a special session to go through every single question on that test. So I booked a room like this at that point I 250 students in my class and I, I went through every single question. I remember coming to a question which in my mind was completely trivial. So I turned my back to them I sketched a few things on the board and I turned around and I said I gave them the answer. I could see from their faces that they were completely confused so I said, “is there a question?” They were so confused they could not even articulate a question. So I thought boy this is serious you know maybe I should you know bring in additional aspects. So I erased the board, I started all over. In the next eight minutes I gave the most brilliant explanation you could possibly imagine, okay the whole board was covered at the end with equations and drawings. I’d worked out every little detail in the most exquisite detail. Of course I’d done this all with my back to them and I turned around, triumphantly, only to see that they looked even more confused. And they could still not articulate a question that I understood. You know how it is, right? When you’re a beginning learner, it’s sometimes harder hard to pinpoint what your difficulty is and as the expert you don’t connect to to that to that to that difficulty. So I didn’t know what to do. I knew however that half of them had given the right answer on the test. So in a moment of despair I said to them why don’t you just discuss it with each other. And something happened I’ve never seen in my classroom. Complete utter chaos. They forgot about me, I could have walked out they would not have noticed it. But what was even more surprising, in just two minutes they figured it out. I was really stunned by that I thought how can it be I the expert try unsuccessfully for ten minutes in two different ways to explain it and they just figure it out. But imagine you have two students sitting next to each other John and Mary. Mary gets it and has the right answer she understands it. John does not and gets the wrong answer. On average Mary will be more likely to convince John than the other way around. Simply the force of logic but this is the crucial point, that’s not the important point. The important point is this, Mary is more likely to convince John then professor Mazur in front of the class. Why? Because she has only recently learned. She still knows what the difficulties are that a beginning learner has. Whereas professor Mazur learned it such a long time ago – in his mind it’s so clear that he can’t even understand why somebody doesn’t understand it. It’s what my colleague, Steven Pinker, calls the curse of knowledge. We developed these blind spots. We forget and in fact you know remember when you were a student often you wouldn’t even bother asking your professor because question would go straight over your head. You’d ask a friend first, “hey, do you understand what was going on there?” because you’d know you’d get a an explanation at the right level. So when I saw that I thought “god that’s what I got to do in my classroom.” So, I step into the classroom, I talk a few minutes not half an hour like I do now and I pose a question. And then after I pose the question I give students time to think, it’s quiet. In fact, I tell my students you’re not allowed to talk to anyone. It’s as quiet as during an exam. “If you talk to your neighbor I’m gonna I ask you to stand up and tell the whole room what you just told your neighbor.” Then I polled them initially, you have a clicker so you may think the technology is important but it’s not important, it’s the pedagogy that matters, initially I just had them and in my talk this morning we did that and in the physics department. Just let them put their hand on the chest with you know 1 2 3 4 5 fingers or you can use flashcards. The clicker is just sort of icing on the cake if you want. So I polled them and I try to design the questions so that between 30 and 70 percent have the right answer. If less than 30 percent have the right or desired answer they’re simply not enough students to convince others. If it’s more than 70 they’re going to very quickly run out of things to say and they’re going to be off task. They’re going to start you know pull out their cell phones and check their email or do something else but if it’s between 30 and 70 there’s likely to be a very lively discussion and many aha moments during that discussion. I tell my students also, holds for you in the back too, in a minute when we try it out if I see you sit alone not talk to somebody I will come and talk to you. And the first few lectures I make a habit of quickly running through that and they very quickly reseat themselves and talk to their to other students. I poll them again and again if it’s between 30 and 70 the first poll it’s not unusual to go to, you know, much larger percentages after. Then explain either by asking students to explain it or by providing my explanation. And then the cycle essentially repeats until class time is up and the learning, the “aha” moments take place there, you actually see students go “ohhh”. I don’t know if I have a little video up here. It’s okay so you can sort of hear the audio there so I read the question with them. They and then they think about it. I let them think for between a minute and two minutes and it’s quiet then. I see on my screen how they vote I do not share that with them. See the aha moment there. That’s one of the most exciting things when you see this aha moment. This light go on in the eyes of your students. And that’s not unusual I do show them that second distribution to close the loop. So what’s going on one, it’s active not passive it is impossible to sleep through my lectures because every few minutes your neighbor will start talking to you. Secondly, now it’s a two-way flow of information. It’s not only information going from to them, there’s information coming back from them to me. Thirdly, it’s continuous feedback on the learning without any threat or there is not any high stakes. I don’t give points for participation. I don’t give points for getting it right. It just has to be completely intrinsically motivated. So, if as a student, you’re still answering A after the discussion and you see that 90 some percent of your class is B. It’s a little warning, right? “Oh I gotta look at this most of my class got it right. I don’t. So it’s a way of continuously assessing your own knowledge in a non high-stakes way. And lastly it personalizes the instruction. Student A can help student B students C can help student D even though B and D have two very different problems, right. So it’s in a sense it crowd sources and personalizes the instruction. So anyway, so thermal expansion deals with the fact that hard solids like wood or concrete or steel expand when they get hotter and they shrink again when they get cold. It’s very important technology. Just to give you one example if you’ve ever heard a train go by at low speed or been in a train. You have trains not very far from here. You may have heard this clickety clack sound of the wheels as it goes from one section of the rail to the other. It’s because they put the section and the sections leaving a little gap, tiny gap between the rail so that if the weather gets hotter and the rail expands there’s space for that expansion to happen. If you don’t do that bad things happen as you can see from this railroad in India that was put, you know, back to back. When you build large steel beam buildings you need to take that into account. Next time that you park your car at a concrete parking structure, after you park your car look down and you’ll see that every ten yards or so there’s a rubber strip. When they put down these concrete slabs they leave a little gap between the concrete slabs which they fill with rubber so that when it gets hotter the concrete expands the rubber compresses. It’s not a hard solid a soft solid so it can take that into account. If you think, “well that’s all you know maybe interesting for engineers it doesn’t concern me.” Well, next time you go to your dentist it does concern you. Because if the dentist finds a cavity then she or he will need to fill that cavity. And if you were to just use some kind of a metal let’s say to fill that cavity then you’d have a serious problem the next time you drink a cup of hot tea or hot coffee. Because the metal would expand and metals tend to expand more than other materials such as the material that your tooth is made from and ouch, you know, crack there goes the tooth. It’s in two parts all of a sudden, so the dentist actually has to use a mixture of materials called amalgam, which expands and contracts the same way that your tooth expands. Now the reason that solids expand is that they’re made from atoms. I’m showing nine of them here and in a solid atoms hold each other in place if they don’t move relative to each other and when it gets hotter the atoms get further away from each other. Cold and hot that’s all there is to it. That’s all you need to know to answer the question I’m going to ask you. Now you may wonder why is it that atoms get further away from each other. I’m not going to ask you about it but but just to satisfy your curiosity the reason is that atoms do not sit still. They vibrate back and forth like this and the amplitude of that vibration is related to what we call temperature. So this is these are cold atoms and these are hot atoms. Cold atoms. Hot atoms. So if you were an atom. You wouldn’t just sit like that you’d actually be shaking back and forth and as it gets hotter you shake back and forth over a bigger amplitude and you need more space. You can’t get crammed together as much because you just push the people around you away and they in turn push the people around them away. And it’s not just those nine it’s all of them. So cold and hot, questions anyone? Thank you for reaffirming that I’m a brilliant lecturer. This is really wonderful. Although, I think I heard a question there but you know I’m not going to simply ask you to regurgitate that same information. I’m going to see if you can take this picture of atoms getting further away from each other all of them and apply that to a different context. Remember I’m going to ask you the question, by then it’s too late to ask me questions okay, and you’re not allowed to talk to your neighbor. If you talk to your neighbor I’m going to ask you to stand up and tell the whole room, out of fairness, what you just told your neighbor. Then we’re going to vote and I’m not going to show you the distribution. I’m going to ask you to find after that a neighbor who has a different answer. So if you turn to your right and that person has the same answer you say, “thank you very much” and turn to the person on your left. If that person also has the same answer you turn to the person behind you or in front of you if everybody around you has the same answer do not assume you’re all correct. Get up walk around find somebody with a different answer. Instructions clear? Good. Okay, so here’s the question. Consider a rectangular metal plate with a circular hole in it. Now imagine that we uniformly heat this plate. What happens to the diameter of the hole as the metal expands? Does it increase, does it stay the same, or does it decrease so so if you don’t know for sure choose whichever you think is closest to what might be the truth. Okay, good, so now find the neighbor with a different answer and then try to convince that neighbor that you’re right and he or she is wrong. Go ahead if you. Now look at that you all got fired up. I’m not here to talk about thermal expansion. I’m here to talk about pedagogy. The answer to this question doesn’t even matter. If you look at small children in a sense we all born scientists, right. Three, four, five, six year-olds they keep asking their parents their teachers why why why why. We’re wired to wanting to understand the world around us. Whether you become a social scientist or an economist or or or a painter or whatever we were all born scientists. And it’s kind of a shame that education, and that’s true everywhere not here but all over the U.S. all over the world really, does a really good job turning this innate curiosity, that we’re born with, off. The good news is I’ve just shown you how easy it is to turn it back on. Right because imagine I had given the same little lecture that I gave you a moment ago but instead of asking you the question I would have said “let’s now apply this to rectangular metal plates with circular holes in it”, if we take one of these plate and put it in the oven the plate will expand and the diameter of the hole will… I’m going to keep you in suspense a little longer… you know, you would have been sleeping through it. I mean, what is more boring than metal plates with circle holes in it? Isn’t it amazing, right? I trust me, if you can do this with metal plates with circular holes in it you can do it with anything. You can do it with absolutely anything. Now you know I want to keep you in suspense a bit longer. Let’s sort of analyze the psychology of this. I asked you a question and then you thought about the question and then you had to make a commitment. Right, I told you to make a commitment. We could have done this hands on the chest would have been exactly the same or by clicking. And then after that I asked you to externalize your answer, not to me, which would have been intimidating but to a neighbor. And something interesting happened we could see it with you continuing to talk and gesticulating. All of you moved away from the answer to the reasoning. All of you were sitting there like this trying to you know talk about not the answer but how you get to the answer. This approach in a sense brings the thinking, I think it was you mentioning critical thinking, back in the classroom. But most importantly you got emotionally invested in the learning process because if I were to tell you now, “Bye gotta go. Got a plane to catch”, you’d come running after me asking “what’s the answer to that question?” Now, before I can give you the answer you have to vote again. So please indicate what you now believe to be the right answer. You know I thought I gave a pretty darn clear lecture here and so did you. Only a quarter of you got the right answer the first time around and unfortunately the method doesn’t work that well because as I said you need at least about thirty percent correct. So in a sense you’ve made a very important point. Because in a sense you have been telling me, by the way you voted, “Eric you’re lecturing sucks” which is actually precisely the point I wanted to make. It seemed clear but in reality you haven’t even begun to learn. The right answer is number one! So let’s see here. Wow look at that, look at that. Let’s compare it to the previous, so you started about a quarter correct and it went up and all the other choices went down. So even though only a quarter of you got the right answer the first time, notice that it moved in the right direction. And right now at 40 percent it’s almost the dominant answer now. I don’t want you to lie in bed tonight at 2:00 a.m. wide awake. So let me take a minute of my precious time to explain this this question. Imagine you have a jar of marmalade in the refrigerator. It’s one of these ball jars. Right, glass jar, metal lid, the medal lid is a ring and a plate. You take it out you can’t open it what do you do? You run the metal lid under hot water. The ring expands and the hole gets larger. You say “yeah, but you didn’t ask about the ring you asked about a plate”, ok ok. Could I have a piece of paper? Can I borrow your pad for just a second the whole pad. Just give me the whole pad. Imagine we have a plate, sorry, thank you, imagine we have a plate no hole in it. Can you imagine that? We take a marker we draw a circle, so now we have a plate with a circle. We put this plate with a circle in the oven. Thank you. We turn the temperature up. The plate expands what happens to the diameter of the circle? Yeah everything gets bigger so the circle gets bigger too. You say “that’s unfair there’s no hole”. If there was a hole then the others would expand into the hole. I’ll show you what’s wrong with that. Let’s imagine that we go outside and we form a big circle each holding hands. So now I hold your hand and so one big circle. Each of us is an atom, one of these dots there, at the edge of the hole. Can you imagine that? So there we are holding hands and now we are at the same time step in towards the center of the hole. What just happened to the distance between us? It got smaller, it can’t get smaller because we’re shaking more we need more space. Well, the only way to make more space is to remove a few of us. But atoms in a solid don’t disappear like that or to make the hole larger. Okay, back to peer instruction. The first time I did it I doubled the learning gains. I doubled it. Not 10% 20, 100% and in subsequent years by asking better questions I tripled the learning gains. I mean it was huge okay. And you know studies have shown that in other fields from from computer science to the humanities similar types of gains and the study occurring now shows better retention. Which makes sense right, because once you’ve had this “aha” moment it sticks. We’ve eliminated this, right. And the question is really how do we effectively transfer the information outside of the classroom? My first impulse when I started doing this was, “lets have the students just watch a recorded video”. I’ve recorded my lectures, why have those lectures again? Let just have them, there’s not that much interaction anyway. Why not have them watch the recorded lectures? But you see there a problem with video and the problem is that the transfer pace is set by the video in the same way that the transfer pace is set by the lecturer. I already said are very few students or no students who will just shut you up during lecture because they need to think. Now with video you could pause it. However if you look at EDX data or pre-lecture data and so on you find that the students do the opposite. They put the playback speed at 1.5 my daughter told me she goes to 2.0. Amazing, how do you get through it as quickly as you can? Giving you even less of an opportunity to think and process. And there’s plenty of studies that have shown that you know when you put people in front of a TV they’re passive. They turn into, the brain turns into a meditative state. Why would you be more actively engaged when you’re watching a lecturer lecture than when you watch something on TV? Also this again from the EDX data you find that the students maybe watch the whole video for the first class in the semester but as time goes on they very quickly discover that the way they’re held accountable is by answering a few multiple choice questions. And they can often simply answer the multiple-choice question by multiple tries or by fishing for the answer or Googling it or whatever. So towards the end of this method they don’t even watch the video anymore. They go straight to the multiple choice question and answer that. But, and perhaps most importantly it’s an isolated individual experience. You the student and the video. Whereas learning deep down is a social experience. So I think if we have students watch video and all that we’re really doing is moving this out of the classroom. Which you know is not the most important. So then I thought let’s have them read books because books have an advantage right? Now you the reader are in control if you read something that makes you think, you stop reading because you can’t think and read. Well, you might read a few more sentences but then you realize that you’re reading without thinking about what you’re reading. You’re thinking about something else and you go back. You are in control of the transfer pace. Also there’s lots of studies showing that the brain is much more actively engaged during reading than during listening and watching. So those are two big advantages of reading over watching or listening. However, we still have a problem because we don’t have any real accountability. How do I know my students are reading if I tell them read chapter 22? And secondly, it’s still an individual isolated experience. What we really want is this, we want every student prepared for every class. And I don’t know about you but ideally we want that without extra effort because we’re already quite busy. This solution suddenly hit me four years ago. It was really like, “duh why did I think about this earlier?” I put all of my effort in making the classroom a more social experience like you’ve just seen. The key was to make the out of class component also a social experience. So this platform, which is free, Perusall at Perusall.com, is essentially a social learning platform that is interactive. Students log in through their preferred social network or through the LMS or they can make an account. All these possibilities are ok. And once you’re logged in it actually looks a little bit like an e-reader but it’s not an e-reader it’s much more than that. The first thing that you’ll notice is you can see who else is online reading that text. But more importantly as you’re reading if there’s something that, you know, interests you or causes you to wonder or ask a question, you can highlight the text which opens up a chat window and then you can type something in the chat window. And as you hit the return the the highlight sticks. So after a while the page will be marked up, you know, with highlights and you can click on any of these highlights and it opens the transcript of the chat attached to that particular passage in the text. So here for example, I don’t understand how this combination of factors tells you anything about blah blah blah, October 20th, midnight. Half an hour later, I think you may be able to think about the direction separately so blah blah. Two days later third student, different from the first two, says “this is a great question to further elaborate on this we can think of this in terms of” blah blah. So what do you see here? You sort of see in an asynchronous peer instruction students helping each other parse the text and understand by sharing. Which we’ve heard mentioned already by a number of people so in Facebook you have a “like” button we don’t have like buttons we have question flags. So if you read a question and you have that same question you can click this button which increments the count. And we have another button which is the “this helps me” flag. So in other words if you see an explanation that it’s helpful you can click it which then turns that green and as more and more students puts it there’s a counter in front of it gets incremented and students attention, attention gets drawn to important questions that have been unanswered and explanations that are helpful. So what if you know you have put a question and you’re no longer online you’re you know at the dining hall or whatever. Well, you’ll get an email and the email first of all reminds you of the question that you asked, twenty-one minutes ago you ask this question and then it says “Ryan just responded to this question by saying, blah blah, if this helps your understanding click the button below.” So there are three things you can do, well, four you can ignore it, but you can reply to the email without going back to the platform and then whatever you type into the email gets inserted in the chat as if you had magically been online. Or you if you don’t remember where you asked the question you can click on that which puts you back into the system at the right place or you can click this button this “comment helps my understanding” which is same thing as clicking that little check button. Here’s the big question, how do we get students to participate? And we essentially use a combination of intrinsic and extrinsic motivation drives. I would love to have it purely intrinsically motivated but as you and I know especially in large courses, you know, students are not necessarily intrinsically motivated. So we tell the students that the body of their message has to demonstrate thoughtful reading and interpretation of the text. Which means that if you all you do is highlight something and write I don’t understand this you get no credit whatsoever because you can do that without reading. If you highlight something and you said I don’t understand this because on page 256 it says blah blah blah you get partial credit. If you highlight sayings and you say I don’t understand this because on page 256 this is blah, blah, blah and you reveal your thinking now you start to get real credit. We want students to have a certain minimum of that and of course you want it to be before the class starts and we want them to be distributed not clustered. So if it’s a twenty page reading assignment you don’t want only annotations on the first page and you know. So with that rubric which is on the Perusall. You can download and give it to your students we get twenty thousand, in a class of sixty like I teach twenty thousand annotations. The students write more in one semester than the author of the textbook. So I can already hear you think you know how do you process all of that? Well, this is the exciting part, it’s fully automated. I got together with a quantitative social scientist on campus who does machine learning analysis of social media because I realized that if you look at you know Twitter and Facebook page and you can do machine learning analysis of that, you could certainly do the same approach to look at annotations in different text books. So we use a specialized machine learning algorithm and it actually assesses intellectual content and we’ve demonstrated, I put one of my students, I give this as a educational research project to one of my graduate students. And in her thesis we demonstrated that the machine tracks the person who has trained the machine better than another human being. So in other words the agreement between trainer and machine is about 80%. If you take two individuals the best they can after even talking to each other is agree on 75% because you know its not completely objective of course. So immediately after an assignment you have a grade book showing, you know, the names of your students and what they have for the different assignments. You can click on a on a grade and see you know how many annotations which one were submitted before the deadline and the students get the same feedback. But it gets much better than this because that is just the stick in a sense let me tell you about the carrot. When I started doing this I realized that those annotations were like a window into the brains of my students. My fingers were itching to click on them and see what are they thinking. I was particular interested because the book we were using was my book but also if I knew what the students were had questions about I could design a better class. Right, because then in class rather than asking the questions that come up in my mind. I can ask the questions that come up in their mind. So in the beginning I was just clicking a thousand times before class to come up with some good questions but then I realized, you know, we should really automate this. So there’s a button in Perusall called the confusion report. Which essentially gives you for whatever the assignment is, this was chapter 24. This was actually from a class at University of Central Florida. You know it tells you the three or four topics that lead to the most confusion. So I can walk into class and say, “thank you for your thoughtful annotations.” It looks as if you are mostly confused about these topics. I don’t show the confusion report, I just makes slide that has those terms. Topic one, topic two, topic three. One of you asked this great question. I just take their questions and they’ll go, “wow he actually reads our annotations.” The nine of them but they don’t need to know that. So now all of a sudden what do we have? We have a combination of intrinsic motivation. One is the social interaction. I’ll show you it’s fun to be online because you know you can talk you can chat with each other you can interact synchronously or asynchronously. And there’s also tie in to the in-class activity. If you take the trouble of reading there’s a good chance that either a student will help you or it will be addressed the next week and next day in class. And then of course there’s the extrinsic motivation, which is fully automated. Here’s what one student wrote in a survey that we do, “I think the Perusall app in my class, Perusall app annotation way better than just reading a textbook normally. I’ve been reading for almost four hours now and haven’t gotten bored.” Okay in all fairness, I have to tell you he was reading my textbooks. Here’s Ohio State where they’re using it in a lot of large classes. This is from a 600 student introductory chemistry class. “It makes the book fun to read. All the other students on my floor are disappointed their professor isn’t using Perusall because they don’t read the book.” Let me show you some amazing data. Ok? These are three semesters, three semesters ago, two semester ago, one semester. We were fiddling with the algorithm. This shows the percentage of students versus the number of chapters missed before class. They had to read over the entire semester, 17 chapters distributed over the course. So this goes on and on and on and on to the door there on the far right. But all the data are zeros, so I’m just omitting it. Look at the first bid nearly 70% of the students misses 0 chapter. Every single chapter is read and annotated. I don’t know about you but I certainly sometimes miss deadlines. I have other things that take my time and you know student’s might get sick. Or they might have an exam another class, so if we add the ones who missed one chapter and two chapters together with those that miss zero we get close to 95 percent. I think that’s as close as we’ll ever come to having every student prepared for every class. There at Ohio State they did it really good because there’s so many students, right? They found that the students using who were using Perusall score significantly higher on exams than those who don’t. Which maybe it’s not that surprising but it’s good to actually see that in in data. So what are the benefits? Virtually hundred percent completion of pre class assignments much better than I ever had with any other technique. It also means improved use of class time because now rather than repeating what’s in the book, you know which parts may need highlighting and already the students have suggested questions that you can use in class. They help you prepare, you only need to read that confusion report. It’s free to use. If you use your own text you can just upload it and done if you use a commercial textbook then what you do is you tell us which book it is and the students get eBook access. Which typically depending on whether they want temporary access or permanent access, you know, it’s as low as 30% of the cost of a textbook. So it’s a win-win-win. It’s a win for your students and it’s a win for you. And again if you want to go there Perusall.com. So, I hope that I’ve convinced you that education is not just about transferring information because we have good ways of doing that now. It’s also not about getting students to do what we do. I want my students to stand on my shoulders. I want my students to be able to solve the problems that I cannot solve. We want the next generation to solve the problems that this generation can’t. We have plenty of problems, you know, from from political problems to you know health, environment, privacy, you name it, lots of problems. We don’t know how to solve them. The next generation better figure it out and that means we must prepare our students not just to solve the problems that we have solved but the problems that have not yet been solved. And I think the only way to do that is to essentially engage the students actively and socially with each other both in the classroom and outside of the classroom. Thank you very much.