John Cochrane of the University of Chicago talks to EconTalk host Russ Roberts about the experience of teaching a massive open online course (MOOC)--a class delivered over the internet available to anyone around the world. Cochrane contrasts the mechanics of preparing the class, his perception of the advantages and disadvantages of a MOOC relative to a standard in-person classroom, and the potential for MOOCs to disrupt traditional education.
Play
Time: 1:01:28
How do I listen to a podcast?
Download
Size:28.2 MB
Right-click or Option-click, and select "Save Link/Target As MP3.
Readings and Links related to this podcast episode
Related Readings
HIDE READINGS
About this week's guest:
John Cochrane's Home page
The Grumpy Economist, John Cochrane's blog.
About ideas and people mentioned in this podcast episode:
Articles:
Education, by Linda Gorman. Concise Encyclopedia of Economics.
Arthur Pigou. Biography. Concise Encyclopedia of Economics.
Alfred Marshall. Biography. Concise Encyclopedia of Economics.
Web Pages and Resources:
"Asset Pricing" at Coursera. John Cochrane's class.
Podcast Episodes, Videos, and Blog Entries:
"Mooconomics," by John Cochrane. The Grumpy Economist, February 9, 2014.
Arnold Kling on Education and the Internet. EconTalk.
Doug Lemov on Teaching. EconTalk.
Highlights
Time
Podcast Episode Highlights
HIDE HIGHLIGHTS
0:33
Intro. [Recording date: March 10, 2014.] Russ: John, you recently taught your first MOOC--and that stands for Massive Open Online Course. And we're going to talk about that experience--what you learned and what you think of the potential for MOOCs to disrupt traditional education. So, let's start with the course itself. What was the subject matter? And give us a rough idea of what it covered. Guest: So, I decided for my first MOOC to do a Ph.D. level asset pricing class. So, it's the first Ph.D. course in asset pricing. And it covers the standard theory of how we think about stocks, bonds, and how they relate to the macroeconomy. I did that course first because it's very straightforward material--not a lot of interpretation; it's kind of math that you learn and learn to interpret. And I thought that would port[?] well to MOOCs in my first effort, and then I could go on and do more discussion and interpretive stuff once I learned the technology. Russ: And what was the--do you have the formal name of the class? Does it have a formal name? Guest: Asset Pricing. That's the name. Yes. Go to Coursera and look up asset pricing and you'll be able to look at it now. I leave it open for people who are interested. Russ: So, anybody can go there now and take it for free? Or do you have to pay something? Guest: Uh, yes. You can go now and go through it. MOOCs are organized into sessions, where actually taking it you have to do it while it's offered, along with the other community of people who are taking it. And then a professor like me can leave it up there so people can go through it. And most of it I think is still available now. Russ: You should point out, in case those listening don't know what a Ph.D. level finance class on asset pricing is about: it's a very mathematical class. It's very challenging. It doesn't have, as you said, a lot of casual intuition about, say, how to be an investor. Correct? Guest: That's right. This is not, How to make money in the stock market. This is--I think it is useful for lots of people. This is the theory of how we think markets come to equilibrium and all sorts of apparently-interesting ways of making money really are just ways of taking on risk. But the language is mathematical. It's not really so much doing math as staring at math and getting the intuition about it. And it's useful if you want to learn how to read the academic literature on asset pricing. Yeah. Russ: So, how many people took the class? Guest: Like all of these, a lot of people were interested. I got 37,000 who signed up. But in the end 4000 were watching the videos. And about 300 took it all the way through and took the exam. This class took about 15 hours a week outside of class. There was--I ported all my Ph.D. level problem sets to Coursera. So that was a lot of work. One thing I learned was there is a larger demand to watch the videos and take some quizzes than there is to do 15 hours a week of hard problem sets. Russ: Surprise. Was it also offered at the same time as a regular class at the U. of Chicago? Guest: Yes. I based this class on my Ph.D. class. Another reason I did this class was because I had it all pretty well prepared. And I did them at the same time. Part of the thing I was experimenting with was the flip-the-classroom concept, where my on-campus class took the online Coursera class first and then came to the class for discussion and elaboration. Not watching my butt while I put equations on the board. Russ: And how many people were in that Chicago class, in the real, non-virtual class? Guest: We had about 30 people in the Chicago class. Russ: So of the 300 that completed the coursework and did the exams, so close to 30 presumably were your regular Ph.D. students, and the other 270 were people from all over the country or all over the world. Do you have any idea where they are from? Guest: Yeah. A good geographic distribution. A lot from Europe. A bunch of Russians. Some Latin Americans. A bunch of Americans. So, people from all over. We didn't really get that many Chinese, but I think that's because my main method of advertising the class was my blog, and my blog is blocked in China. So they didn't hear about it. Maybe next time. And that's one of the interesting things, is I did get learning who is interested in this. A bunch of Booth (U. of Chicago Booth School of Business) alumns were interested in it. Ph.D. students at other universities. A lot of people in industry who wanted to deepen their knowledge of where the models came from that they were using. So, interesting spread of people. Faculty at other institutions, who wanted to catch up on asset pricing--they might have been experts in something else. Russ: And when you are talking about that survey, that distribution of folks and the differences between them, that would be true of the 4000, not just the 300? 37,000 were interested; 4000 watched some or all the videos; and 300 did all the work. Is that a good way to describe it? Guest: 4000 got through all the videos and 300 did all the work. MOOCs usually have about a 10% per week falloff rate and mine did as well. There is a quick falloff as the shoppers leave. And then about 5-10% per week after that. And that's one of the big issues of MOOCs, is keeping people engaged and getting them to finish. The initial falloff, I don't think is so bad. Because people shop, and that's the whole point. The whole thing about MOOC is it's free to sign up. So the 37,000 look at my first lecture--which was a review of stochastic calculus--and they said, Hmm, that's not for me. And so that's appropriate. Russ: Yeah. I don't understand this idea that if people don't finish there is something wrong with the course. It's like saying you should finish every book. When I was younger I finished every book I started. It's a bad policy. It took me a long time to grow out of that. There's some value to it. But this idea that somehow if you try something and you don't like it, there's something wrong with what you tried seems like a strange idea. Guest: Well, I think there are two aspects to that. One is, you are exactly right. We shouldn't look at that initial drop-off. However, the big question is: Do MOOCs substitute for conventional education? And one of the functions that conventional education does is it's a social and precommitment mechanism. You get in here and even though around week 7 you think, Spring is coming; boy, I'd rather go outside than finish this class--it's on a schedule. The threat of the bad grade keeps you to keep going. So--it would be nice if MOOCs could serve some of that function as well. Russ: Yeah, we'll get into that. So, talk about the technology of what you had to do to actually get the course up online. How much time it took, what the experience was like. Give us a little bit of the flavor of that. Because I think most people think, oh, you just get in front of a camera and they film you and that's nice. But that's not what happens. Guest: Yeah. I thought so, too. Which I quickly learned to regret. MOOCs are, like all technology, an innovation that gives you 0 marginal cost and a very large fixed cost. So, adding an extra student to the MOOC costs nothing, whereas of course classrooms are a limited size and expanding is hard there. That's a good part of MOOCs. The downside of MOOCs is like all technology the fixed cost is high. It is much more expensive in time and effort to get something up as a MOOC, and to revise a MOOC, than it is to just put your notes together and go give a lecture. Part of that is learning the technology, and then part of it is adapting the technology to what you do. Like designing a webpage. It takes a lot of time to do it. So that was my first--not just time and technological expertise. It takes a team. I had a great support team here at the U. of Chicago who not just knew the technology but knew the pedagogy, knew how to do it. I thought: Oh, yeah, I'll just take my class. 'No, no, no, no, no. You have to prepare short videos with in-video[?] quizzes, get that all set ahead of time.' And I was very grateful for this [?]. So, to put together a good MOOC, you really don't just turn on your laptop webcam and start talking. It really helps to have a team of people who know what they are doing, who understand the pedagogy of it, understand how to use the technology. The technology is already getting to the point where you don't just want to sit down with this and put the MOOC on. You want help. I could go on. Russ: So, in thinking about-- Guest: Sorry. I could go on and on about my experience with the technology and how it works. I'll let you probe.
9:44
Russ: First I want to go on the other side of the screen. I want to talk about the student experience. And I've looked at a little bit of the class. I was a little surprised. It is you filmed at the blackboard. That is, it's not the blackboard blown up to fill the screen with your voice as a voice-over. The filming was you writing down equations, talking about them, gesturing, pointing like a regular classroom. So if I'm watching this as a student, I'd see that the way I would see you in the classroom. What's different? What role do the quizzes play? Talk about how they were set up. And other assignments. How were they graded? And office hours, to the extent they were possible. Guest: Okay. So the student experience, at least in my MOOC. And there is an interesting question, how do we use this technology? Because there are lots of different ways to set up the student experience. But the student experience in my MOOC was, you would go to the website, which would give you your weekly tasks; and it would say: Do some reading, maybe. And then it would say: Watch the lecture videos. The lecture videos were designed--they were about 5-10 minutes each. They were supposed to be 5 and sometimes they were a little bit longer. This class is about looking at equations, looking at graphs, looking at empirical results, understanding how finance works. One of the challenges is how do you both talk and present visual material? That's a challenge for how do you do that stuff. In my case, I primarily put things on the board, because I have a very visual style. And I don't like slides, because slides, you lose track of what was on a previous slide. So the entire lecture is on the board and then we can point to things. You, the student, can also get a PDF of what the board looks like, so you can see the whole thing in front of you. And we point to various equations and talk about them, and I'll put a table up with numbers and we'll talk about what they mean. Then there's a little in-video quiz to try to keep you awake. So you watch the video. They you go on and do the quizzes, so sort of weekly set of assignments. The first set of quizzes are multiple choice. The quizzes for now are basically multiple choice or numeric entry--really the only thing that works reliably. So, you take some multiple choice quizzes about the concepts we did. And then for my class you go on to another set of harder problem sets, which are like the sheet[?] problems [?] deducing equation derivations and I check that you got the right answer by asking you to fill in numbers to the final equations. So, you get stuck, what do you do? And there are forums. And I think as this thing develops, the social aspect is the part that really needs to develop a lot more. We have that now in the forums. It's a discussion forum. The students talk to each other: Help, how do we do this quiz? How do we do that quiz? What was going on? I had TAs (teaching assistants) monitoring the forums, and they were very helpful. Especially if there were any typos or anything, they were in there quickly. I monitored the forum; it didn't take a lot of time. Once a day I would check in and see how people were and chat back and forth on the forums. We also had Google Hangouts, where we would have sort of like office hours. It worked best to have students email questions; my TAs would go through the questions, find some good ones, ask them, and then I would answer them in real time. Obviously Google Hangouts are a limited technology, and that needs to be spiffed up; and I hope Coursera will put that in its own technology as well. That all develops that sense of community, which is where I think MOOCs really need to go. So that's roughly the experience. Russ: How many people would come to those Hangouts? Guest: That's a good question. Now Google Hangouts are limited to 10 active people, and we found that Google Hangouts, the voice in our direction [?] didn't really work. So the answer is in some sense we don't know. We were getting emails from, I don't know, 20-25 people typically. And then we would post the Google Hangout onto YouTube, so people could watch it either at the same time or then they could watch it whenever--because they are in different time zones. So I don't know how many people were actually watching the Google Hangouts/YouTube thing. But the students reported that they liked them a lot. It gave them an informal time to talk. We talked about things other than the class--you know, how do I become an academic? When the Nobel Prizes came through we talked about all the finance Nobel Prizes, and gave them a chance to know me, and they said they liked that a lot. Russ: That's nice.
14:12
Russ: You've written that the MOOC is the equivalent, the modern equivalent of a textbook. What do you mean by that? Guest: Well, so there is a question: what is the MOOC, where will it go? One note that we talk a lot about how it threatens college education. Well, one thing I learned is that mostly MOOCs right now are adult education. They are people who are not going to college or interested in going to college. And mine, too. Most people were not people who were a threat to the university's business. But as far as textbook: people keep saying, is this going to displace us? No one will teach classes again. And my funny story is--yeah, in 1492, Gutenberg invented moveable type, so textbooks will be presented; they'll just read the textbook and no one will come to our lectures. Well, that didn't work out. Russ: Although it does in some classes. I'm sorry to say. Guest: Well, yeah. But I do think MOOCs--finally enough funny stories. How will MOOCs function in the marketplace? One strong function already happening is that the MOOC class is something that a group of students at a university might take under the supervision of a faculty member; and then the faculty member would use it for his or her own flipped classroom. You take class at a university, your assignments are: Go to Cochrane's MOOC for week 1; then come in and we'll talk some more; and you'll take final exam here. So in that sense, functioning very much like a textbook. The same flipped classroom I used, somebody else could use as well. Or, if you like just reading a textbook, read a textbook. But I do think--the MOOC experience is not just a complete substitute for taking a class. It is also a set of tools and materials that are the foundation for somebody else teaching class. Much the same way a textbook is. Russ: Well, let's talk about the flipped part of your class, the live part that you did at the university. Had you done that before? What was it like? What do you think the students got out of it compared to the students that didn't get the flipped part, who just did it online? Guest: Yeah, I thought it worked well. You know, I hate to sound cold, but young people today don't read. And if you give a reading assignment and you say, Come in on Monday and discuss it, they look at you with blank faces. If you say: Watch a set of 5-7 minute videos, do some multiple choice tests, multiple choice quizzes--the quizzes also can refer to the reading assignments; by the way, I'm checking those quizzes and you are going to be graded on them and you've got to get them in Sunday night at midnight. Lo and behold the students come in Monday morning just very well prepared. They've gotten the basic factual material that I present in a me-talk/you-listen lecture and ready to challenge me and really think about it. So--now the flipped classroom is a lot harder for the faculty member. It's really easy to just stand at the board, and I pretend to teach, you pretend to learn, and we write equations down. So, running a discussion of some really thoughtful Ph.D. students, that takes work but I think it's much more satisfactory when you are done. So, bottom line: I think the students in class got a better experience than the students on the MOOC. And also, it freed me up in class to do more advanced material that MOOC students didn't get. But, not everybody can spend 4 years of their life at the U. of Chicago going a Ph.D. program. So I think that people who were taking the MOOC class got a really good half of what the people on campus got. They get less of--so, where does traditional teaching still [?] less, and where do MOOCs have to go? A lot of being a teacher is not know the right answer. It's knowing all the wrong answers. And when you come in and say, 'John, I really didn't get it because I thought that when the expected return went up we would make more money', is I'm: 'No, no, no, no. That's mistake 32b-slash-5.' Russ: You've heard it 7700 times in the last 10 years. Guest: Yeah. And even the ones that I haven't heard before, we can synch through to get it. Now that is the experience that is much harder to replicate on the MOOC. The social part of MOOCs and the forms make it that that's where it goes. But really, that's what distinguishes, that's going to be the challenge. It's easy to put online: How do you pass your pilot's license test? Material that's very cut and dried. You think it's somewhat easy to put online material where we all really know what the material is and there is a well-studied pedagogy. You know, Algebra 101. Putting online things where the actual knowledge is closer to the research frontier and it's less well-studied, that's going to be harder. And getting that experience--what teachers actually do, of noticing your mistakes and helping you to correct them--that's a social part of online that I think is, it's there now, but I think that's what you still get in a good classroom. That's what's coming. Now, on the other hand, what's great about it--I'm snobby. I'm at the U. of Chicago. If you are out in the middle of nowhere, and your faculty members aren't that good, well, I think that MOOC provides the great experience--it can provide a much better experience than a third-rate college education. Russ: I think that's particularly true at the high school level. I think people complain about a video class can't substitute for a degree, face-to-face teacher. No kidding. But most people don't have a great face-to-face teacher. I'll keep my personal bias about the U. of Chicago to myself. But we all would agree that there are teachers at every university--perhaps even at the U. of Chicago--who are not so great in the classroom, not so great face to face. And the chance to learn from a great teacher anywhere in the world is a glorious thing. Let me ask you a tougher question. Do you think the people in the flipped classroom at, in Hyde Park in Chicago, got a better education than the people who just did the MOOC online? How would you compare the experience of the people who did the MOOC at Chicago with people who took the class from you with just the classroom 3, 4, 5 years ago without the MOOC technology, without the flipped part, and watched your rear end, as you put those equations on the board? And let--you know, you could chat about the Nobel Laureates that week if you wanted or as you added. It's a different educational experience. Do you have any feel or idea of what that new flipped classroom was like across those two choices? Guest: I think it was very good. And, even if one doesn't go MOOC using these online technologies for regular classroom instruction is going to improve, can improve classroom instruction a lot. Now, then, once you've created all the online materials, the marginal cost of opening it to a MOOC is zero, so that's part of where the economics methods may go, is that we're going to create these materials for in-class use and then it's free to open them up. But I think it was very good. My students came more prepared. My students told me they liked this. So my lectures, my 5-minute MOOC lectures, are very condensed. When you give a real lecture, you stop; you tell a joke; you let things sink in. You try to get a sense of are people following, are they not. Half of them aren't, and they are just kind of hoping they can go back and review it. And several students said, Look, this is hard stuff, and I loved being able to stop and rewind it, go back, check something, and not feel like I was the dummy who was holding up the class. So, even just the flat lecture part, they said they got more out of--they liked me better online than in person. And of course it let us go into deeper discussions, more intuition, and more advanced material in class. I particularly--one reason I did this is that the modern business school I think can benefit greatly from online technology. We--business schools have gone pretty much to teaching 3 hours once a week, and it's a big strain on the human brain to get 3 hours of new material once a week. Okay? I find it hard to stay away in an hour-and-a-half seminar. And three hours once a week is a lot. Well. And a lot of what we do now is 6 hours every two weeks, or international programs that are 5 concentrated days every 6 weeks. That's very hard in the traditional lecture and problem set style. Now, think of that setup with online materials, where every day you watch a snippet of lecture; you do a little bit of a quiz; you get on the forum; you talk to your professor, your TAs, you talk about the intuition on the forum. And then you come together once a week to review what you've done. If you have a professor who can get a set of exercises, a set of things we do together that really build on those things, you have a much better learning experience.
23:37
Russ: Yeah, I think that face-to-face part is--it's not--it's such a subtle thing. And it obviously varies tremendously by subject matter. One of my all-time favorite undergraduate classes was a class on Faulkner and Conrad. And we would read--I came into the class loving Conrad and not having much taste for Faulkner. But I figured I'd put up with the Faulkner so I could enjoy the Conrad. By the end of the class I loved Faulkner. I wasn't such a big Conrad fan any more. And it wasn't because my teacher taught us a lecture that I could have watched on video or face to face about how great Faulkner was. It was the conversation that took place in that classroom about--I think it was Professor Patterson, by the way; thank you. That conversation was, can't be--if somebody watched it, the film of that conversation, they wouldn't get anything close to as much out of it as those of us who participated in it. And I think that's what great learning is about. And I think the ability of the web to deliver that is limited. But positive. And it depends on the material. Guest: But we can't be snobby. Not everybody can have a great professor like you did. And so what I think the web, the online stuff, certainly does, is it brings much better access to that sort of thing to people who are out of college or out in the middle of nowhere. And, I also think this highlights, so I think the next step for MOOCs--MOOCs are now very much Web 1.0. It's one-directional. It's web pages. You are looking through a web page. The beginnings of discussion forums. But you know, we've had discussion groups since the 1990s. I think the challenge and the opportunity for it is to move into Web 2.0, where that kind of social and discussion thing can happen remotely on a fairly large scale. And then I think you'll get a lot better experience. It never will be as good as face to face with a great professor. But certainly it'll be face to face with your fellow students, and some of that discussion thing. And we've got to remember, you know, young people are very good at this. They text each other across the rooms. My kids text each other from different rooms of the same house, and seem to achieve a lot of core[?] communication that way. Russ: That's for sure.
26:05
Russ: Let's talk about the potential of MOOCs to disrupt either high school or college or university education. What do you think? Is it just a gimmick? Is it going to catch fire as the 2.0 version comes along? There are a lot of barriers to it, obviously--a lot of institutional barriers certainly at the high school level. And it requires a lot of different skills. I think the skills it takes to be a great flip-classroom teacher are not the same as the skills to be an enthralling lecturer to 500 people in a giant lecture hall. So, where do you see that going? Guest: Well, for right now, MOOCs, their market is adult education primarily. A group of people, and Coursera especially, take lots of classes. And I don't mean that disparagingly. There's a whole group who take finance classes together; and they showed up for my MOOC and helped each other out with the hard equations. Now, in a classroom setting, the hard question for MOOCs is certification. When do you get college credit? When do you get something other than, you know, your certificate from Coursera that you took this MOOC? When do employers start looking at MOOC certificates or groups of MOOC certificates as being a certification equivalent to undergraduate degrees? That's a harder question. Now, that's starting to happen. Universities, some universities, they are sort of using the MOOC as textbook and then granting--their job is to say, here's 10 MOOCs that we think are worthwhile, and we'll give you credit for taking these MOOCs, maybe with an exam that we proctor to see that you really know what you are doing. So, that's the step of disrupting college and high school, is intimately related with the step of when does taking a MOOC count for the kind of credit and credential that we give to people who take colleges and high school. That's a hard step that everyone's thinking about. Russ: It seems to me a lot of it is cultural. I mean, if I had to choose personally between or for my kids, and I have college-age or near college-age children, I think about what advice I would give them about what to do with the next four years of their life, as they are about to go to college. It's hard to imagine saying, Well, what you should do is you should stay home with us because it's cheap--and you love us, of course; nothing you'd like more than living with us for another 4 years as you enter young adulthood. You should live with us and for 4-6 hours a day you should be online watching videos and doing the homework and doing the quizzes. And you'll save--let's say we've solved the certification problem. At the end of this year you are going to fly to the U. of Chicago or wherever and you are going to take the exams for these classes. And it'll cost a tiny, tiny fraction of what it would cost otherwise. It's hard to make that leap, it seems to me. I know there are some people trying to create an all-MOOC university. I don't know how that's going. But socially, the social part of college, is one, is a whole separate question, which is a huge part of what I think people pay for when they go to college. But just on the education, part, if you want a technical degree, a degree in a STEM (Science, Technology, Engineering, Math) field that lends itself to MOOC-style precision. It's an interesting option, but it's hard to make that leap, culturally, I think. Guest: Well, culture, yeah. Let's be economists for a minute. The culture follows the economics. For the moment, if somebody comes to you and says, 'I want to fix your computer and here's my 10 MOOCs that I took,' does he really know what he's doing? And the certification part is: do they really know what they are doing? So, the college part-- Russ: Oh, John, come on. You are going to tell me that just because they have a B.A. from University X that has a nice reputation, I'm confident they know what they are doing? Guest: No. But I think that's a lot of the function of what universities do. Russ: Yeah. Guest: We don't necessarily do a very good job of it. Why do you hire somebody with a U. of Chicago or Harvard degree? Well, that's a brand. And it's a brand that is pretty careful about its reputation. It wants you to think that our students are good and you can hire them with confidence and that we have certified that they know what they are doing; and you point out, correctly, that maybe we're not as good at doing that as we say we are; and that we had better shape up. And that that's the challenge for MOOCs, too--especially if you want to start monetizing them instead of making it for free. To be able to say that you are providing that branding, that service that colleges try to provide--and some do and some don't. Russ: Well, let's talk about that monetizing, right? Because this technology in principle--let's move away from economics. I'm going to go to math. It's like economics in that there's a lot of really bad economics teachers, I think, teaching at the college and high school level. And I know that's true of math also. So, let's take math. I apologize--I can't remember the professor's name who I saw teaching a MOOC on calculus, but I watched the first--I don't know--half an hour or so, and it was fabulous. How many first half hours of calculus classes in America are fabulous? Well, there are a few. There's maybe 100. Maybe there's 50. Maybe there are even 400. But there aren't 40,000. And so when you think about all the high school students suffering through bad calculus teachers, the value of that half hour and the subsequent time is so extraordinarily great. And now we've got a technology that delivers it. And I'd love for that professor to get fabulously wealthy. I'd love for that professor to become a millionaire. Just like a great textbook writer can become wealthy. But this to me is more valuable than a great textbook. I'm not sure I could say why, but it just feels more valuable, and it seems like it's more transformative to do a great MOOC than to read a great--I guess the simple answer is generally textbooks aren't written to be read. They are written to be tasted. So here's somebody who can deliver hours of genuine insight into a very subtle and difficult area. That person should get fabulously wealthy in a market-based transformation of the education field. Can that happen? Guest: It should. But let's go back just a minute and remember. We had the technology to play videotaped lectures for 20, 30 years, and that never really took off. So, a successful MOOC is more than just great lectures. However, I agree with you totally, especially having suffered through lots of bad math classes myself, and that math is not cut and dried. It's about intuition, it's about passion. And even perhaps your bad high school math teacher might be able to sit in the flipped classroom and help students through problems that they encounter after having done the great MOOC. But yes, so, the economics of it? A large base, you can put a lot of pennies together and get a superstar economy, that's one of the monetizing ideas, is that this will make teaching like so many other things where a few superstars earn a lot of money; and other people are there at best to be glorified TAs, and won't make lot of money. That's a possibility. The flip side is that this is a 0-marginal cost business. Hard to charge money for a 0-marginal cost business. Russ: That's correct. But the certification is where I think the monetization comes in. To the extent that if people could verify that you got an A in a very hard class, a genuinely hard class. In fact, the certification could be better with a MOOC because you could actually be transparent about what an A meant. Whereas at many, many universities now an A is much less valuable than it used to be. And as a result-- Guest: It means you are breathing. Russ: What? Yeah. Guest: It means you showed up to the final exam and you can breathe. Russ: Shame on who--I don't know who; that's the problem. It's an emergent phenomenon, this phenomenon of grade inflation. It's an interesting thing. But if you said here's what you had to do to get an A in this class, it actually could have a lot more value. And you wouldn't care if 80% of the people got an A, because you'd know what it meant. You'd know, in a way you can't know about a calculus class at a university level. So I think the potential to do that is very large and I'm not sure what's holding it back. I think both at the university and the high school level. The high school level has an immense amount of material available, say at Khan Academy and elsewhere, at the level that high school students can really teach themselves a great deal of material. The areas where it's weak obviously are writing, and what we might call thinking. I don't think a lot of thinking gets taught in most high school classes, anyway. So--maybe I'm wrong. What do you think?
35:57
Guest: I think you are exactly right. There's this danger in discussing things like education, health care, to say, everybody has to have the absolute best. But if the majority of high school students could come out knowing calculus, knowing basic grammar, getting sentences out, and how to program a computer, even without the deep appreciation of literature, it would be lovely if they had--a huge step forward in our world. Russ: Seems that way. Guest: But, what you just said is interesting about the economics. You said that there's a function, the branding one, and that is: so there's 100 calculus MOOCs; which are the good ones? Maybe the economic rent will come to the person who says, Look, I've studied all of these and here's the 5 good ones. I give you the Russ Roberts Degree because you've taken this list of MOOCs, as opposed to the actual--[?] they could split it with the professor himself. The other model is, you know, our universities do a tremendous amount for brand name recognition, and once something is 0-marginal cost, we may end up just giving it away for free because everyone wants to be the big brand name of the global university. We'll see how that works out. Russ: That's going to be interesting. What I'm waiting for, and I don't know if I'll wait in vain, is for John Cochrane and 25 other, maybe 40 other, fabulous business professors around the world to offer an MBA (Master of Business Administration) that is disembodied; it's not in a physical location, except maybe for the exam, and maybe we could solve that problem as well. And that would be a very--ironically, it would be an inexpensive degree to acquire but very lucrative for the faculty who work or are involved in it. Of course, it's a different experience than being a faculty member at a normal university. You don't have the collegiality and stimulus of the research side. But it's an interesting model. We're limping toward that model. I don't see anybody sprinting toward it. Guest: Well, except--so what does the bricks-and-mortar business school do that that doesn't do? A bricks and mortar business school is selective about who they let in. In fact we are often accused at simply being really good at selecting smart people and then giving them a 2-year party. We have connections to employers. We have a fantastic office that gets them jobs. And we have an alumni network. Now I think you can form an alumni network. The people who met online in my class still stay in touch with each other, which I think is a very interesting phenomenon. And I wonder in your model whether you could be picky and say, Look, before you are allowed to take this class or be part of our MBA, you have to score so much on the GMAT (Graduate Management Admission Test) or something of the sort to do the selection part as well. But I think it's a very interesting model. Maybe I should get off my butt and do it. Russ: And the high school model is I think even more disruptive. We can joke about how modern university education has many problems, but I don't think it has as many as the modern high school. And the opportunity to home school--enough people don't like their high schools, for a variety of reasons, that they home school their kids. And this is already happening, of course. People are using these online resources with great zeal. But the opportunity to put together an entire high school experience, be an online education--which, by the way, so much time in high school is wasted. So, you could spend your mornings in class and you could spend your afternoons at museums, or building whatever you wanted in your garage or coding or doing a thousand--learning the flute--there are a thousand things you could do more effectively than sit 30 people in a room on [?] lessons and hope they behave themselves. It seems to me that the opportunity to transform high school education is huge. Guest: Absolutely. And there we've also made a transition. High school education is material that is by and large codified, even where in some cases it's codified wrong. We do not ask high school teachers to be researchers, to think of new things. We ask them to be professional pedagogues and to know how to get a codified set of material in. And there are certainly economies of scale. And, one of the great things MOOCs are doing is learning about pedagogy. At universities we are pretty bad at actually self--we are supposed to be empirical social scientists but we don't really look at how we do that well. And the MOOCs are looking at what works and what doesn't work in the way of all technology. When you get to university, the idea of the research university is that you are at a level where the textbooks are a little bit out of date, and so you get together with a professor and kind of think through things that are still at the fuzzy edge. That's going to be harder to bring to MOOCs, but frankly so little of what we actually have to teach people is at that level, even introductory university classes are pretty cut and dried as well. So, I agree. That could be a great savior of the American high school experience. And you say we'll miss the social side. I'm not sure that the majority of people think the social side of high school is a great thing. Russ: Yeah, it's a feature, not a bug.
41:27
Russ: Let me ask you a question that comes from Arnold Kling, who has been on here to talk about these issues as well. I have a couple of insights through him I want to hear your reaction to. One is that education is feedback. So, what Arnold, I think, means by that, is that the way you learn--think about this--we call feedback: I gave him a quiz, or I gave him a homework set. But what he's really talking about is a wide range of subtle things. Which is--does the student know whether the student is catching on? And I think so much, when I think of my failings as a teacher, it's when I just sort of said, Well, it's up to them; they've got to figure it out; and I'm not going to give them--I loved and still do, open-ended questions that don't have precise answers. But my students would say, 'But how am I doing?' And I'd say, Don't worry, you are learning. That's what this is about. This isn't about, I got a 73, I need to study more; I got a 96 so I can slack off. This is an intellectual adventure. Well, most students don't like intellectual adventures. Some reasons they don't like them are not so good. But some are good. They don't really know what they know. And so I think the biggest challenge of the MOOC is that. How do you give feedback? How do you help students understand what they know and don't know, to lead to mastery? Guest: The two parts of education are the social structure that keeps you going and the feedback. With a great teacher, the person who as we talked about before who knows all the wrong answers and can help you quickly straighten out a misconception, quickly see where you are kind of thinking about things wrong and bring you back to the correct path, let's you see how you are doing relative to some standard of mastery. Now, they have their quizzes. And the good part of it, actually: formalized quizzing is in some sense better than: Write an essay, I'll have a graduate student grade the essay to see if you know what you are doing. Well, that graded feedback in the traditional university, and so the old marginal cost thing--you know, once I write a good multiple choice quiz, 100,000 people can take it and get some sense of where they are. The quizzing I found was one of the weakest points of current software that I used and clearly one that can be improved dramatically. And maybe students will get--they'll certainly get better assessment than they often do now. The challenge is to get that feedback to help them get out of bad ruts. And there I think the Web 2.0 social development is going to be key to making it work better. Russ: Yeah, it's going to be interesting to see what happens there. But what I think is bizarre about the modern university is that so many professors give multiple choice exams out of what I have to assume is laziness. I've given a few in my time, but most of my exams were open-ended, challenging questions that didn't have easy, straightforward answers. In fact, they were so not-straightforward I often would grade them myself rather than using my teaching assistants. And that was very time intensive; and I think that was when I was the best teacher that I was, the more open-ended and challenging those questions were. The funny part is, when you are teaching 100,000 students you can't grade them individually. You can't go through them. You want to exploit the economies of scale you are talking about, but I think unfortunately at many modern universities, 200 or even 80 is too many for people to grade thoughtfully. So they just use multiple choice as if they are teaching--they are going to exploit the economies of scale even though they are supposedly in a small class. And I think that's just a terrible tragedy. Guest: Well, what you are saying is that what the MOOC really endangers is the 150-person lecture class at a second rate university. Because that the MOOC can do with a multiple choice quizzing and no real feedback and a graduate student who barely speaks English as your TA. That the MOOC can do a lot better. The 20-person class where you get great feedback from a great professor, that will still be there. But that's like saying, you know, Southwest Airlines isn't as good as a private jet. Well, thanks. But it is true--so one of my biggest frustrations with the current state of MOOCs in doing my quizzes and assessments is that like you, I organize my classes around weekly problem sets. And they were almost all open ended. Particularly a puzzle-type page. Here's a natural misconception; how do you resolve this question? And you've got to go resolve this question. I couldn't figure out how to ask that in machine-gradable form. Or: here's a theorem; prove it. You can't ask that in machine-gradable form. Russ: And when you do, you take away the learning. You take away the experience the student has. If you say, here's four answers, one of them is the right one, three are awful--the student can often tell, yeah, 'I know what he wants; he wants a. because b, c, and d., he never talks about those things.' But you don't learn anything from that. You don't learn anything the way you do when you have to get up in front of your study mates and defend an answer to a puzzle that you think is right and they think is awful. Guest: Now, on the horizon I think they can do a little better. So, you certainly can do, ask a question and write a paragraph, and you can think of ways of grading a paragraph that can mechanically grade[?] or I think. So I often do this, my actual grading--I hate to tell this; my actual students might be listening--I'll ask: Here's a puzzle; the expected return on one strategy looks better than another; how do you solve this? And the way I grade it is if the word 'beta' appears in the answer they get credit, and if it doesn't, they don't. Russ: Oh, John. I'll edit this out. Yeah. The equivalent for me is: Here's a puzzle that I thought of for my kids. This is what I call dinner table economics. The puzzle is: Why do lightbulbs last longer than they used to? Because a lot of people think, lightbulbs wear out because they want to sell you more lightbulbs. It's a really bad answer. So if that was one of the answers--again, if you wrote that down as one of the answers, compared to, say, the correct answer--which I'm not going to say--it's not a hard question but you should think about it out there--the right answer is, you've got to think about it, but there is a key word in that right answer and if I saw it, yeah, I could grade it pretty quickly. Guest: Yeah. Now, on the other hand, I've gotten pretty good at multiple choice questions. Well done multiple choice questions. Russ: Yeah, they can be done well. Guest: You can ask people to think. But how do you machine grade assessments--that's one of the big challenges. Russ: Yeah. I'm not sure it's ever going to be fully solved. But maybe it will just be solved pretty well and that will be good enough.
48:30
Russ: I want to give you another, a different kind of challenge. Let's take me. I'm your student. I'm your only student. There's a famous, I think I've talked about this on EconTalk before. I think Pigou--tell me if you know this story. I think Pigou took a class from Marshall where Marshall came to teach the class and Pigou was the only student. I may not have the names right but there's a famous British economist who showed up every day or a couple of times a week. And he had one student; and he taught the class as if there were 80 students there. And he just lectured, and the student, if it was Pigou, took notes; and that was the class. Now, Arnold Kling again raised the following question: he says, I think that the best way to think about teaching is to think about what would you do if you had just one student in the room, rather than thinking about wow, if you get 100,000 students to watch this lecture. So, let's take me. I'm your student, John. I know a little bit about finance, not that much. I'm okay at math; I'm not a star. I can do some of the math; but some of the math in your class is challenging for me; I'd have to tool up. I show up at your door and I want you to create a class for me. Now of course the ideal isn't--you don't have time to create a class for me. So I want you to create an online class but it's for people like me. Is that different from what you do for 100,000 people? That's my question. Guest: Yes. So, two answers to that question. One thing that implicitly you bring up is that MOOCs may allow a lot more customization. You can do MOOCs on subjects--so in the modern university you cannot teach a one-person class. Sorry. We watch the bottom line. But if you are interested in early 13th century poetry-- Russ: We can teach three people sometimes. You're making it sound like-- Guest: But if you want to teach a class in early 13th century Iranian poetry or something of that sort, there are 150 people around the world who might be interested in that. So one thing this might do is to give us classes that are both more specialized in a topic and more specialized to the person. There could be--there are already 100 Introduction to Finance classes. And there is one out there that is exactly right for your interest and your level. So to some extent the MOOC is going to do that. The thing it's not going to do, which I would do with you for a 1-on-1, is of course, I would not give a lecture. We would talk and it would be a lot more of me listening. In my other instruction life, I'm a flight instructor. Which is done one on one, and where assessing the student's competence is really important. And where assessing the student's misconceptions about how things work is really important. And that's what you do when you are one-on-one and the guy needs to learn how to fly the plane. And by 1-on-1 sort of quizzing, I'll pose a puzzle; you tell me the answer; I'll go, is that really how it works? We really explore what you understand and what you misunderstand. That's the way you teach 1-on-1 classes, and that's the thing that's hard to do on a MOOC. Would you really trust a pilot of your plane who said, I learned to fly on a MOOC and a simulator? He might be darned good. And he would certainly have run through all sorts of accident scenarios that the MOOC and the simulator did, but there might be a few remaining misconceptions about things that had gotten through the process that you might worry about. Russ: But to think about me again, and I think your point about specialization, customization, is exactly right; and I think we've just scratched the surface on this. Right now the way it's customized is: Oh, yeah, I know this material; I'll just fast forward through it. Or: I don't know it; I'll play it three or four times. And that's great. But obviously there is going to be a way where you can say, I want to go down this path or that path, and there is going to be menus and options that aren't available right now. And one way to think about this is you have the high school student who is a superstar and is being held back--you just give them extra problems, you give them extra reading, and so on. And I think the MOOC has the potential to do that with extraordinary success. But I'm thinking about just the pedagogy for a minute. Like you said, you wouldn't lecture to me, right? So what would you have me do? Would you have me read? Would you have me watch the videos and then we'd talk? Would you probe and quiz and challenge me to see what I knew? What would be the pedagogical differences between that one-person class and a bigger class? Guest: Yeah. If I had all these materials, I actually would have you watch videos for just straight explanation stuff, rather than me give a personalized lecture, one-on-one. I would save my time. We're also presuming my time--I hope your example wasn't one where we presume my time is completely worthless. Russ: Right; you're just at my beck and call, John, for all my dumb questions. That's what it is. Guest: As with my flight instruction students: I say, Look, I am not going to give you a one-on-one lecture. You are going to read the book, and you are going to come back and we are going to talk about the book. And so I would have you, I think the MOOC materials would be great. But then we would spend a lot of time one on one probing what you know and what you don't know, and helping straighten things out. Working on problems together. That would be the ideal thing to do in the one on one. But, you know, not everyone can have that. I think the economics you set out exactly: How do we leverage the talents of the best possible ones to give something that's pretty darn good to 100,000 people? And maybe more tailored to what they know than even being in the class with 1 great professor and 25 other dumb students isn't that great either.
54:26
Russ: My hope is that--I want the technology to leverage something you referred to in passing, which I think is under-rated, which is the wrong answers. I've always dreamed about a problem-set based class which could be delivered to lots of people because I've done it so many times I know all the wrong answers. I know the likely mistakes you are going to make, so when you make those, I don't just say, 'Wrong', and you get to guess again. I say, 'Wrong; here's a hint.' And you get to go back into the tree and figure out which branch you should have gone down. I think that's the goal. That's the standard that we could maybe hit some day. [more to come, 55:06]