Daphne Koller of Coursera talks with EconTalk host Russ Roberts about online educational website Coursera and the future of education both online and via bricks-and-mortar. Koller, co-founder of Coursera with Andrew Ng, explains how Coursera partners with universities, how they try to create community and interaction, and the likely impact of widespread digital education on universities and those who want to learn. The conversation includes a discussion of why Koller left a chaired position in computer science at Stanford University to run a for-profit start-up in a crowded field.
Play
Time: 1:02:44
How do I listen to a podcast?
Download
Size:28.8 MB
Right-click or Option-click, and select "Save Link/Target As MP3.
Readings and Links related to this podcast episode
Related Readings
HIDE READINGS
About this week's guest:
Daphne Koller's Home page
Coursera.org.
About ideas and people mentioned in this podcast episode:
Articles:
"C-Path: Updating the Art of Pathology", by Mike Martin. Journal of the National Cancer Institute. Aug. 21, 2012.
Podcast Episodes, Videos, and Blog Entries:
Arnold Kling on Education and the Internet. EconTalk.
John Cochrane on Education and MOOCs. EconTalk.
D.G. Myers on Cancer, Dying, and Living. Orson Scott Card on editing. EconTalk.
Lant Pritchett on Education in Poor Countries. EconTalk.
Doug Lemov on Teaching. How to teach teachers. EconTalk.
Highlights
Time
Podcast Episode Highlights
HIDE HIGHLIGHTS
0:33
Intro. [Recording date: July 31, 2014.] Russ: So, I want to start with Coursera. It's a for-profit company that collaborates with non-profit universities. Tell me how it works and what its goals are. Guest: So, our goal is to provide free education offered by our university partners to learners everywhere. We partner with a university. They are responsible for developing content; they put the courses up online on our platform. We host and stream them to learners worldwide. We have learners in every country around the world, and we help deliver the content, market it, and the hope is that we will be able to provide some revenue to help sustain the effort which gets shared back to our university partners. Russ: And talk about, first, what the magnitude is of your reach right now. So, how many people typically are listening; how many from different countries; how many universities have you partnered with? Guest: So, we currently have 110 university partners from over 20 different countries. We have top universities, not only in the United States but also from countries like Switzerland, France, the United Kingdom, Germany, Singapore, China, Taiwan, Australia, Canada, Mexico, and many others. We have over 8 million learners worldwide, and they are in every single country around the world. We even found recently that we have learners from North Korea, which we hadn't [?] Russ: Impressive. So, what does Coursera provide in common across all the different course offerings? You say the universities tape the course; they put it up in the way that they choose to. Some people film it; some people use visuals with a voice-over, I assume. Everybody has a different style on their courses. What's common to Coursera's offering besides the platform that people are using to access it? Guest: First of all, I think that there is a certain commonality. Most people use video as the primary mechanism for the delivery of the content. We also very much encourage the delivery of very short video modules that are specifically designed for online consumption. We strongly discourage and have hardly any lecture captured from the back of the classroom. It just doesn't give a good online experience. The other commonalities are that these are fully self-contained courses that not only have the video content but also meaningful exercises and assessments, a community of learners that are helping each other in kind of a peer-teaching model. And in the vast majority of our courses there is also a form of credential that our students get at the end, which is not a credit-bearing credential in the sense that it doesn't lead to a degree, but is nevertheless something that a learner could potentially take to an employer and get some tangible benefit from. Russ: And that right now, that certification, is just that you took the course, completed the assignments? Is there are grade associated with some, all of these? Guest: Well, in order to get the certificate, you have not only to complete the assignments; you have to complete them to a certain level of satisfaction of the threshold the instructor provides. So, there is no grade per se but there is a passing threshold. Russ: Okay. Now the space of online education offerings particularly at the university level has gotten more crowded in the last few years. There's a lot of people trying to do what you are doing in different ways. Talk about where you fit in, in that space, and who you see as your competitors. Guest: So, first of all, we're the first in the space, what launched this whole thing. We're the original Stanford MOOCs that were launched in September, October of 2011. Russ: A MOOC being a Massive, Open, Online Course, right? Guest: It is, thank you. And so, we're not only the first, we are also the largest. So, we're 3-4 times the size I think, or even more, of our closest, the next closest platform in this space. So, you know, there's a proliferation of different entrants in this space, as in any area that's really exciting. I think one of the interesting phenomena that are happening recently is the proliferation of national platforms. There is a French platform, a Chinese one; United Kingdom, Australia; one in the Hispanic world; and so on and so forth. I think that some countries are viewing it as a matter of national pride--their very own platform. In addition to those, two of the bigger platforms that have typically been associated with the space are, first, Udacity, which has since moved more into content that is primarily provided by companies and less so by universities, and so they've taken a somewhat different trajectory. Closer to us is edX, which is a joint effort of Harvard and MIT. They have also recruited additional universities worldwide. They are more similar to us in the vision, however significant the differences in platform. And as I said, we're considerably larger. Russ: Are you a threat to the university? Guest: No. Most definitely not. Russ: So explain why not. Guest: So, initially a lot of the media coverage had it as, Oh, you are going to put universities out of business. And what we're discovering is that our target audience by and large are continuing education learners, people who either have completed their education or in the case of some developing countries maybe never had access to education to begin with. They are now working adults and they are looking to make a better life for themselves, either by enhancing their career, in some cases just by expanding their minds. Less than 15% of our learners are current learners in existing undergraduate institutions and they are mostly taking it as supplementary material, akin to reading a book. I do believe that this will eventually transform higher education because people will realize that the educational experience cannot be about the deliver of content, because content is now ubiquitous. And so what I think we'll see is a move away from content [?] as the prime component of educational experience and have it be more about skill building, collaborative work on projects, and so on and so forth. Which I think will give rise to much better learning outcomes.
7:24
Russ: You mentioned Udacity. Sebastian Thrun launched Udacity with a great deal of fanfare, and a great deal of resources. And the early offerings, some of them were quite spectacular, and quite successful in the sense that a lot of people took the classes. And he got very demoralized by his experience in trying to take that material into a university. Do you understand why? I don't, fully. And as a result, as you said, they have shifted more toward corporate training and other areas. I don't get it. What are your thoughts? Guest: I think one of the experiences that caused Sebastian to shift away from the original trajectory was the experience that he had with San Jose State, where he tried to take some of the courses, or at least the philosophy behind this teaching, and apply it to remedial classes. I think the real struggle is that students in these remedial classes are ones for whom the educational system has largely failed. That is, they didn't develop in school, in high school, the study skills that they needed in order to succeed. Which is why they are in the developmental track. And if you take those students and you just plunk them in front of a computer, I don't think it's a great recipe for success. Whereas there are other populations of learners that this is a much more successful model for. Russ: So, his discouragement was the fact that they didn't do very well in the tests and retention or whatever measures that were used. But I thought it's sure awfully early to be turning your back on this technology because of, really, one data point. I don't get it. Is there more to it than that? That you think? Guest: I agree with you; and that's why we have not turned our back on this technology. I think it's important to identify the right--you are not going to find a silver bullet to education. There's not going to be a single technology that works well for all types of content and all populations. So I think that particular instance was a mismatch between the technology and the kinds of learners at which it was aimed. I think those learners, and there are studies that prove that benefit tremendously from more blended learning approach; and some of the other results even in San Jose State as well as a number of our partners, show significant improvements in learning. Because when you do blended learning that involves both technology and some kind of face-to-face interaction, the technology that we're currently using in terms of the open access, the ones that go direct to consumers, they work really well for a different type of population. And that's great. Russ: The other part that I find strange is people get discouraged by the lack, the percentage of students who finish the course. And I'm thinking--so what? So, they only go two-thirds of the way through. Does it mean that they "gave up"? It could be that they got what they wanted out of it. They were changed by it. Do you have any sense of or feedback or measure of those kind of students, who "don't finish"? Guest: So, first of all, it's a very astute comment. And in fact it's important to realize that signing up, the enrollment numbers for any of these courses, is akin to putting a little x in the course catalog of your university saying, I might drop in on that class if I'm inclined. So half of the people who put that little x never even show up to the first lecture. Half of the ones that do realize after watching the first lecture that this isn't what they were looking for. They thought astrobiology was about UFOs (Unidentified Flying Objects) and it turned out that it's not. And so, of the ones that, after a couple of weeks of what you might call a shopping period in a college, are declaring that they are committed to taking the class, the fraction of those that complete the class [?] 63%. Russ: How much? Guest: 63%. Russ: Which is not bad. Guest: Which is not bad for an online course. And remember this is a free online course. They didn't put any skin in the game. If they put a little bit of skin in the game and sign up for our verified certificate program, the so-called Signature Track, then the completion rates get close to 90%--85, 88%. And that's $50. It's not the same as paying for real tuition at a college. And nevertheless we're getting these fairly impressive completion rates. Russ: So, Khan Academy, which is related but different--they tend to be focused on high schools and K-12 learning--but they are in this same space. They are trying to make progress in this question of how do we teach people virtually and digitally. My understanding is they spend a lot of time trying to give feedback to teachers about how much time their students spend on the website. It's one way to monitor, for younger students, whether they are actually doing the assignments. Are you, is Coursera and people in the older-age-group space doing anything like that? Are you learning anything about engagement and those kind of variables? Guest: We're definitely monitoring questions of engagement and thinking about what are the right ways to increase engagement, which I believe are likely to be different for a working adult of 35 versus a 12-year-old trying to learn multiplying decimals. So, we're looking at that, and we have some ideas on how to develop that. In the context of adults, often a social structure is a really great way of increasing retention and engagement. So, one of the efforts that we've made along those lines is our Learning Hubs project, which started out as a collaboration with the U.S. State Department and has since expanded to a bunch of other partners, including your public libraries, the Carlos Slim Philanthropic Foundation in Mexico. So there's now dozens or maybe even hundreds of sites where students get together-- Russ: Physically. Guest: Physically, once a week, to discuss the material, often under the facilitation of another person. And the completion rates according to the data collected by the State Department in those learning hubs is around 70% is typical and sometimes north of 90%. So, I think it's a very different type of audience and we need to find the right solution. Like I said, no silver bullet. Russ: I totally agree with that.
13:49
Russ: Let's talk about some of the challenges of online education generally. Arnold Kling was a guest on these topics a while back and he said something I think was rather extraordinary. He said--and he's a Ph.D. economist but he also teaches high school and he's very interested in education generally--he said, Education is feedback. And the way that students learn is they find out whether they are learning. And then they find out they need to learn something else, or they are not really grabbing what they want. And I think, as a university professor, that education as feedback is not one of my strong suits. And when I was involved in the classroom, I tended to just have the attitude, Well, you figure it out; if you don't, that's up to you. Students in my classes don't always like the fact that they don't always know how they were doing. And I understand that. But Arnold's saying something more important, which is, it's not just comforting; it really changes the educational process. So, how are you trying to give that feedback in this online setting where professors can't see the faces of the students and repeat stuff, etc.? What's Coursera up to there? Guest: First of all, I completely agree that getting feedback on whether you are learning or not is an absolutely essential component of successful learning for the vast majority of people. And so that was something that we built into the platform from Day One, for example first by the introduction of auto-graded assessments, so that you don't just submit or do the work blindly and then hope that you got it right-- Russ: Eventually. Eventually you might get told by your professor that-- Guest: Well, in this case, there is no professor who is checking your work. Because the kind of scales that we're dealing with, you are not going to have someone grading the assignments of 100,000 students. So if you want to give somebody any feedback, you had to do it using some other mechanism. And so the first one that we put in place is autograding, where the computer checks your assignments and provides you with feedback on whether you are right or wrong. Now, what turned out to be a nice thing about that is that not only was this a good scalability mechanism; it actually turned out to have pedagogical benefits. Because the fact that you are able to give someone immediate feedback, and then also give them a chance to try again--because the computer doesn't mind grading the same work twice--then you actually have the opportunity to really let people achieve mastery in one topic before you move on to the next. And there's studies that date back to Benjamin Bloom, 30 years ago, that really already demonstrate that mastery learning is a really valuable way of improving learning [?]. So that's one source of feedback. The other source of feedback that we've built into the experience is feedback from your peers--from other learners in the class. And that turns out to be incredibly valuable, because, first, it not only provides you with a scalable mechanism for feedback, it also creates a sense of community. So you feel like there's other people learning with you. Which is really important. And then finally, and I think even most importantly or at least as importantly, is that the process of giving feedback to someone else is an incredibly valuable learning experience. Learning how to critique someone teaches you what works well and what doesn't. And then you, in the peer grading mechanism that we've put in place, you can go back and assess your own work using the same set of criteria; and say, 'Huh. I could have done that better.' And so it's something that we are constantly told by our learners is one of the most interesting components of their learning experience, participating in the feedback process. Russ: Yeah, we talked in a recent episode about the technique that I learned from Orson Scott Card of, when you want to teach writing, you really want to teach editing. You want to teach students to critique each other and then maybe they'll get good at editing their own work. Which is what--I think rewriting is the great unappreciated skill of writing. Guest: Oh, fascinating. Russ: It's genius. It's fabulous. It's great. And he would grade the students not only on how well they wrote, but how well they critiqued. Guest: That's awesome. Russ: And so instead of just saying, 'Oh, that was a nice paper,' you found yourself working harder to give that feedback, which meant that you learned that process and could critique yourself.
18:11
Russ: Let's talk about the multiple choice issue and autograding. Because I think one of the challenges is--I didn't like multiple choice questions. They are easier to grade, obviously; but I generally didn't use them when I was teaching, even in large classes. Which meant that grading was very painful. But the challenge is: You can write a good multiple choice question. It's not easy, but it can be done. And you can write a multiple choice question so that the student actually learns how well they understand the material as opposed to their ability to spit it back. But the question is, in your framework, you've got hundreds of faculty around the country and maybe some around the world, developing these courses; who is writing those exams? Those autograded experiences? Are you helping them? Because some of them, I'm sure, aren't doing it very well. Guest: So, first, before I answer your actual question let me just say that multiple choice is only one of several autograded types of assessments that we can provide. In addition, you can also use short answer questions that can also be autograded. Things that have a clear formatted output can be autograded--like the output of computer programs or computer models or Excel spreadsheets; math can be autograded. I think there are a lot of other things beyond multiple choice. Russ: Meaning, you can have an open-ended question and you can either get it right or wrong and have a shot at it again, as opposed to just a choice of 5 answers. Guest: That's right. That's exactly right. Which is a better learning experience already. Although you can't ask people to write an essay and have it autograded. That's why we're doing the peer grading. In terms of the question you actually asked, which is, Do we provide our instructors with help? So, the answer is, Yes. We have a fairly extensive set of documents that we've put together on how to construct a good peer-graded assessment, how to construct a multiple choice question; we have a set of partnership managers who work with instructors as they are putting the course together and help provide them feedback. But I think even more important, we work with the instructional design team at the different partner institutions and teach them the skills because that is a much more scalable way, because they can then use what we teach them for multiple instructors at that institution. Russ: But that design team is domestic, or yours? Guest: That design team is domestic. It's at the institution. And most of these top institutions have a center for teaching and learning, for teaching excellence. So that's one way that we provide them with input. The other way is because we have this amazing community of 110 top university partners from around the world, they also communicate with each other. We have a partners portal, where people say, 'Hey, look, I did something really cool with--for example, how to teach a lab course online. Here's what I did.' And other people say, 'That's interesting; I could use that in my class.' And so that's how we create a community of practice and innovation. Russ: Obviously a lot of this is, people not [?] carefully reinvent the wheel. We should steal a lot of stuff here. We could. Guest: Totally. Russ: Give me a feel for the logistics. So, some of these are courses that are offered in real time and people are taking them. But a lot of them are sitting there right now; anybody can take them whenever they want. Correct? Guest: Well, the second half is correct. Real time is a little bit of a-- Russ: I didn't mean that. I interviewed John Cochrane, who taught a Coursera MOOC in finance; and he had students taking it as part of his university class, and they were being graded. But at the same time people were taking it online, around the world. Guest: That's right. Russ: But now I can take that class after the class. Guest: True. Russ: Take the same quizzes, right? Guest: That's right. Russ: So, give me a feel, if you are able to, for what some of the more popular classes would be, at least in terms of type of classes. And what would be some of the numbers we're talking about here--and I assume there's a very long right-hand tail. I assume there's a handful of classes that are unbelievably population and there's a lot of classes that are not so popular. But give me a feel for that. Guest: Sure. So it's actually challenging to characterize our most popular classes because they come from a variety of disciplines. So, you might come in thinking that the most popular ones are going to be the ones in business and finance and so on. If you think about our top 3 or 4 classes, there's probably 2 philosophy classes, a psychology class, also 1 or 2 computer science classes and a couple of business classes. If you look at that top 10 list, it comes from a range of different disciplines. Which I think is really cool. There is indeed, as you said, a long tail. The smaller of our classes have an initial--Oh, I forgot to tell you. The largest of our classes have an enrollment, initially, of 180,000, 200,000. The largest we've had is a social psychology class from Wesleyan, at 250,000. Now, it's important to remember, this is initial enrollment and then not everyone shows up. Median for us is about 40,000; 30-40,000 for the median class. And then as you said there's a long tail of classes, where the smallest goes down to, you know, around 10,000 for initial enrollment. But oftentimes those are very niche topics or in languages other than English, and so appeal to a relatively smaller population. Russ: Those are amazing. Must make you feel good when you go to bed at night. Because that's an extraordinary thing. So, one of the issues we've talked about here before and I think about a lot, and I talk to my wife about it--who is a math teacher; my wife is an extraordinary math teacher. I'm biased, obviously. But she's good at teaching math, and good at motivating students. And face-to-face is very powerful. We know that. But of course, she gets to teach, whatever it is here, 50, 60 kids each year in a few classes. The idea that a great teacher could teach 250,000 is really exhilarating. So, what do you think the potential is for, say, the best calculus teacher, to teach the world? Dominate the market? That, everybody says, 'We don't have to offer calculus any more at our school, or a high school, or university, because you can just get on Coursera and take the best one? Guest: So first I think that thinking that there is the best one is an incorrect perspective, because what's best for me might not be what's best for you. I might prefer a different type of pedagogy or subset of material, and so on. Russ: There's visual learners; there are people that like to listen; there are people that like to watch, read. Guest: Sure. Russ: So, there's three. There's 3 great calculus teachers. Guest: Maybe 15. Or 20. In the same way that there's more than 3 calculus textbooks. Russ: Correct. Guest: And I think there's at least as much room for variability in the online instruction as there is in textbooks, because of style-- Russ: Maybe more. Guest: Exactly. More. Russ: You can change it; you can make a module-based way to make it even more customized. Sure. Guest: And also the style of presentation. Text is text. But the style and how do you present things and how do you use visuals can be very, very different. So, 15, 20, 30 different versions of this. But I agree--we don't need 3000. Russ: It would probably be more than that, right? There's more than 3000 people who should be doing something else, instead of teaching calculus, is what I'm thinking. Guest: So, I think that's the question: is, whether the right thing to do is to say a person, the student should just go online and just take the material entirely online in that format, or whether, for a large majority of students, especially those who are younger and less formed in their thinking, the right thing to do is to have a blend, where they get some of the content delivered online and then they come into class and the teacher actually teaches as opposed to delivers content. So, puts them together in groups and says, 'Let's solve the calculus problem together, and I'll walk around and give you advice on how you are tackling this and whether you should be doing it in a different way.' And so I think what we're likely to see is not the elimination of teachers, but rather a transformation of the teaching profession back to what it used to be. Russ: What did it used to be? Guest: Well, when people had the luxury of teaching a relatively small number of students in their class, before colleges became the only path towards a better life and we had to scale up on-campus teaching, so now we have to shovel 400 people into an auditorium, people were able to have that more 1-on-1 interaction with their students. We no longer have that luxury in most of our large state institutions. And even in some of the private ones. And so how do we provide a mechanism by which students can regain some of that experience--as can teachers?
27:27
Russ: So, let's talk about the flipped classroom. You are talking about 'blended,' and I assume by blended you mean a mix of people watching video and then other times teachers delivering the material face to face. So, the blended classroom gets a lot of love right now. The idea of course is that you watch the video on your own and then you come in and you are talking about where the teacher moves among, typically, groups. My wife responds to that--my wife's a skeptic. She's not antagonistic but she's skeptical. And she says, if you are really going to master, mathematics, say--and this would be true of many things--you kind of have to work hard. You have to bear down. You have to focus. And you can't be sitting in a group of 3 or 4 people where there's maybe a quiet person or there's maybe a bully or an aggressive person, and there's one person who takes charge. And I do think we have a little bit of romance about group learning. I'm a big fan of it. I think, in graduate school, my study groups were by far the way I learned most of my education in economics. Is that going to work at the K-12 level, and in say a big intro class? Do you think we know anything about the effectiveness of these kind of approaches? Guest: I think the data that's coming in is demonstrating good success rates for this type of blended learning format. I am a big fan, for example, of some of the studies that have been done at University of Wisconsin, Madison, where they have been doing blended learning in their introductory engineering classes for about a decade and have pretty much perfected how they go about it. And are demonstrating very significant improvements in learning outcomes, especially for disadvantaged students who come in with less background and were in the failing population prior to the move to blended learning. The results from the joint Duke-National University of Singapore Medical School in Singapore are very similar. They draw from a population that's much broader than the Duke medical campus does, people who haven't, [?] are not pre-meds. And they teach them in this blended learning format, using content that they get from Duke University. And the students in those classes who come and do this kind of group work perform on their board exams at the same level as the Duke students who come in from a much better background. So, I think that there's a lot of data to support the validity of this model. That being said, it's not a one size fits all. We've already said that. And if you are a learner who really needs to hunker down quietly and think of things for yourself, then maybe this isn't for you. But for those students, too, I would argue that even if you think the group work out and they do better by working in their own, they are still going to do better by watching the videos at their own time, at their own pace, with the ability to pause and think than they do by sitting in a lecture hall where they are certainly not interacting with anybody and the professor just moves on at a constant pace that may or may not be the right pace for them. Russ: Yeah. [?] We were talking about group learning and the classroom, and I wanted to ask--so until recently you were a faculty member at Stanford, and I'm sure like all of us with Ph.D.s went out and taught without any training in how to teach. That sometimes works out quite well. Sometimes less so. But of course, at the K-12 level, we have this thing called Education School that a lot of people go through. And I happen to be a skeptic about the value of educational theory in making great teachers. I think it could be helpful; often isn't. But what's clear to me is that we don't have a lot of training in the kind of skills that we are talking about right now. What I would call coaching, mentoring, exhorting, encouraging--the flip classroom skills are not the same as the skills that you would typically learn in a formal education experience. Do you think that's going to change? Do you see any signs that there's growth or enthusiasm for flipped learning, is going to change the way we train teachers? Because right now I think teachers figure it out for themselves as they go along. Like we did. Guest: Exactly. As you pointed out, that's exactly how we learned to "teach." I remember when I showed up and I was at Stanford on my first day, and they said, 'This is the class you are going to teach, and here's the pile of notes that your predecessor in this class has been using. Good luck to you.' Russ: And there was a training experience. Guest: A training experience. Exactly. And I actually have a lot of sympathy for the students that took my class the very first time. I can't imagine how horrible an experience it must have been for them. So, I'm hoping I got better over time. I think that clearly we need to train teachers better; and in Higher Ed, we need to train them period. Which is currently not happening at all. I do expect that as this form of education becomes more popular we will start training at least the K-12 teachers, who are the only ones that are currently receiving any kind of training in doing this. And again, it's taking time for people to realize the benefits of this approach. But the data are starting to accumulate in ways that are hard to argue with. And so I think that--the system changes slowly. Russ: Well, especially this one, which is rooted in cultural stagnation, inertia, whatever you want to call it. Political, regulatory stagnation and inertia.
33:21
Russ: Let's talk about the potential for that to change. What do you see as the potential for online learning to be truly disruptive? So, right now, obviously it's in its infancy. We have a number of high schools using video and virtual learning to help supplement or even replace regular classroom instruction as a way to save money or possibly to improve quality. A number of universities have farmed out their certain types of classes--calculus and other things--to the digital world's way of saving money. What do you think the potential is to really be transformative in terms of the K-12 and the university world? Guest: I'm going to start with maybe not the answer you were looking for, which is I think the first big transformation is likely to come not here in the United States. The reason is that here in the United States we have a solution--and we can argue that it could be improved. But it's hard to remove it [?] solution and replace it with something else. In large parts of the world there are these huge gaps in educational availability. And it's not like there is a satisfactory solution or even a semi-satisfactory solution. So, I'll give you an example of a case in point. India wants to increase its post-secondary completion rate from 13 to 30%. To put it in line with where OECD (Organization of Economic Co-operation and Development) countries are. In order to do that, they are going to have to build 1500 academic institutions and train a million new instructors. Now, the point is, even in their current academic institutions, even in the very top ones, there is a lack of instructors. There is not enough to cover the classes that are already there. So, where are you going to get all those new ones? And so I think, if you want to solve this problem within this generation or even the next one, you are going to have to find an alternative mechanism for delivery of education in a more scalable way. And so I think what we're going to see as that those other countries are going to leapfrog the United States in education in the same way that they've leapfrogged the United States in cellphone coverage, in electronic banking, and in a whole variety of other places that they didn't have the infrastructure that provided them with these less-than-satisfactory solutions. Russ: And they have more of a blank slate and so they took what was off the shelf and that turned out to be pretty good. Guest: And they have bigger needs than we currently have. Russ: But in the United States--we're taping this in Coursera's headquarters, which are located where? Guest: Mountain View. Russ: So we're very close to Palo Alto. I used to point out that Palo Alto often finished 99th--in the 99th percentile--in the city, in California, on high school exams, and East Palo Alto, which was separated from Palo Alto by the 101 Highway would often be in the 1st percentile. Someone had to be in the first--hard to realize that sometimes but there is a worst school system. So that's a pretty big gap. And when I would argue for reform such as vouchers or getting rid of the public school system, some people would say, Well, that would enhance the rich. And I'd say, well, it's hard to make it bigger than 99-to-1. So, what are your thoughts on that gap here in the United States and what online education might do to make that a little smaller? Guest: I think that the hope is that by providing the resources of really spectacular instructors, you can improve the outcomes for those students who are in that 1st percentile. I think it's important, however, to be watchful for overzealous politicians who are going to view this primarily as technology for decreasing costs as opposed to technology for improving outcomes. I have nothing against decreasing costs; I think we can do a better job of educating people at a lower cost today. But I think especially for that 1st percentile or that lower quartile, it's important to view this as a way of enhancing outcomes. So you shouldn't just say, Oh, we're going to take the teachers out of the classroom and just send them all to the online world. But rather, let's do that blend in learning. It's not going to decrease costs but hopefully it will allow people to perform better. As we saw in the Madison example, for instance. Russ: So in 2025, 10, 11 years from now, what do you think the landscape of high school education will look like and university education will look like? Like it looks now, with a few more online options? Or do you think it will be radically different? Guest: I think in a decade we are going to see radical differences, maybe not everywhere, maybe not every discipline. But for example I think that universities that are currently building large lecture halls, their presidents ought to think differently about how to invest their resources. Because I don't think we're going to be seeing a huge demand for large lecture teaching, except for the occasional inspirational lecture. I think we're going to see a significant use of online resources and having the classroom be much more about interactive activities as well as the development of the softer skills that are so important. Russ: Are some universities or a significant number going to be gone? Because they are not providing--can't compete with the options of a great university that has extended its reach via technology? Guest: I think that's a good question. I think that some universities will undoubtedly disappear. I would say those are fairly the ones that are unwilling to see the future and unwilling to adapt. There's some universities that are taking the so-called ostrich strategy, which is, I'm going to close my eyes really tightly and hope this all goes away. You know how well that worked for journalism. Russ: I was thinking the exact same thing--that worked great for newspapers. Guest: So, I think those will disappear. Others could find I think a value add, in terms of, for example, that blended learning experience, or the social structure that surrounds the educational experience, connecting students to alumni, to local internship opportunities, to volunteering opportunities abroad, whatever. You can imagine universities that really just focus their mission more on those other skills which we know are so critical to success in the modern world. So I think those universities will do fine.
39:59
Russ: Now, we have a lot of romance about university education in the Western world. And part of the romance is you are going off, when you go off to college, to hang out with great minds. And you find out that you are in a room with 500 other people and you don't get to hang out much, and the office hours are short, etc., etc. But I'm wondering how much people are going to want to go hang out with great coaches, right? A lot of the appeal of a first-rate university is, again somewhat misplaced sometimes--you don't always get the best teachers, and they don't teach much. But in theory you are mingling and having interaction with great minds. But you can do that online. Is there going to be any purpose to having a university, other than the few that are actually providing the content? Guest: So, first of all let me point out that that romantic ideal of those students on green grass lawns, hanging out-- Russ: I see togas--the togas are either on the faculty or the students, I'm not sure which. Guest: Let me point out that the number of students who have that experience is a tiny handful. So, I think the numbers that I heard was that 85% of students are actually in commuter colleges. Most of them are nontraditional learners who are not even in the 18-25 year old range. And so they are not getting that experience even today. So that's one part to remember. Now, I do believe that they are going to those educational institutions not necessarily because they get to hang out with the great minds--not all of those instructors are great minds--but there is a value to the sort of interaction that they get with their other classmates, the ability to go and ask questions of the professor, and even just the fact that someone is looking you in the eye and saying, Give me your homework. So, I do think that there's going to be value in that, especially at the undergraduate level. I think there's a bigger question of what we're going to see in the transformation of education at the graduate level. Because those are often mature people, often ones who already have a job. Russ: Well, we were older. I'm not sure 'mature' is the right word. But I know what you meant. Guest: Well, speak for yourself. And for them to take two years off and go get a Masters degree is very challenging. And I think we're going to see a big transformation in the graduate space with people taking education in smaller units that are more just-in-time. So, I need to learn a new skill; that might require a three-course sequence, but not a full two years' worth of Masters degree. And for that three-course sequence, I'd still rather do it in evenings and weekends and still maintain my fulltime job and my apartment and be with my family. Russ: So, let's look again into the future. Isn't it possible, if not likely, that instead of going out to get an MBA (Master of Business Administration), I'm going to take the best Finance class from this university, the best Marketing class from that university, and disaggregate the degree? Just the way I disaggregate my newspaper. I don't read one newspaper. I read my favorite parts of 25 different websites. Won't I be doing that, showing my mastery through some sort of assessment, and doing it while I'm keeping a [?] on my job, so I can keep the costs down? Guest: So, having known a good number of people who have gone through an MBA, I can tell you that the actual content that's being delivered as part of the MBA is a relatively small component of what people are getting. Russ: Yeah, I wish you wouldn't say that. Yeah, I think that's true. Guest: And I think the best MBA programs recognize that, and they build the softer skills. They build the networking opportunities, both internally in the class as well as with people who are alumni or otherwise associated with the institution. So I think people will still find significant value in those institutions, because learning is a social experience. Unlike reading a newspaper. And in the same way I think it's useful to think of a different analogy, which is that of the music industry. You could have said, you can get a song on iTunes right now for $.99; that's going to put live concerts out of business. Why would somebody drive an hour, stand in line, and then sit in this large space and pay $100 to see the singer from the 50th row? And nevertheless, live concerts are on the upswing. So I think we are going to actually see if anything an increase in enrollments, at least at the better universities.
44:33
Russ: So, in your previous life--you had a very interesting life--you were a Professor of Computer Science. And you were also in the Pathology department, and you helped develop a thing called C-Path--Computational Pathologist. And I want you to--this isn't just a change of pace here, but I want you to talk a little bit about what that does, which is rather extraordinary. Talk about that, and then we'll come back to Coursera. Guest: Okay. So, my expertise as a computer scientist was in the area of machine learning and big data. And one of the things that always drew me was applying those ideas to understanding human biology and human health. The C-Path approach was effectively a way of applying machine learning techniques to do what I called data-driven medicine. I'm going to digress for a moment to say that the first time that I heard the word, evidence-based medicine, I was actually rather shocked. Not because--I hadn't realized that there was any other kind of medicine. In fact, that this was the exception rather than the rule was rather an unpleasant surprise to me. Data driven medicine you can think of as the next step beyond evidence based, because not only are we using the evidence of a particular hypothesis that somebody tested, to assign treatment, we're actually letting the data speak for themselves. So, we're not coming in with a preconceived hypothesis. Rather, we are taking a lot of data--in this case pathology images from cancer patients--putting in a lot of different ways of looking at those images, like how many cells there are in this region and how are the nuclei shaped and whole bunch of features which may or may not be in any way relevant to cancer--we don't know--and then letting the data tell us which are the ones that really are relevant to prognosis, for example, and which are the ones that are not. And so one of the things that we uncovered in doing this in a completely unbiased way that didn't come in with preconceptions is that if you look specifically at breast cancer tumors, it turns out that characteristics--features, as I call them--that surrounds the cancer cells are actually more predictive of prognosis than the features of the cancer cells themselves. So there's something about the ecosystem of the cancer cells that is really important to how aggressive the cancer is and how quickly it develops. Russ: And was being ignored by the human pathologists who were examining the tissue and the results. Guest: Yeah. And what inspired us to look at this question was the way in which cancer is graded using pathology images hasn't really changed for about a hundred years. And the question we asked was, is that because the 3 characteristics that were being assessed by pathologists, they just happened to nail it the first time? Or are there other things that we just never thought to look at? And it turned out that the latter was the case. Russ: The other part of your work that really fascinates me, it can help answer the question that I think every patient wants to know, which is: I can take this treatment, which is really unpleasant, or I can take this treatment, which is less unpleasant; or I can do nothing. And people often say, Well, what are the odds that it's going to work? And most doctors are very uncomfortable in my experience answering that question. And then, you are tempted to say, well, what's the evidence? And the answer is: They don't have any clue. Some of that's legitimate, because treatment is person-specific and it may be very misleading to talk about an average in the case of an actual person. But that would be useful information in the case of most people. Talk about what machine learning can contribute to that question. Guest: So, to me this is an incredibly exciting direction and one of the things I regret about having come here to Coursera is that I'm no longer working on that. Because I think it's really important. We're at a tipping point in the world of medicine because of the digitization of a lot of the health information about many, many patients. And so we're now at the point where we can start saying, here's a thousand patients, or 10,000 patients, who have all received a certain treatment, and here's the things that characterize those patients, both at the imaging level, genomic data, various other characteristics like symptoms, medical histories, and so on. All of these features, not come in with any preconceptions, again, about which of those are likely to be predictive about the success of a particular treatment. Throw it all into a machine learning algorithm and see what we think the odds are of treatment x working for a patient working with these characteristics, versus treatment y. And you are absolutely right that this is very personalized; but putting in a lot of information about each particular patient, we are getting pretty close, or much closer to being able to make the predictions that are right for you. Russ: So, one of the things people worry about, of course, is that, oh, this is good for people with cancer, and I think they count a lot more, but I just want to say that it's not so good for pathologists. So, some people worry that smart machines are going to just eliminate most of the jobs that we already do. You worry about that? You think about it all, your colleagues who are in this area? Do they ever talk about it? I'm not worried about it so much, but I know people are. Guest: I think it's impossible and foolish to try to stop the progress of technology so as to save people's jobs, when machines can actually do those jobs better. What I think will happen to pathologists is they will move up the food chain, in the same way that we've seen other professions move up the food chain. I think teachers will move up the food chain. I think that the right thing to have happen is to transform professions rather than trying to keep them unchanged just because we don't want people to have to transform their jobs. Russ: Yeah, those blacksmith jobs are really--they are lost. Guest: Not to mention the people who shoed horses. Russ: Yeah. They're gone. It's a good thing. Mostly. Russ: So, what I was going to ask you, which you already kind of answered, but I'll let you talk more about it, is that, that's an incredibly exciting field, obviously--the opportunity to apply machine learning to outcomes. And I think you seem to agree we are on the cusp of a real revolution there, that I think will be really transformative. And so I'm asking: How hard is it to walk away from that? You are doing something radically different. Exciting in its own way. But talk about what it's like to be an entrepreneur now, too. You are not just a faculty member who has come up with something useful. That's unusual enough, to start with. But one who is an entrepreneur, running a company, is even more unusual. Talk about what it's like. Guest: Um, it was a wrenching decision to stop doing that and do this instead. It was not something I took lightly. But I think the thing that really drew me to Coursera and to what I am doing now is the culmination of the magnitude and the immediacy of the impact that we're having. I mean, I could have continued to work on personalized medicine. And, you know, optimistically, we would have been able to extend the work that we did to other types of treatment; and then we would have been able to maybe license it to a large pharmaceutical company or something. And after additional FDA (Federal Drug Administration) trials, and so on and so forth, maybe 5-8 years, we would have been able to be in a position to affect where the technology has affected patient outcomes. Patient treatment. Russ: But only three years, you could have a journal article. It might take 5 or 8 years to get that FDA thing, but in 3 years, you get the excitement of a publication. Guest: Exactly. Russ: That's still three years. Guest: Exactly. And here, we have touched the lives of millions of learners in a matter of a year or two. And that number is only increasing more and more rapidly. So, it was just--unfortunately life is about choices and tradeoffs-- Russ: It's economics. Guest: I'm in a field, to be in a fortunate position where I had two such directions to pursue. And making a decision is just hard. And I made the decision that I felt would give me the biggest opportunity to make an impact in the world. When we were starting Coursera back in, or making the decision to start Coursera, back in the late fall of 2011, was when Steve Jobs passed away. And one of his favorite quotes that was repeated often at that time, is that, your goal in life should be to make a dent in the universe. And this is my dent. Russ: Yeah. That's really beautiful.
53:47
Russ: So, there's a temptation to think, as an outsider, outside it kind of runs itself. There's a platform, and there are all these teams around the country helping professors prepare their lectures and get them up online. And to some extent Coursera is, like many of these extraordinary websites and efforts that are going on now, they are matchmakers. You are taking great teachers--just like Uber takes people who have some extra time and a car that's not being used and find someone who is lonely and on a street corner. AirBnB are taking people who want to stay somewhere nice and there aren't any good rooms left. You are taking teachers at who are great at what they do and you are finding hungry people to be fed with their knowledge. It's a beautiful thing. Guest: It's called a two-sided market. Russ: Right. So, what to you do all day? As the president. Besides applaud and watch the numbers grow. What kind of--you can't talk about some of your strategic ideas, but what's happening here in this building? It's a pretty large building. I don't know how big it is; I don't know how many people work here. What is Coursera itself doing other than creating this format that's already out there now for people to match with each other? Guest: So, there's a number of things that we do as an organization. Maybe the most obviously visible one is the building of the platform that makes this all possible. It's not an easy task to construct such an artifact. [more to come, 55:16]