Noah Smith of Stony Brook University and writer at Bloomberg View talks with EconTalk host Russ Roberts about whether economics is a science in some sense of that word. How reliable are experiments in economics? What about the statistical analysis that underlies much of the empirical work in modern economics? Additional topics include the reliability of the empirical analysis of the minimum wage, the state of macroeconomics, and the role of prejudice or prior beliefs in the interpretation of data and evidence.
Play
Time: 1:08:34
How do I listen to a podcast?
Download
Size:31.5 MB
Right-click or Option-click, and select "Save Link/Target As MP3.
Readings and Links related to this podcast episode
Related Readings
HIDE READINGS
About this week's guest:
Noah Smith's Home page
Noahpinion. Noah Smith's blog.
Noah Smith on Twitter.
Noah Smith archive at Bloomberg View.
About ideas and people mentioned in this podcast episode:
Articles:
"Check Your Economics Bias," by Noah Smith. Bloomberg View, October 30, 2015.
"In Search of the Science in Economics," by Noah Smith. Bloomberg View, October 14, 2015.
"The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con Out of Econometrics," by Joshua Angrist and Jörn-Steffen Pischke. National Bureau of Economic Research, 2010.
"Comment on Angrist and Pischke," by Christopher Sims. Journal of Economic Perspectives, Jan. 4, 2010. Princeton U., PDF file.
"Tantalus on the Road to Asymptopia," by Ed Leamer. Journal of Economic Perspectives, Spring, 2010. PDF file. Response to Angrist and Pischke.
"Does Monetary Policy Matter?: A New Test in the Spirit of Friedman and Schwartz," by Christina D. Romer and David H. Romer. 1989, NBER reprint from Macroeconomics Annual, vol. 4, 121-170. PDF file.
"The Effect of School Choice on Participants: Evidence from Randomized Lotteries," by Julie Berry Cullen, Brian A. Jacob, and Steven Levitt. Econometrica, Sep. 2006. Uchicago.edu. PDF file.
"How I Became a Keynesian: Second Thoughts in the Middle of a Crisis," by Richard A. Posner. The New Republic, September 23, 2009.
Some discussions of the identification problem:
Is theory getting lost in the "identification revolution"?, by John Huber. The Monkey Cage blog, June 14, 2013. Basic discussion of the identification problem and causality.
Why is supply and demand so confusing?, by Scott Sumner. The Money Illusion blog, July 28, 2009. Basic discussion of the identification problem and causality.
"Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments", by Joshua D. Angrist and Alan B. Krueger. Journal of Economic Perspectives, Vol. 15, No. 4, Fall 2001. MIT. PDF file.
Some discussions of the Civil War natural experiment in monetary policy:
"Prices, Income, and Monetary Changes in Three Wartime Periods," by Milton Friedman. American Economic Review 42 no. 2 (1952): 157-197. JStor.
"Money and Finance in the Confederate States of America", by Marc Weidenmier, EH.net.
Some discussions of the post-WWII 8-month GDP dip--a recession or some other kind of temporary economic adjustment?
Keynes vs. Reality-2 by Russ Roberts. Cafe Hayek, July 14, 2011.
Post-war "austerity", by Russ Roberts. Cafe Hayek, February 6, 2012.
"The History of U.S. Recessions and Banking Crises," by Geoffrey H. Moore. NBER reprint from Measuring Recessions. PDF file. NBER notes the different character of the 1945 post-WWII recession and excludes it from some statistical studies of recessions. Includes tables.
"The History of U.S. Recessions and Banking Crises," by Jeffrey Rogers Hummel. Alt-M, October 22, 2015.
Some discussions of minimum wage changes:
"Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," by David Card and Alan B. Krueger. American Economic Review, Vol. 84, No. 4, pp. 772-793. PDF file, Berkeley.
"Large Increases in the Minimum Wage Are Likely to Destroy Jobs," by Robert P. Murphy. Library of Economics and Liberty, October 5, 2015.
Some discussions of Neo-Fisherianism:
The Neo-Fisherian Question, by John Cochrane. Blog post, November 6, 2014.
Neo-Fisherians: Unite and Throw off MV=PY and Your Phillips Curves! , by Stephen Williamson. New Monetarist Economics, November 11, 2014.
Intuitive Neo-Fisherism, by Stephen Williamson. New Monetarist Economics, September 1, 2015.
Some basic definitions and biographies from the CEE:
Keynesian Economics, by Alan S. Blinder. Concise Encyclopedia of Economics.
Monetarism, by Bennett T. McCallum. Concise Encyclopedia of Economics.
Minimum Wages, by Linda Gorman. Concise Encyclopedia of Economics.
David Hume. Biography. Concise Encyclopedia of Economics.
Friedrich Hayek. Biography. Concise Encyclopedia of Economics.
Milton Friedman. Biography. Concise Encyclopedia of Economics.
Paul Samuelson. Biography. Concise Encyclopedia of Economics.
Irving Fisher. Biography. Concise Encyclopedia of Economics.
Web Pages and Resources:
Confessions of a Supply-Side Liberal. Miles Kimball's blog.
The Grue Paradox. Henry Folse, Loyola U., New Orleans. Green or Grue?
Podcast Episodes, Videos:
Joshua Angrist on Econometrics and Causation. EconTalk. December 2014.
Alex Rosenberg on the Nature of Economics. EconTalk. September 2011.
Ed Leamer podcasts. EconTalk.
Alex Rosenberg on the Nature of Economics. EconTalk. September 2011.
The Economics of Parenting, with guest Don Cox. First EconTalk episode. March 16, 2006.
Bryan Caplan on College, Signaling and Human Capital. EconTalk. April 2014.
William MacAskill on Effective Altruism and Doing Good Better. EconTalk. September 2009. Discussion of deworming.
Brian Nosek on the Reproducibility Project. EconTalk. November 2011.
Highlights
Time
Podcast Episode Highlights
HIDE HIGHLIGHTS
0:33
Intro. [Recording date: December 1, 2015.] Russ: We're going to talk today about a number of issues related to economic philosophy and methodology you've written on, and that you and I have disagreed on at Twitter, and a number of people have suggested we should have an EconTalk episode on. So this should be fun. I want to start with the role of ideology in economics. Do you think it plays a role in economic policy debates among academic economists? And do you think it should? Guest: I mean, sure it does. You know, we don't always know exactly what policies work; we don't always know which of the evidence is right. Things like that. And so you're going to have to rely on priors a lot. And people a lot of times get their priors from what they want to be true--wishful thinking. Which is really just, I think, what ideology is. And so obviously it's out there, it's playing a role. Bias exists. And I'm not going to try to claim it doesn't. It's obviously a thing[?]. And I think in the media it's more than in the halls of academia. It's still there in academia, but in the media you get more of it, because the people who really are strongly ideological, who really want to push a point of view are more likely to go to the media than people who just want to, you know, assess the facts as neutrally as they can. Those people are probably a little less naturally drawn to the spotlight. Russ: And of course they get rewarded--I think if you are more ideological you are more likely to be chosen to get that spotlight than if you are kind of a mere truth-seeker. Guest: Absolutely. Absolutely. There's politicians who have an agenda; there's very ideological people out there. And then there's just normal people who have confirmation bias, and they hear something that's sort of--an economics media person saying something that supports what they want to be true, and they want to trust that person or believe them more than someone who says the opposite. So you've got huge incentives for ideological people to get the spotlight. Russ: And my listeners are pretty aware what my biases are. I'm going to lay them out and I'm going to invite you to do the same, just as a prelude to the rest of the conversation. I am a free marketer; I call myself a classical liberal, which means I believe in limited government, personal responsibility. I'm not an anarchist; I think government has a role to play but it's much, much smaller than the role that the government plays in the United States in our economy or in our daily lives. I have another ideological bias though, as I prepared for this conversation that I want to confess. I don't think it will be a surprise to listeners, but I want to get it on the table for you to think about; and you can respond to it with your own position on these issues. I realize I'm very prone to believe that people respond to incentives. I'm very--so this--this is not a philosophical ideology or a political ideology. It's a methodological ideology. I'm prone to believe that incentives are important, that people respond to them; and in particular nonmonetary incentives, not just money. And along with that, I have a strong belief in the power of emergent order--that a lot of problems get solved by individuals either individually, working voluntarily together or sometimes not in any planned way, but that market forces or incentives push them to solve those problems. Of course, I'm aware that there are many problems where that doesn't work very well. Environmental issues being an obvious example. But I have a strong bias, or ideology, or methodological belief that incentives are very powerful. And of course they can be counterproductive for achieving certain ends if they are set poorly. So, that's my take. And I just would also add, before turning it over to you, that when I was younger, meaning--I'm 61--when I was, say, in my 30s or 40s, nothing made me madder than being identified as a conservative economist. First of all, I'm not very conservative. I'm much more of a free market or a libertarian or classical liberal, whatever you want to call it. But second of all, I hated the idea that somehow I could be labeled--that my views would be, then, well-known and that I was just some rigid ideological. But I've come to embrace the reality that I am, to some extent, an ideologue: I have priors about the way the world works. They can come from a lot of places. They can come from empirical evidence, of course. But I think a lot of it is complicated and not easily justified to an objective observer. And I think it's important to admit that, so when people who want to trust me or not trust me, they should know just how much salt, how many grains of salt they should take my positions with.
5:46
Russ: Okay. So, your turn. How would you describe yourself? Guest: All right. Well, first of all before I do that, I don't really think that thing about incentives mattering and emergent order and things like that, I wouldn't even call that an ideology, myself. I mean, you know--I have strong priors that differential equations can accurately predict the motion of heavenly bodies, but I wouldn't call that an ideology. That's just sort of an assumption, a prior, about what kind of, you know, scientific things really predict the world and explain the world. Russ: Well, I'm going to come back to that. I'll come back to just why I think it's important: even though I like to think what you're saying is true, I'm not as convinced. But go ahead; I take your point. Guest: Because I think that incentives matter a lot, too. You know, I mean, that's pretty obvious to me. And lots of things seem emergent, at least we might as well describe them that way sometimes. And in terms of ideology: I think of myself as a pragmatist. I don't have a very self-consistent, well-defined ideology to myself. I can't really--I think of myself as a Humean [David Hume]. I have these moral instincts--I think, 'Oh, well, you know, this is too much a restriction on individual liberty' and 'I don't like that.' And then in some other case I'll think, 'Oh, look at these poor people,' you know, 'We've got to have the government do something to help those poor people.' And there's not necessarily a rhyme or reason to that. It's just sometimes I have one instinct and sometimes I have the other. And I think of those moral instincts as sort of the primitive, primal, basic elements of morality, and they don't always work--they don't always form a self-consistent whole. Now, you know, when I look at what really works for nations, what I think looks good for nations, you know I think the rich countries of the world are all doing pretty well: the United States is a great place to live; Canada, Japan, France, Germany, Singapore--I don't really know as much about Singapore. But these are all really good places to live. Every place I have seen is good. And there are some small taste differences. I mean, life in France is not going to be quite like life in Japan. But they are all doing pretty dang good. And, you know, I mean, maybe a radical overhaul where we dramatically decrease the size of government would be even better. And maybe some country will try that. You know, I'm for experimentation; maybe someone will try that. But so far all the countries we see with these great qualities of life, these very successful countries, all have big government in them. And to me, that says that we should at least be pretty cautious before we toss out big government completely. These [?] radical experiments. Russ: And you view that as a pragmatic conclusion. Guest: Yeah. Pragmatic conclusion. I mean, you know: Be careful. Cross the river by feeling for the stones. Go with what works and tweak it a little bit in the direction you like, and don't just launch a radical experiment. I mean, that's what got us in trouble with communism--these people came up with a theory about how everything can be so much better; and it was just a calamity. And it was the radicalness of it and the huge break and radical departure from what's worked in the past. Maybe that makes me a Burkean. I haven't read Burke [Edmund Burke], but-- Russ: Scary thought. Guest: [?] the other day. Yeah. Russ: [?] It might. Guest: I don't know. I mean, I think there's so much we don't know, that we'd better be careful. You know? We only have knowledge about the little bubble of the world close to us, the little epsilon ball right around us. Huge experiments are more like to fail than succeed, so maybe some small countries who do them first, or maybe some country in an extreme situation can be pushed to do that experiment. Like medicine. We don't just go doing crazy medical experiments on human beings because most of the time you are just going to kill someone. Once in a while you'll find some cure for something, but most of the time it's going to be horrible. But I think of it in a similar way.
9:58
Russ: Okay, so let me--this is sort of off topic in a way, because now we've gotten into the question of: What's the appropriate role for government, especially when we are thinking about small changes versus large changes? But it's really interesting. So I want to comment on that. So, I think people on my side of the fence, the free market types as well as conservatives, I think fail to notice what you've pointed out, which I think is really important. Which is: Life is pretty good in America, say, where you hear people complaining that it's all going to hell in a hand basket, and how horrible the country is and we have to change things because it's awful. And I agree with you that it's basically pretty good; and in particular, I'd agree with you that it's really good for me. And it's really good for my kids. So, my kids who grew up in a house of educated people with lots of books and limited access to television and devices until they got older and their dad and mom got worn down a little bit--my kids have a pretty good shot at economic success--true happiness, moral success: those are different questions. But in terms of just thriving materially, I think my kids will do well even in an economy where government is 40% of GDP [Gross Domestic Product] measured across Federal, state, and local. Which is not much different from France or the United Kingdom or Germany or the places you also mentioned. So I think that's true. And I think that people who bemoan the size of government often ignore that very uncomfortable fact, despite what you'd think-- Guest: Right. Russ: If you think government is part of the problem and not part of the solution, you do have to deal with the fact that it's going pretty well. So, my response to that--I agree with that part; I think you are right. I think that should give pause to people who want smaller government. There are two things, though, I think that are still worrisome. One, I would call the 'boiling the frog' problem--that, it's true that it's going pretty well right now, but something akin to Hayek's Road to Serfdom, which I think has been misunderstood as saying 'We're going to be communists because we have big government,' or 'We're going to be fascist'--that isn't what he said. He said--it was a warning bell. And I think that people like myself who are thoughtful about--who try to be thoughtful, at least--about the size of government and the fact that the economy is going pretty well, I think there's a worry that it's going well now, but if government continues to get bigger there will come a point where it's not as sustainable. And when we look at countries on your list, such as Western Europe or Japan, we do see some very disturbing signs, in the labor market in particular for young people who are struggling to get access to this engine of prosperity that you and I are doing okay on. Or our kids might do okay on. The other point I'd mention is that, while it's okay for my kids, there are a lot of people's kids I don't think are doing very well. They are handicapped by a really lousy school system. They are handicapped--and we'll come back to this; I want to talk about it in more detail because you've written about it--by a minimum wage that I think makes it harder for them to get experience. Doesn't help my kids--my kids' market values are already above the minimum wage: as 18- and 19-year olds they know Excel--one of my kids knows Excel so he got a job with a great set of learning and opportunities because he was good at math. And so for my kids it works fine, because they're not fighting at the $9-dollar-, $7-dollar-an hour slot. But for the people whose skills are mediocre, they are handicapped by the fact that they don't have good skills; they are handicapped by the fact that their labor market opportunities are cut off by, I believe, a minimum wage that makes them too expensive to hire relative to their skills. So, when I decry the size of government, I am worried about people who are less skilled, who have fewer chances, fewer connections. And their life is not so good. And I worry about how that's going to change over the next 10, 20, and 30 years if government continues to make some bad policy mistakes. What are your thoughts on that response? Guest: Well, first I am pleasantly surprised that the more libertarian person was the first to bring up inequality. That's kind of cool. But anyway, I completely agree. I think a lot of people in America are not doing well, or at least not as well as they'd like. You know, they have a better standard of living than Americans 200 years ago, but that may not be enough to make them happy. It may not resonate. They may have had higher expectations. And they have problems that certainly didn't exist 200 years ago. You know? Like heroin, or something like that. Family breakdown. And so I completely agree and I hear you on this. And I think that, you know, I think about this. I'm not a huge fan of minimum wage. I think that something like wage subsidies, which Jim Pethokoukis of AEI (American Enterprise Institute) has been pushing, and I've been pushing for--it's similar to the EITC (Earned Income Tax Credit)-- Russ: That's the Earned Income Tax Credit. Guest: That's right. That, you know, this third way compromise. I think Milton Friedman was a big advocate of negative income tax, and you know, Bill Clinton pushed it as well. I think that's the sort of best short-term measure for these people. In the long term there's lots of institutional tweaks I think we need to look at. I think there's things we don't understand yet about how, you know how things like corporate ownership and globalization and these big changes may have changed the job environment for people and may have resulted in inequality, may have made some rigidities in our system that were previously tolerable now more severe. And so I think that it's a complex, difficult problem that we are dealing with and I don't have simple, easy answers. My own simple, easy answer is kind of the EITC wage subsidy increase; and I think it's mainly a stopgap. Russ: So, let's go back to the ideology. You described yourself as a pragmatist. What does that mean to you? The way I would take it is you go on a case-by-case basis. You look at what works here, what works there. Is that an accurate description? Guest: Yeah. That's right. Case by case. Russ: So, I think that's unusual, but I don't think it's rare. Right? I think there are economists who are not heavily ideological, who would describe themselves even as scientists--as I think one EconTalk guest said: 'I do have an ideology but I take that hat off when I do science.' And some people may be--I don't think that's possible, but [?] that's the problem with that. And there may be people, and you may be one of them, who basically just look at what's effective.
16:53
Russ: So, let's turn to the question of science. Because you and I have written some, had some fun exchanges back and forth on Twitter. So, my claim is that economics isn't much of a science in the way that people in everyday English think about it. I'm not going to get into a heavy philosophical debate about what a science is. But what I mean when I say economics is a science: It doesn't produce reliable empirical relationships that can be used as the basis for economic policy. So, do you agree with that? And if not, why not? Guest: Well, when you talk about science it's really a question--and I am going to get into the philosophy here, briefly--it's a question of how do you know how the world works. In lab sciences you do experiments, and you control a bunch of stuff with experiments. And usually in economics we can't do that. A few times we can, and then we get really, really good results. But usually you cannot. And of course you can just look at correlations and you can say, 'Look, you know, people who got this kind of job ended up doing better in the economy in this, or any millions of billions, or this kind of policy happened and then also the economy did worse at that time so maybe that policy is bad.' And then between those you have this approach of natural experiments, also called quasi-experiments. And this has become much more popular recently. You can often exploit works of how policies happen, of the real-world policy-making process, to get clean identification. Right? And there is stuff that you can't control in a natural experiment that you can control in a lab. And most importantly, the most important difference is that with natural experiments, you can't replicate them. You can't repeat them, do them again. Because each natural experiment only happens once. So it's not as good as lab science. But, it is better than simply looking at correlations. And this is just exploding in the economics profession, this way of evaluating policies. And a lot of people have complained about it: some of the advocates for this kind of approach have been a little, you know, zealous and messianic in their advocacy. But I think most people agree at this point that what they call the 'credibility revolution,' which is really just these natural experiments, has really been a huge boon to our ability to understand the effects of policy. Russ: Let's take an example of that, so that listeners have a better idea what we're talking about. And we had Joshua Angrist on the program a few months back, and he's one of the--I'd say he's one of the two or three most outspoken advocates in favor of this, and a champion of what you've called the credibility revolution. So, just to clarify things: I was trained a lot earlier than you were--and I don't mean in the morning. I got my Ph.D.--I didn't get up--I tried to schedule afternoon classes, as many of us did. Guest: I went to Macro at like 7:30 in the morning. I'm still mad at Michigan about that. Russ: Well, you probably paid a terrible price for it. Your macro probably isn't as good as it could have been if you'd taken it at a civilized hour. Guest: Certainly not. Russ: But, I was trained in the late 1970s, and my econometrics professor was Arnold Zellner. And I was taught all the standard techniques of the day. And that was a long time ago. There have been some improvements. But I wonder, and of course there are many skeptics on this, how effective those improvements can be, have been. So, when we say a 'natural experiment'--I'm going to give two examples. Okay? And then you can tell me if I'm on the right track. So, one would be one that was exploited by Milton Friedman and Anna Schwartz in their book A Monetary History of the United States. So, looking at inflation during the Civil War, and there comes a point in the Civil War where there's high inflation in the South, in the Confederacy. And then the printing press in the Southern States, the Confederacy, gets overrun by the Union Army. So they can't print money any more. And inflation stops. So that's one kind of natural experiment: a dramatic change in the policy regime that allows you to be pretty sure--can't be 100%--that what follows on that is caused by that. There's still uncertainty about that, of course. The second would be, a state increases its minimum wage, but neighboring states don't. And I might have towns that are near the border of the states that change its wage; and I can look at see how the labor markets in those towns did compared to towns that were nearby. So, we might think of them as being in the same labor market or having roughly similar skills; and yet only on one side of the border is there a policy change. Are those good examples? Guest: Those are probably good examples. Depending on how they are done. Right. That's right. So, you have your diff-in-diff [difference-in-differences] approach; you have your regression discontinuity design. And these are some of the standard methods for dealing with natural experiments. Do you know, I've never actually read the Monetary History and didn't know that example about the Civil War? But that's really interesting. That's kind of cool. Russ: I think it's true. [Original source: Milton Friedman, "Prices, Income, and Monetary Changes in Three Wartime Periods," 1952, American Economic Review. Referenced in Friedman and Schwartz, A Monetary History of the United States, p. 59-60, n. 64.--Econlib Ed.] I read it a long time ago, and I have to confess even though I went to the U. of Chicago, I have not read the entire book. So I may be remembering that incorrectly. Or I'm probably remembering it second-hand. Go ahead. Guest: Right. Well, so, that's exactly that kind of thing. Natural experiments are very hard to find in macroeconomics. They are extremely hard to find. Chris Sims wrote a reply to Angrist where he gave reasons why one of the most popular things that people think is a natural experiment, via the Romer-Romer paper about the Fed and monetary policy changes--why that might not actually be a natural experiment. I think Sims is very convincing. Basically, with macro, you don't have a lot of these boundaries where the policy is working in some places and not working in other places. You just have this--because it's macro, it's kind of global. And that can even be true when you look between countries, there's a lot of linkages between countries. So it's just extremely hard to find these clean, natural experiments in macro. And that's why I think macro is always going to be more of the domain of priors than, you know, what we might call micro. In fact, that's almost the definition of macro versus micro. Macro is almost defined by the places where we have worse data. And that's why I think you see ideology start to play a larger role, because it's one of the things that determine people's priors. You start to see it play a larger role in debates. Certainly in the public sphere. Because you can't just point to some of these clear natural experiments and say, 'Well, look, we know this because this. Here's the evidence; very clean identification here.' Several different natural experiments all showing the same thing, so we know what works and what doesn't. You can basically not do that in macro. Very, very rarely. And so that's why I think we see this kind of jadedness in macro debates--simply because the evidence is worse.
24:50
Russ: Well, let me push back on that a bit. And when I disagreed or challenged Angrist, I used that Sims article, and Ed Leamer also wrote a similar critique of some of the combinants[arguments?] that the credibility revolution claims. Maybe it's misplaced. Guest: Can I interject a second here? Russ: Yeah. Guest: Leamer's paper is showing why natural experiments are not as good as lab experiments. Which we know. We definitely know that. And he's right about that. And that doesn't mean they are no good. It just means that they are not as good as lab experiments and never really will be. But anyway, okay. Russ: I think he's less--he's much less friendly than that, I think, to the Angrist claims. His claim is basically: You think you've done something to insulate yourself from the problems that he identified in his "Take the Con Out of Econometrics" paper. But he basically concludes by saying, 'Not really.' Well, we'll post those articles and people can take a look at them. Much of them are readable to a non-technical reader. You can enjoy the fireworks I think, especially, no matter what. Guest: Right. And those are some great papers. Those are two of the best philosophy of science papers I think ever in economics. Those two papers: the credibility revolution paper and Leamer's critique of that are super-good reading for anyone-- Russ: Yeah. I agree. They are really interesting. But the point I want to challenge you on is this issue that you can't get a clean experiment. So, it's true you can't get as clean an experiment across space, but you still can get in theory a clean experiment across time. Which is why I think the printing press example is a good one. The one I like to point to, being a non-Keynesian, is the conclusion of WWII. A lot of Keynesian economists, and most prominent being Paul Samuelson, said the end of WWII would lead to a devastating depression if the United States let military spending fall dramatically, as it normally would at the end of a war. It had reached a very high level, you know, an historically high level; and his worry was that the end of the war would lead to a disastrous economy. He implied it would be worse than the Great Depression. He was writing this in 1943. The war ended in 1945. There was no attempt to cushion the blow; nobody listened to the Keynesians, or if they did they couldn't get their policies implemented. And as a result, the end of WWII brought a massive decrease in government spending, an enormous increase in the number of people suddenly looking for work. And there was basically--nothing happened. There was no real dramatic anything. Guest: That's not quite true. There was a very deep, very sharp recession right after the end of WWII. Russ: It was a recession that was not called a--I would say it differently. There was a sharp drop in measured GDP that the NBER (National Bureau of Economic Research) did not call a recession. There was no sharp increase in unemployment. There was a recession I think in 1947 or 1949 but nothing of the magnitude that had been predicted and certainly not even clearly related to the change in military spending. The change in measured GDP--you have all kinds of problems. You've got changes in the removal, I think, at that point of wage and price controls. All kinds of other things going on. I don't remember what the reasons were that the NBER didn't classify that drop as a recession. And maybe I'm wrong about that. We can certainly look it up in more detail and fight over that if we had the charts in front of us. But just sticking with the theoretical question now: that was a natural experiment. It didn't shake, interestingly, very much of the Keynesian religion that people held. And I went back for fun and looked at the American Economic Review and the Journal of Political Economy, two of the more prominent economic journals of the day, and what people wrote at the time. And it's amazing how easily they shrugged that off, if they were on the Keynesian side. And similarly--I'll pick on my side now--the largest increase in the balance sheet of the Fed--my side being I'm something of a monetarist, trained in Chicago--the balance sheet of the Fed went from $800 billion to $4 trillion and every economist on "my side" said that's going to cause massive inflation. And nothing happened. Now, they would say, 'It just hasn't happened yet.' And that may be true. I'm willing to cut them some slack. But I think they should also consider the possibility that their understanding of the relationship between the Fed's balance sheet, velocity, and the level of prices might be imperfect. Guest: Right. And I think--you know, these are good. And this is how knowledge in macro really does progress, to the extent that it does. You have some theory where some people make some pretty bold claims. And I think monetarism makes some pretty bold claims; and Keynesianism makes some really bold claims. And a lot of times you have some situations where the bold claims make some strong predictions, and they don't pan out. In fact, I can't really think of any theories that made these bold claims that ended up not having such cases where they didn't pan out. They all have big problems. And if you try to fit these things to past data and you try to do some rigorous statistical tests, they would be rejected. Even if you let your theory be the null hypothesis--which is cheating--you still can't get these things. They'll all be rejected. So, macroeconomists have retreated mostly to using these things as thought experiments, or the sort of weasely phrase that people will use is, 'Ah--they organize our thinking.' They are a map between the assumptions and conclusions. Russ: But I'm a big fan of that phrase. And I think that's where economics has most of its power. But do you really think it's true that most macroeconomists concede--let's take the more prominent Keynesians and monetarists and anti-Keynesians and anti-monetarists. Do you think they'd say, 'Well, you really don't have much evidence for abuse[?]; it's really too complicated to try to tease out any of these effects.' So we're just kind of stuck with--what? I don't know. Guest: Well, there's always going to be true believers. Are there always going to be true believers? I think that I have observed that there are often true believers. My Ph.D. adviser, Miles Kimball, really believed that if you just take interest rates negative enough then you'll get your big output boost. And I think that so far we haven't really seen much evidence in favor of that: interest rates are going somewhat negative in Europe right now, and there's no obvious boost from it. But, it could just be-- Russ: I don't know his work, but you know what he'd say. And I know what he'd say. Guest: More? Russ: 'They just didn't go negative enough.' Guest: Just do more? Right. And I can't tell him that he's wrong. I'm highly skeptical that he's right. And of course Miles is an extremely good guy. He's a true believer in this kind of theory without being ideological about it at all. So, actually that's what's interesting. Miles is sort of my counterexample, one of my counterexamples; the most extreme counterexample to this idea that everyone is really ideological when you get right down to it. Miles is almost pathologically non-ideological in his ability to separate this sort of desires of how the world should work with his assessment of how the world really does work. He's really good at that, better than most. And I think that he would be a great guest for EconTalk in the future. You can sort of see what I'm talking about. Of course, that degree of personal objectivity, if you want to call it that, is rare. I haven't seen that. He's an extreme example; but I think he's a big counterexample to this idea that everyone in the end, you know, or maybe just in macro, really just comes down to their ideology. You can have strong priors: you can be a true believer in a theory without have a political ideology like small government, big government, you know, help this group of people, help that group of people, etc. You can still have very strong belief that this theory will work. Russ: Absolutely. Yeah. Although, having said that, there aren't--well, I'm not going to go down that route. I was going to talk about interface between macroeconomic ideology and philosophical ideology. I think that's--let's stay away from that swamp.
34:02
Russ: But, I want to go back to our original question. So, I don't think economics is much of a science. You seem to think it is something of a science. But you've now said that in half of the field--some would say 80%, some would say 10%, but I'll say half, just to make it simple--which is macroeconomics, it's really--there's not that many natural experiments that allow us to tease out causal relationships and design policy accordingly. So that leaves us with microeconomics. So, what are the natural experiments? I want to pick on the minimum wage, partly because it's such a hot button topic and also there's a huge amount of empirical work on there. Do you think we have learned much that is reliable about the impact of a minimum wage increase on employment? Guest: I think we have--I think that we see that very rarely are the--if we take the size of minimum wage increases that are usually done, they very rarely have a perceptible significant, economically significant, impact immediately on employment levels. So, what employment effect they do have is probably over the longer term, in terms of things like employment growth. And people can argue about that. But we see that--and of course if you hike the minimum wage to a billion dollars per hour, you would obviously get huge, distorted--the above-ground economy would immediately collapse and everything would become black market. But, for the types of increases that we tend to see, very rarely is there a large, big sudden drop in employment. I think that's been a pretty reliable conclusion. That doesn't minimum wage is good. And it doesn't mean minimum wage doesn't have long-term employment effects. And it doesn't mean that very big minimum wage increases wouldn't have an effect. And it doesn't mean any of those things. But it is something. You know. It is interesting and it is something. Russ: Well, until the Krueger and Card paper of 1994 that suggested that the minimum wage would have little or no effect and I think possibly even a positive effect on employment, there had been dozens and dozens, maybe a hundred studies that, in the minds of the people who wrote those studies and in the minds of the people who believed them, very powerfully showed that the minimum wage is bad for poor people; that it reduces employment by a non-trivial amount. And there's disemployment as a result: people lose their jobs or will not get jobs. And then there was a change. That paper came along. And it launched a whole literature saying, 'Well, maybe it's not so bad.' So, as a Martian without--just a truth-seeker, 'I have no ideology; I just want to know the truth'--which of those literatures do I accept? And of course there have been many critiques of the modern literature that finds little or no effect. How do I know which one's right? Guest: I mean, you have to look at the methodology and you have to look at the identification strategies. That's what you really have to do. And, you know, better identification is better. How do we tell which is better identification? Well, we've got to make a guess. But, you notice that's always true in all of natural science as well. If you are going to believe the results of an experiment, you've always got to make the leap of faith that all the reasonable stuff has been controlled for. Right? That the experimenter got good controls. And that's an assumption and a leap of faith you have to make in every natural science experiment. And it's also something you have to look at in natural-experiment literature. You just have to say, 'Look, this is good identification and this is bad identification.' If you go to seminars, that's mostly what people are doing. That's what they are talking about. Russ: Could you explain that for the non-technical--or try? It's a tough question, not an easy thing to explain. When you say 'good identification' or 'bad identification,' what do you mean? Guest: I can explain a super-super-extreme version of that. Right? So, suppose we roll a ball down a ramp like Galileo did, and we see that balls of different weights will still fall at the same rate--will roll down these ramps at the same rate. Okay? We find that out and we conclude there is a law of the universe: Balls always roll down ramps at the same rate regardless of their weight. And let's say that that is only true in a certain region of the Milky Way Galaxy, and tomorrow, our planet, which is always whirling around the center of the galaxy, leaves that region and goes into another reason where the laws are somewhat different, and balls actually roll down ramps at slightly different rates. And then suddenly we re-do the experiment tomorrow and everything is completely different. Now, to believe that the first experiment, you know, really told you a law of the universe, you've got to believe that there is such a thing as a law of the universe. And you've got to believe that there was no other funky thing going on in the cosmos that was making your experiment come out one way that might make it come out a different way tomorrow. And that's really the leap of faith you have to make. Now, that's the fun, goofy example. In these economic experiments what you are really talking about are basically exogeneity. You are talking about: Is the experiment I did basically just a purely random variation or was there some omitted variable that's causing the effect that we see? Russ: Or is my friend Don Cox, who was the first guest back on EconTalk back in 2006, said: 'We're talking about the dreaded third thing.' That there's this thing you didn't observe--you think there's some correlation between these two that's causal, but there's a third thing that you either didn't have data on or didn't notice or didn't think about that's really driving things. Guest: Yes. That's right. That's exactly right. Russ: But there's always a dreaded third thing, isn't there? Guest: There's always a--it just depends on how much dread you have. You know? In rolling balls down ramps, there still is a dreaded third thing. And I realize this--do you know the emeralds or Green or Grue paradox? Do you know that one? Russ: No. Guest: The philosophy thing they ask first year philosophy kids and undergrads. It's the idea that emeralds could be green, or they could be another color called 'grue,' which is green until January 1st 2025 and then blue afterwards. No experiment we do today will be able to distinguish green from blue emeralds. But it might make a difference, because come Jan. 1, 2025, all of your emeralds might suddenly turn blue. If you want to be afraid of that, you can be afraid of that. You can never rule it out. There is always the dreaded third thing if you have enough dread. So it really comes down to a bunch of sort of grumpy economists in a seminar room yelling, 'I don't think that's clean identification because what about this and this?' and trying their darnedest to think about what that dreaded third thing might be. How plausible those things might be.
41:44
Russ: So, I think, if you ask me to gamble on whether my emerald is going to turn blue on January 1st 2025, I'd ask you: What's the underlying causal relationship between the data on the calendar and the color of the emerald? And you'd say, 'Well, I don't know what it is, but there could be one.' And we're just unaware of it. We don't have a full understanding of color, etc. But we have a pretty good understanding of color. I think that's part of the reason that I--I think it's hard for these empirical techniques to reliably establish causal relationships with any precision about the magnitudes that are involved. So it's interesting. I would suspect that every economist who is in favor of increasing the minimum wage would concede the possibility and even the likelihood that there would be some employment loss in response to that. They just think it's perhaps smaller than I think it would be. Do you think that's a fair summation? Guest: That's absolutely fair; and by the way, as an aside, I'm not really very big on the minimum wage. I don't think it's that great of a policy at all. I'm not really a fan. I think national minimum wages are an especially bad idea. Local minimum wages certainly not nearly as bad. But I think that there's just lots better stuff than the minimum wage out there. And, so that's what you're talking to: I'm talking about this empirical literature, but at the same time it's not really a policy that I'd do if I were in charge. Russ: So, let me ask the question a different way. Guest: Yeah. Russ: The way I sometimes ask it. Can you think of a study that was so decisively performed in terms of the crossing of t's and dotting of i's that the identification and all the econometric challenges were met with such impressiveness that people on the other side of the debate had to throw up their hands and say, 'Well, I guess I was wrong. I've got to change my view.' Because I can't think of one. I can't think of one. And if that's true, then I would suggest that economics has some serious problems in claiming it's a science. And I would say, especially--and you can rule me out here if you want--but especially about macro. We just had an $800 billion dollar stimulus package that--forget 1945 or the end of WWII--we just had a very nice natural experiment. And we can't get people to agree on whether it was successful or not. There are people who say it was fantastic: it saved us from the Great Depression. There are people who say it wasn't big enough: if had been, we would have it bigger, we would have had a better recovery. And there are people who say it's the reason we had a mediocre recovery. They are all really smart people. Most of them. The ones I'm thinking of. They have great credentials. They are at great institutions. Some of them have Nobel Prizes. And yet they can't convince each other. Why not? What's going on there? Guest: Well, first, before we go on to there, I want to say that--you know, when you talk about things being a science, my whole point earlier was that even science isn't a science in the way that some people like to imagine that it is. Russ: I agree. Guest: You have the smartest people--the thing I said about maybe balls roll down things differently in a different region of space: That's a real theory. I'm not kidding. I didn't make that up. Okay? On the spot. That's a real theory, and you had some really smart people saying, 'Well, maybe this theory is right.' You have some of the smartest physicists in the world right now who think that in different regions of space, different laws of physics might actually prevail. And it's an active debate. You see people on both sides. So, even science is not quite as sciency as people may hope--this shining idealized version of it. You have the smartest people in the world radically disagreeing on things in physics, too. So maybe the bar for being a science isn't quite as high as people think. Russ: Fair enough. Good point. Guest: But as for economics, I think that the thing about the stimulus speaks to the point I was making before, which is that when you--it's not at all clear. The stimulus was not a natural experiment at all. It was not randomly decided upon. Okay? Nor are the variations from state to state necessarily random--which are often used. And even the effect of those variations depend on some assumptions about the national level effect, too--which again is not a random thing. It's not as if Obama just woke up one day and some quantum state flipped in his brain and said, 'Let's do an $800 billion dollar stimulus' and then it happened. It wasn't like that. There was a reason why it happened. So if you want to say, oh, we did this whole stimulus and the economy didn't recover, that's proof; someone else can say, 'Oh, it would have been even worse.' And given how bad the Great Depression was, maybe they are right about that. Or some people say, 'Well, that's the reason'-- Russ: But they could say the same thing about any empirical study. Very rarely do we get a truly random imposition. Certainly that would be true of the minimum wage. Guest: Sure. Russ: It's very possible that states that have lots of poverty are more likely to pass them, or vice versa--that there are states that are particularly healthy so they pass them to look good but they are not really going to have much effect because actually the market-clearing wage is well above the wage that was passed. Guest: Sure. Russ: But going back to my original point, can you name a study that you think people go, 'Well, wow, that one was so good I have to throw up my hands. I give up.'? Guest: Well, I don't know about throwing up your hands and giving up. I think a lot of people who change their minds will not be despairing or giving up. They'll think, 'Wow, this is cool. Today I learned something.' Russ: Well, that's what I meant by giving up: I'm giving up my old position-- Guest: But I can definitely draw a contrast-- Russ: I'm doing[?]-- Guest: I think that definitely happens all the time-- Russ: I'm doing the old Keynesian thing, which was evidently not really said by Keynes, which is: 'When the facts change, I change my mind. Don't you?' Guest: Right. Russ: Evidently that isn't necessarily Keynes's quote, but it gets attributed to him. So, has there ever been a study where people said, 'Yep, I have to change my views now.' Guest: Well, I don't know because I haven't done those before-and-after surveys of economists' opinions. I've seen snapshots through IGM [Initiative on Global Markets] or some of the other surveys of economists' policy positions that have been done. But I just don't have the data to answer your question. One study that's been quoted to me as a study that made people change their mind was the Cullen and Levitt--and I think there was someone else; I hate leaving one person out but I can just remember two [Cullen, Jacob, and Levitt] --Cullen and Levitt's study on the Chicago education lotteries when they randomly let some people go to good schools and other people not; and they showed that the people who went to the good schools don't really show any better academic outcomes, but they do show better behavioral outcomes. And there have been a couple more studies like that, that basically show the same thing since then. That was quoted to me by one of my Econ teachers as a study that really made him re-evaluate his ideas. You can easily see a qualitative difference--immediately and easily see a qualitative difference between that kind of lottery experiment and the 2009 Stimulus Bill Russ: [?] Guest: There is a qualitative difference in that. Russ: Fair enough. Guest: And the idea-- Russ: I'll concede that one. Guest: Great. The idea that you'll never convince everyone, I say, well, even in physics you'll never convince everyone. I mean, not-- Russ: No, for sure. Guest: Even in physics you'll never convince every single person. Russ: Yep. Guest: You'll still have that crazy guy, maybe at the Steklov Institute. Anyway, then, you have--it's on a sliding scale that I don't have data with which to measure, of convincingness. You know? And the stimulus is pretty low on that convincingness scale because it's so non-exogenous. It's obviously so non-exogenous. And those lottery studies are just much higher on the convincingness scale. On the human convincing power of its identification assumptions. And I wish I had really good evidence on a whole bunch of studies on how they made economists change their opinion and I don't. And someone needs to do that kind of study where they measure a bunch of people's ideas and then they wait till after some studies and then they measure their ideas again. I don't know how they'd do that. Someone should be working on this. But I think you'd see big differences. Russ: That's the difference between you and me. I'm pretty confident that that wouldn't decisively determine whether I was right or wrong. I'd rely on less formal evidence. But I like the claim. I think it's an interesting point. Guest: Well, I can prove to you that you can never convince everybody. I can program a robot that will not be convinced by evidence, okay? Russ: That's not the question. Guest: I can program you a robot that just says, you know, 'Stimulus is awesome.' And I can program this robot and I can even have it tweet 'Stimulus is awesome' and I can program another robot that says 'Stimulus is terrible' that I can have that one tweet. So you know that there's obviously some sort of being out there, maybe an artificial intelligence that can't be convinced of things. But what does that prove? It proves nothing. It's trivial.
51:36
Russ: I'm making a much weaker claim. In the aftermath of the stimulus package and the Great Recession, Richard Posner wrote a piece that got a lot of attention, where he said, 'All my life I made fun of Keynes, but it turns out he was right.' More or less. Guest: I saw that, yeah. Russ: It's not exactly what he said, but it was close. And, that's one person--he's not an economist but he does a lot of economic writing. I'd[?] call him an economist even though he doesn't have a Ph.D. in economics. And that was interesting. It was brave. I didn't find it convincing--I wouldn't. So, I'm not talking about a case that a study comes along that convinces everyone. I'm talking about a study that convinces anyone. Because most studies, my claim is, don't convince anyone. They find what's wrong with it; they easily dismiss it because they have priors against it--whether they are ideological or methodological. Which suggests--what I'm really saying--natural experiments-- Guest: Well, that's a hugely strong claim. That's not a weak claim. Russ: Noah, here's what I'm really saying: Natural experiments are really good for getting publications in economics journals. They are not so good for convincing people. So it's not that they are not as good as, not quite as good as real experiments. It's that they are really only good for generating journal articles. As opposed to getting people to go, 'Wow. I never knew that.' That's my claim. Guest: Wait a second. Okay. Back up your claim, now. How are you going to back up that claim? Russ: Because I don't see a lot of people writing about the fact--other than Posner, which is kind of like the exception that proves the rule. Someone that writes an article, says at a workshop, blogs, that says, 'You know, I always thought that the public school system was awful. But now, after that study of its efficacy, I'm wrong.' Or, vice versa: 'I've always been a big critic of the public school system, but now that I see these results, I have to admit I'm wrong.' I don't see that. Ever. And especially I don't see it-- Guest: Well, what about me? I have changed my mind in response to some of these studies. Including the lottery study I just talked about, the Cullen and Levitt--and possibly someone else, I forget. Russ: So, what did you change your mind about there? Guest: I thought that being able to go to a really goo