‘The Why Axis: Hidden Motives and the Undiscovered Economics of Everday Life’ by Uri Gneezy and John List (PublicAffairs; October 8, 2013)
Table of Contents:
i. Introduction/Synopsis
PART I: ECONOMICS GOES TO SCHOOL: USING CONTROLLED FIELD EXPERIMENTS TO IMPROVE EDUCATION
1. Dropping Out in America
2. The Stay-in-School Experiment
a. Prelude
b. The Experiment
c. The Results
3. The School Violence Experiment: The Culture of Calm Program in Chicago
4. Maximizing Academic Achievement
a. Prelude
b. The Griffin Early Childhood Center (GECC) Experiment
c. The Results
PART II: ECONOMICS GETS CHARITY: HOW CONTROLLED FIELD EXPERIMENTS CAN HELP MAXIMIZE DONATIONS
5. The Seed Money Experiment
a. Prelude
b. The Experiment
6. Brian Mullaney and Smile Train
7. The Direct Mail Experiment: The Power of the ‘Once and Done’ Option
8. Raffles, Lotteries and Tontines
a. Raffles and Lotteries (and the Beauty Effect)
b. Tontines
9. The Matching Effect Experiment
PART III: ECONOMICS AND BUSINESS: HOW CONTROLLED FIELD EXPERIMENTS CAN HELP MAXIMIZE PROFITS
10. Netflix
11. The Price of Wine Experiment
12. Field Experiments in the Business World: Intuit
a. Prelude
b. Intuit
13. Conclusion
i. Introduction/Synopsis
Until quite recently, the field of economics was dominated mainly by theory-making. Specifically, economists applied their intellects to the human world, and developed abstract models to explain (and predict) the unfolding of economic events. At the heart of all this theory-making stood homo economicus—a narrowly self-interested individual who responded to incentives and disincentives in a perfectly rational way.
In the past half century, though, various economists have added new wrinkles to the field’s repertoire. To begin with, pioneering economists such as Amos Tversky and Daniel Kahneman introduced controlled lab experiments (among other things) into the fold. And these experiments succeeded in adding nuance to our understanding of economic-man (he’s not quite as one dimensional and rational as he was once taken to be), as well as texture and complexity to our understanding of economic phenomenon.
More recently, economists such as Uri Gneezy and John A. List have stepped in and showed that controlled field experiments also have a place in economics. For Gneezy and List, the world is their laboratory: the two go about slyly manipulating the natural environment in a controlled way (often fiddling with incentives and disincentives of all types) to see how we humans respond to the tweaks. Gneezy and List have been practicing this approach for upwards of 20 years now, and in this time they have helped shed light on everything from how to decrease crime rates; to how to improve school success; to how to encourage more charitable giving; to how to promote healthy living and decrease obesity; to how to set prices on products (so as to maximize profits); to how to understand (and limit) discrimination (to name but a few lines of research of theirs). And in their new book The Why Axis: Hidden Motives and the Undiscovered Economics of Everyday Life the two catch us up on their experiments and their results (while also touching on the experiments of other like-minded practitioners).
Take education, to begin with. Gneezy and List have gained a fair bit of attention recently for showing how monetary incentives can be used to help improve grades and graduation rates (particularly with at-risk students)—and even curb school violence; and here we are apprized of the ins and outs of the experiments that were used in this research. What is less well-known is that the authors have also recently become involved in a massive longitudinal study that is designed to test the effectiveness of different approaches to pre-kindergarten education. Though still in its infancy, the study has already yielded some very interesting results; and given that the researchers intend to follow their experimental subjects throughout their lives, the study should help shed a great deal of light on just what approach to early childhood education is most effective.
When it comes to charitable giving, Gneezy and List’s experiments have worked wonders in showing just how to encourage as much charity as possible—and have challenged many of the industry’s long-held beliefs in the process. The authors cover everything from how much seed-money is needed for a project to maximize donations; to how to approach follow-up requests made to established donors; to how to leverage raffles, lotteries and tontines for best success.
On the topic of business, Gneezy and List remind us how a failure to use an experimental approach can lead to business disaster (as illustrated by Netflix’ 2011 decision to modify its business model without experimental research—a decision that drove hordes of customers away, sent the company’s stock plummeting, and nearly sank the business outright). The lesson: business tweaks (including changes in pricing) should be tested in a controlled way in a small market (say a given city) before being adopted across the board (an approach that has been utilized to great effect by such companies as Intuit and Humana).
*To check out the book at Amazon.com, or purchase it, please click here: The Why Axis: Hidden Motives and the Undiscovered Economics of Everyday Life. The book is also available as an audio file from Audible.com here: Audio Book
The following is a full executive summary of The Why Axis: Hidden Motives and the Undiscovered Economics of Everyday Life by Uri Gneezy and John A. List.
*To address all of the lines of research that Gneezy and List have involved themselves in would mean that I would be able to touch on each in only a very cursory way. Thus I have decided to limit myself to discussing 3 lines of research of theirs that I deem to be most wide-ranging and important: education, charity and business.
PART I: ECONOMICS GOES TO SCHOOL: USING CONTROLLED FIELD EXPERIMENTS TO IMPROVE EDUCATION
If Gneezy and List’s work may be said to be about one thing it is this: demonstrating that controlled field experiments can be a very effective way to learn about the human world, and identify practises that will help improve it. This sounds reasonable enough, but the fact is that Gneezy and List’s experimental approach has really found very few adherents in most fields of human endeavor.
Take education, for example. While enormous amounts of money are pumped into education all over the world, very little of this money has ever found its way into controlled experiments meant to determine just what methods of education work best. And this is a real pity, because in many countries (including the United States) the education system may be said to be churning out some fairly lackluster results.
1. Dropping Out in America
One of the very saddest things about the education system in America is how it is failing its most impoverished students. Indeed, the dropout rate among schools in poorer neighborhoods is frighteningly high. As Gneezy and List explain, “dropout rates for low-income Americans hover four to five times higher than for students from high-income families. For example, in 2008, 2 percent of high-income students dropped out, compared to 9 percent of low-income students. And dropout rates for inner-city schools often exceed 50 percent” (loc. 936).
Unfortunately, dropping out has negative knock-on effects that last a lifetime. Just in terms of future earnings alone, the numbers are staggering. As Gneezy and List explain, “dropping out is like throwing away a lottery ticket. The data tell us that for each year of school that a student misses, his or her earning power drops by roughly 12 percent. Indeed, the average annual income for a high school dropout in 2009 was $19,540, compared to $27,380 for a high school graduate. Multiply that number by twenty years, and you see an earnings differential of $156,800. That’s a real winning lottery ticket—enough to buy a house outright in many parts of the country (loc. 1011).
And that’s not all. The fact is that high school dropouts are also much more likely to get in trouble with the law, and spend time in jail; and thus dropping out not only negatively affects the dropouts themselves, but all of society (loc. 2088-91).
Given the damage that a high dropout rate causes, Gneezy and List decided to see if they couldn’t find a way to help keep potential dropouts in school (loc. 1005). Specifically, the two decided to run an experiment to see how monetary rewards might help potential dropouts stay in school.
2. The Stay-in-School Experiment
a. Prelude
Gneezy and List chose to carry out their experiment in an impoverished district south of Chicago known as Chicago Heights. This community, “located thirty miles south of Chicago… has an average per capita income well below the poverty line… 50 percent of the students are Hispanic and 40 percent are African American. More than 90 percent come from poor families on food stamps; many come from foster homes; and most receive free or reduced-fee lunch. As is the case in other urban schools, roughly 50 percent of the high school students drop out before graduation, many between the ninth and tenth grades” (loc. 985).
The superintendent of the Chicago Heights school district—one Tom Amadio—was receptive to Gneezy and List’s idea. However, when the 2 economists came to visit the school board, they found that their idea was met with, in their words, “something close to scorn” (loc. 1041). And it is not that this response was merely a knee-jerk reaction. For example, some of the board members brought up research which suggested that offering a monetary reward for school work ran the risk of being counter-productive, for it had the potential to ‘crowd-out’ other, purer motives (such as the intrinsic pleasure of learning) (loc. 1045).
This was very ironic, for Gneezy and List were well aware of the studies that the board members were referring to, and they had themselves been involved in experiments that showed that the ‘crowding-out’ of intrinsic motives was indeed a very real phenomenon.
For instance, the authors had earlier set up an experiment wherein they offered students varying amounts of money as an incentive to raise donations for a charity (loc. 372-76). Specifically, one group of students was told they would be given 1% of the proceeds they collected (out of a fund that was not tied to the charity money itself), another group of students was told they would be given 10% of the proceeds they collected (again, out of a separate fund from the charity money), while the control group was not given any monetary incentive whatsoever (loc. 376-81). What Gneezy and List found is that the control group actually ended up raising the most amount of money (the 10% group came in second) (loc. 381). Why? Because the monetary reward effectively overrode the intrinsic reward of helping those in need, and the former was simply not as powerful a motivator as the latter (though presumably this may have changed should enough money have been offered [loc. 392]). The lesson: money is not the only (or even the most powerful) motivator, and where intrinsic rewards are at play, offering a monetary reward can actually extinguish the intrinsic reward, thereby becoming counter-productive.
Be that as it may, Gneezy and List had a ready response for the board members’ argument: crowding-out of intrinsic rewards may be a very real concern where these intrinsic rewards are present, but this is decidedly not the case with students on the brink of dropping out when it comes to school work. For potential dropouts, school work is drudgery, plain and simple. Thus Gneezy and List replied to the board members with this bit of wisdom: “when there is nothing to crowd out, money talks” (loc. 1048). Resignedly, the board members accepted this argument, and Gneezy and List were given a pass to try out their controversial experiment (loc. 1050).
b. The Experiment
The only obstacle now was to come up with the money needed to get the experiment off the ground—not an insignificant matter. Happily, though, Gneezy and List were able to secure the support of two very generous benefactors: Kenneth and Anne Griffin—who committed to funding the project with $400,000 (loc. 1092).
Armed with their funding, Gneezy and List set out to design a monetary reward scheme that would get the most bang for their bucks. To begin with, the two decided to focus their resources on grade 9 students, for the statistics indicated that the greatest number of dropouts in high school occurred between the freshman and sophomore year (altogether, Gneezy and List invited 400 grade 9 students into the experiment [loc. 1125]).
In designing their experiment, the authors made use of some very clever psychological tricks. For one thing, the pair decided to issue their monetary rewards at monthly intervals, as opposed to in one lump sum at the end of the semester (loc. 1051). This would ensure that the incentives were shifted to the near term, which is better for appealing to immediate gratification (loc. 1051). (Most people are motivated more by rewards the more immediate they are [loc. 1013], and this is particularly the case with children and teens [loc. 1013-19, 1208]. What’s more, teens at risk of dropping out are even more susceptible to immediate gratification than others, as the very act of dropping out shows a preference for immediate rewards over long-term benefits [loc. 1022]).
The specifics here were that Gneezy and List offered each student in their program $50 per month for staying on the path to graduation (they had to maintain at least a C grade in each of their courses, as well as avoid unexcused absences and suspensions [loc. 1101]). Actually, in half the cases the reward was offered directly to the students, and in the other half it was offered to their parents—another bit of psychological manipulation that the authors felt may pay dividends (loc. 1061-63).
In addition to this monthly incentive, students who successfully met the experiment’s criteria (or their parents) were also entered into a draw. The nature of the draw was such that “if a student who met the standards criteria won, he or she (or their parents, depending on the treatment) would take home the grand prize: $500 in cash (as well as a giant Ed McMahon—style pseudocheck) plus a ride in a white, chauffeur-driven Hummer stretch limo, complete with comfy leather seats, tiny blue and green interior lights, TV consoles, ice compartments, and all the other trimmings” (loc. 1116).
The idea behind the draw was this: we humans tend to overweigh the likelihood of low-probability events (such as winning a lottery, or being involved in a plane crash), and thus the low-probability of winning the high-yielding raffle would have an oversized effect on motivation (and more of an effect than, say, adding an additional $5 to the individual monthly incentive) (loc. 1057-61).
So as not to alienate those students in the experiment that were having trouble meeting the monthly criteria, Gneezy and List set up special provisions for them. Specifically, “if the kids didn’t meet the monthly standards… the researchers would make suggestions for catching up. The researchers even gave the students reminder calls during the month to ask them how they were doing in their classes” (loc. 1118).
c. The Results
So, what were the results of the experiment? Here are Gneezy and List to explain: “our overall results showed interesting gains. We estimated that the program helped about 50 borderline students out of the 400 in the experimental group to meet the ninth-grade achievement standards. Among the students who were on the brink of failing, we figured the program had increased achievement by about 40 percent. Happily, these students continued to outperform their un-incentivized peers after the program ended in their sophomore year. In fact, our estimates suggested that about forty kids who would otherwise have dropped out would receive their diplomas because of our program. (We also found that students’ performance increased slightly more if their parents, rather than they, received the reward.) Given that every additional year of secondary schooling increases lifetime earnings by 12 percent, offering such students an incentive during their freshman year seemed to be a clear, cost-effective intervention” (loc. 1130).
3. The School Violence Experiment: The Culture of Calm Program in Chicago
Finding a way to increase graduation-rates in a cost-effective way was surely a significant discovery for Gneezy and List, but the two wondered if their approach might not be able to help with an even more serious problem: school violence. In an effort to find out, the two teamed up with one Ron Huberman, the head of Chicago Public Schools (loc. 947).
Huberman had been developing a program (called the Culture of Calm) to help stifle school violence by targeting the city’s most at-risk students with extra help and attention (loc. 2147). However, Huberman also wanted to take advantage of an incentive-driven angle, so he invited Gneezy and List in to design and run an incentive scheme.
The scheme that the 2 economists came up with was designed to leverage both competitive spirit as well as star-power. Specifically, the two designed a contest wherein the school that managed to “most profoundly change its culture for the better” (loc. 2155) would win an intimate concert with the international superstar (and hero of urban kids everywhere) Kanye West—who agreed to put on the show pro bono (loc. 2149).
In order to help the participating schools towards success, the project arranged for each school to establish a Culture of Calm committee, made up of student leaders (loc. 2155). The committee’s responsibility was to lay-down the school’s goals, and also to apply pressure towards reaching those goals (loc. 2163).
Happily, the program succeeded in spurring significant improvements in all of the 32 schools that took part (loc. 2170). However, one school in particular really stood out, and that was Farragut High. Here are Gneezy and List to explain Farragut’s key to success: “Farragut High students started by forming a Culture of Calm committee comprised of student leaders—not just the class president and student council, but ‘influential’ kids who played football and so on. It was this committee’s job to decide the basic rules, and they also agreed to two big, overarching requirements: a marked improvement in school attendance and a reduction of violence-based incidents not just in school, but outside school as well. Motivated by the competition for the prize, the kids went to work applying peer pressure. The incentive worked like magic. While all the schools in the Culture of Calm program showed dramatic reductions in violence and a boost in attendance, Farragut reported that incidents of misconduct dropped by a whopping 40 percent” (loc. 2166).
While Farragut High won the contest and was the only school to enjoy the free concert, all of the participating schools (and all of their students) reaped big benefits from the program. Gneezy and List put it this way: “the concert incentive isn’t what really turned things around. The opportunity to see West, in fact, merely legitimized what the kids already wanted: a safe place to learn. ‘They cared about seeing him, but even more importantly, they felt free to stand up to say “We want a safe school,”’ Huberman says. To that end, the students succeeded beyond their wildest dreams. At all of the thirty-two schools in the program, the culture at the schools has remained calm. Teachers are in the hallways; kids don’t pick fights. And violent incidents such as shootings have dropped by 30 percent” (loc. 2171).
4. Maximizing Academic Achievement
a. Prelude
Aside from being interested in ways to help curb the biggest problems that currently face American schools, Gneezy and List have also been involved in experiments designed to help maximize academic achievement. Many of these experiments (as with the experiments mentioned above) have involved testing out various extrinsic rewards (including monetary ones) (loc. 1131-1287); however, Gneezy and List are well aware that the more sustainable approach to maximizing academic outcomes must involve leveraging good teaching and learning strategies (loc. 1287-1302).
The problem, though, is that the education industry simply hasn’t spent much time conducting controlled field experiments to identify what teaching and learning strategies are best. In other words, our education system is built more on hunch than science. As Gneezy and List explain, “the truth is that we have not systematically used field experiments in education to figure out what really works, how well it works, and why it works. In short, we have failed to use the thousands of school districts across the United States as laboratories to create an educational policy that relies on science rather than guesswork and anecdote” (loc. 973).
It’s never too late to start, though, and Gneezy and List are just the pair to take on the effort. There’s one big problem though: if you think that bribing students is an expensive approach, just imagine how expensive it would be to run a proper controlled experiment testing different approaches to education.
Happily, though, there are people in the world who recognize the importance of the experimental approach, and who (more importantly) are willing and able to fork over the funds necessary to get this approach off the ground. In this case it was Kenneth and Anne Griffin—the very benefactors who stepped in with $400,000 to help Gneezy and List with their stay-in-school experiment. On this occasion, the Griffins offered a staggering $10 million to help out the 2 economists (loc. 1331).
With this amount of money, Gneezy and List could afford to set-up a very sophisticated and in-depth experiment. Still, the two wanted to make sure they made the most of their money, so they decided to focus on the one area of education that they felt stood to yield the greatest benefit: pre-kindergarten.
The fact is that by the time students enter grade 1 there is already a wide gap between the highest and lowest achievers (loc. 1471). What’s more, this gap largely reflects the socioeconomic status of the families from which the students come (loc. 1471). Therefore, unlocking the mystery of the best approach to early childhood education not only stands to help all students get a head start on their education, but it also stands to help with closing the achievement gap (loc. 1473).
b. The Griffin Early Childhood Center (GECC) Experiment
With these goals in mind, Gneezy and List decided to carry out their experiment in the district of Chicago Heights—the same underprivileged community wherein they had conducted their stay-in-school experiment (loc. 1328). It is here where, as the authors put it, “the Griffin Early Childhood Center (GECC) was born” (loc. 1331).
Now what the pair needed was a few different approaches to pre-kindergarten education to test one against the other. From their preliminary research, they decided on 3. In the first approach, the curriculum is more traditional in nature. As the authors explain, in this particular scenario, “the curriculum is more traditional and academic. In this school, students work on learning their numbers and letters a la Sesame Street, and are introduced to basic reading. Small groups of children huddle around the table with their teacher and help each other identify shapes. Several children read to each other in the cozy reading corner, assisted by the teacher, who walks around and helps them. The theme one week comes from children’s author Eric Carle, and children’s colorful drawings of their own renditions of The Very Hungry Caterpillar line the walls. The students in this particular arm of the experiment proceed through a curriculum called Literacy Express” (loc. 1356).
The second approach is quite about more experimental in nature. Specifically, the second approach focuses more on learning-by-doing, and there is a special focus on non-cognitive, character-based traits, such as delayed gratification (loc. 1341). As Gneezy and List explain, “the so-called Tools of mind curriculum is based on social skills and structured play. Here preschool children learn to defer gratification… The kids in this school play different roles as they work and stroll through the school ‘town.’ In the bakery section, a little girl is pretending to sell cupcakes to a little boy who has chosen to be a customer. Another little boy pretends to bake pies and cakes on the play stove. At the ‘school,’ one child is a teacher and others are students. At the ‘doctor’s office,’ a young nurse and doctor visit with another little patient. Later, the children practice playing games in which they see who can stand on one foot like a ballerina, or act like a quiet guard. In this way, the children develop the noncognitive skills that are so important to successful functioning—learning to socialize, to be patient, to make decisions and follow directions, and to listen” (loc. 1345).
The third approach is known as the Parent Academy. Here, the students in the program are not placed in an institutional setting. Rather, their parents attend classes and learn how to implement the strategies practiced in one of the 2 main educational approaches (loc. 1356-59). In addition, the parents are offered monetary rewards for their children’s academic progress, and these monetary rewards are doled-out either in the short-term or the long-term (loc. 1361). As the authors explain, “in this arrangement, parents attend group meetings twice a month and learn one of the two curricula taught in the preschool. They also receive financial incentives (up to $7,000 per year) based on their attendance and participation, as well as their child’s developmental progress. These financial incentives are either short-term or long-term. For example, parents in the ‘cash’ treatment receive their money when the results of the regular assessments come in. Parents in the ‘college’ treatment receive an injection to the child’s college account: if their child attends college, they can use their earnings toward tuition and fees. If the child skips college, they forfeit the money” (loc. 1362).
The really wonderful thing about the experiment is that the Griffins’ funding will allow the researchers to track the participating students throughout their lives (loc. 1465). This kind of longitudinal study is necessary in order to understand just what the effects of the different approaches are in the long-term.
And in the short-term, the researchers are testing the students 3 times a year to track their progress (both in terms of cognitive skills, “such as vocabulary, basic writing and spelling, basic problem solving, counting, and pattern matching” [loc. 1468], as well as noncognitive skills, “such as testing for impulsivity” [loc. 1468]).
c. The Results
Though the experiment has only just begun, the early results are very encouraging. (Before taking a look at the results, it should be noted that, at the beginning of the program, the children in the experiment were tested to be in the 30th to 40th percentile in terms of cognitive development—when compared with other children nationwide [loc. 1470].)
Let us look at the two institutional approaches first. As Gneezy and List explain, “overall, the preschool curricula are both working beautifully. Over the first ten months of the program, students in the Literacy Express program have leapt ahead by more than nineteen academic months on their cognitive scores, effectively doubling that of the average preschool-aged child. That is, for every month that has passed, the students have learned almost two months of material. We’re proud of these results. Students’ cognitive scores have also increased considerably in the Tools of the Mind program. Those children are now testing roughly at the national average for cognitive test scores, and are doing quite well on noncognitive skills such as self-control. Students in the two preschool programs are now doing better than the average child nationwide when tested on both cognitive and noncognitive skills. In summary: when the right kinds of incentives are applied via the scientific method, poor kids can do just as well as rich kids within ten months” (loc. 1483).
As for the Parent Academy, the students here are also showing improvement—though the improvement has not been as strong as with the children in either of the two institutional settings (loc. 1484). Also, there is a significant discrepancy between the short-term reward group and the college treatment. As the authors explain, “the short-term incentives seem to be quite strong: kids who have parents in the cash treatment perform much better than those whose parents are in the college treatment” (loc. 1486).
Importantly, while the results at this point favor the institutional settings over the Parent Academy, there are indications that the parental approaches (and particularly the college treatment) may yet pay off big dividends in the long-run (loc. 1486-91)—and Gneezy and List remain hopeful that this will be the case.
One final point of interest: Gneezy and List found that the greatest gains (across all of the groups) occurred in the first few months of the program (loc. 1491-94). Thus the two conclude that this “opens up the possibility of ‘kinder-prep’ programs that can be completed in the summer months directly before kindergarten—when teachers and schools are readily available” (loc. 1497). This is an approach that the two are now testing (loc. 1497).
Here is a short clip about GECC:
PART II: ECONOMICS GETS CHARITY: HOW CONTROLLED FIELD EXPERIMENTS CAN HELP MAXIMIZE DONATIONS
Like education, the field of charity is one where very little experimentation has been done to determine just what works best (only here the goal is to coax money out of the many). Part of the reason for this seems to stem from the fact most charities find it quite easy to survive (and even thrive) with relatively little effort to maximize efficiency. The businessman turned charity champion Brian Mullaney (whom we will learn more about later) puts it this way: “‘Most charities are very inefficiently run by do-gooders… No matter how inefficient or incompetent you are, it is almost impossible for a charity to go out of business. As long as you have a set of PowerPoint slides with pictures that can make people cry, you can raise enough to stay in business” (loc. 2866).
5. The Seed Money Experiment
a. Prelude
Take seed money, for example. When a charity receives a large donation from a generous benefactor or organization, it often likes to use this money to launch a campaign around. The pitch usually runs something like this (as an example): ‘We have collected $5,000 towards our goal of sponsoring the community of x, in the country of y develop a clean water system. This leaves us $10,000 short of our ultimate goal. We ask that you consider contributing to our cause, that we may help the community of x with this important project.’
The idea is that the seed money acts as an indicator that the charity and the cause are worthwhile and legitimate, and thus acts to help spur donations. As Gneezy and List explain, “donors are busy people. They don’t have time to investigate every detail of every charity, so they may look for signals from other donors. Saying you have raised a lot of seed money from an anonymous donor conveys that an ‘insider’ has done her homework and given a large gift” (loc. 2535). (The principle at play here is known as the follow-the-leader effect—for obvious reasons [loc. 2535-39].)
The question, though, is this: how big should you set your ultimate goal in order to maximize donations? Should you choose an ultimate goal that is 90% larger than your seed money? 70%? 50%? 30%? 10% What? A little seed money is good, but the problem is that as your seed money comes to represent a higher and higher percentage of your ultimate goal, the more it runs into the problem of the free-rider effect. As the authors explain, “the closer a charity comes to achieving a fundraising goal, the less a donor… feel[s] she has to give to help reach the goal—she can ‘free ride’ on the donations of others” (loc. 2532).
As it turns out, one of the authors (John List) once had a very personal reason for wanting to know the best answer to the seed money question. You see, in 1997 the University of Central Florida (where List had recently landed a job) decided to start up a Center for Environmental Policy Analysis (CEPA) (loc. 2463). The university bestowed $5,000 on the new center, and put none other than our friend John List in charge of using the funds to raise more money with which to get the center off the ground (loc. 2467).
Being the scientist that he is, List wanted to investigate just what approach would help raise the most amount of money out of the $5,000 that he had. When he looked into the matter, what List found is that there was a rough consensus in the industry that the best approach was to set your ultimate goal to be 3 times the size of your seed money. In other words, your seed money should represent 33% of your final goal (loc. 2475)—as in the example mentioned above. However, what List also found is that this belief was based more on tradition than any scientific evidence (loc. 2469, 2491, 2505). Indeed, he discovered that “absolutely no quantitative research existed about how much money was required to start a campaign” (loc. 2469).
So List decided to begin this research himself, and to use his current opportunity to run a little experiment. Or, more accurately, it was at this point that List came to Gneezy so that the two could run the experiment together (loc. 2511). Together, Gneezy and List (and a couple of their colleagues, James Andreoni and David Lucking-Reiley) came up with a few different approaches to test, and then designed their experiment accordingly.
b. The Experiment
The experiment was to run like this: the researchers would send out mail solicitations to 3,000 Floridians claiming that they were trying to raise $3,000 to buy a computer for their new center (CEPA). However, they sent different versions of the letter to different households. As the authors explain, “in asking people to consider making a contribution toward the purchase of a $3,000 computer, we suggested different seed level amounts in a variety of treatments. In one letter, we said we’d already obtained 10 percent of the cost, so we asked for money to cover the remaining $2,700. In another letter, we said we’d raised 33 percent of the cost, so we asked for help in garnering $2,000 more. Another letter stated we’d raised 67 percent of the cost, so we hoped donors would chip in an additional $1,000… All these different letters were accompanied by the usual ‘thank you,’ a contribution form, and a postage-paid envelope. We sent the letter out and waited” (loc. 2525).
And what did they discover when the waiting was over? They found that bumping up the seed money from 10% to 33% did indeed raise the amount of donations that they drew in (loc. 2527). However, they also found that the 67% condition outperformed them all (loc. 2527). As Gneezy and List explain, “the 33 percent number some of the experts had given us was completely wrong. As it turned out, giving increased when we told people we had 33 percent of the funds already raised, but it increased even more if we told them we had already raised 67 percent of the goal. At lower seed levels (say, 10 percent), the contributions dropped off. It looked like the good people in the charity industry had been leaving free money on the table by focusing so intently on the 33 percent seed-money figure” (loc. 2529).
Put another way, the conventional wisdom regarding the relative power of the follow-the-leader effect and the free-rider effect was shown to be dead wrong (loc. 2535). But this forces us to ask: if the free-rider effect does not yet dissuade donors at the 67% seed money mark, just when does it kick in? Gneezy and List do raise this question in the book; however, they have not yet been able to test it empirically (loc. 2538).
6. Brian Mullaney and Smile Train
While the charity industry as a whole may not have practiced much experimentation to this point, this is decidedly not the case with the (two) charities that have been run by Brian Mullaney (Smile Train and WonderWork.org).
Mullaney made his millions early in life through his (co-owned) ad company called Schell/Mullaney, and then turned his attention to charity in 1998 (after both he and his business partner sold their company off).
The first charity that Mullaney started was called Smile Train. Smile Train provides free surgeries to children with cleft lips in developing countries all over the world (in the developed world, cleft lips are corrected in-hospital shortly after birth [loc. 2796]). All told, Smile Train helps some 100,000 kids per year (loc. 2799).
Not that this success has come without hard work. Much of it is the result of the fact that Mullaney has always applied an experimental approach to his work—and this approach has paid off with ever-expanding donations (the organization now draws in over $100 million per year) (loc. 2885-88).
For example, while the industry had always held to the practice of showing ‘before and after’ photos of the unfortunate people (or animals) that they had helped, Mullaney once decided to experiment with this practice a bit. So he ran a campaign wherein he used ‘before and after photos’ on some pamphlets, while on others he used just ‘before’ photos (loc. 2869-72).
And what did he find? As Mullaney explains, “’when we rant the test, we found when we only ran the ‘before’ photos, the response rates went up 17 percent. Why? The picture of the kid with the cleft palate [Mullaney surmises] haunts you’… Donors felt they had to help the kid with no upper lip” (loc. 2876). (Mullaney also experimented with the racial background and emotional expression of the cleft palate sufferers in his photos, and found that “the sad-looking Caucasian child (who was Afghani) drew the most response. Why? Brian conjectured that white donors—who comprised the majority of the donor pool—preferred to help someone who looked like them” [loc. 2882]).
7. The Direct Mail Experiment: The Power of the ‘Once and Done’ Option
While Mullaney has always been an adept experimenter, he was not so arrogant to think that he could not learn a thing or two from the likes of Gneezy and List, so he invited the two economists in to see if they couldn’t improve on “the best direct-mail letters that he had personally spent years developing and refining” (loc. 2888).
Gneezy and List took on the challenge, and decided to try something truly radical. Specifically, they took half of the 150,000 direct-mail letters that were part of the campaign and emblazoned the following words on the front: ‘Make one gift now and we’ll never ask for another donation again.’ (loc. 2894). Inside the envelope, “the letter told prospective donors they could exercise this right by checking a box on the reply card that said, ‘This will be my only gift. Please send me a tax receipt and do not ask for another donation.’ Donors were given one more option; they could also elect to receive ‘limited mailings’ (which could prove to be a boon for Smile Train in postage savings)” (loc. 2897). The other half of the letters (the control group) consisted of Mullaney’s standard issue, and contained “no special text or slogan on the outer envelope” (loc. 2894).
To give you an indication of just how radical Gneezy and List’s approach was, consider that “many fundraising experts, manuals, and guides would mock the very idea, because one of the most important tenets in fundraising circles is to develop a so-called donor pyramid. In a donor pyramid, the base includes dedicated donors who will give to your cause again and again. When you find such donors, why on earth would you tell them ‘thanks for helping our cause this one time! Now, we will never contact you again’?” (loc. 2899).
Still, Gneezy and List pushed through with their plan; and it’s a good thing too, because the idea (as counter-conventional as it is) turned out to be an enormous success. As the authors explain, “in response to the letters sent out in April, the standard letter raised $13,234 from 193 donors, whereas the ‘once and done’ letter raised $22,728 from 362 donors. In total, the experimental treatment raised much more money and engaged many more donors than the standard letter did. Interestingly, only 39 percent of donors checked the opt-out box” (loc. 2904).
In a follow up study on a larger scale, Gneezy and List sent mail solicitations to over 800,000 people over two years (once again, through Mullaney’s Smile Train charity [loc. 2906]). And once again, the results were just as impressive. As the authors explain, “the ‘once and done’ letters generated a response rate nearly twice as large as the standard letter. It also brought in slightly larger gifts (on average $56 versus $50). Consequently, the ‘once and done’ campaign raised more than twice as much initial revenue as the standard letter ($152,928 versus $71,566)” (loc. 2909).
Of course, the danger of this approach is that it may lead to a decrease in donations in the future (since all those donors who chose to opt out can no longer be targeted). However, the researchers found that this is not what happened. As the authors explain, “if subsequent donations were lower in the ‘once and done’ group, then conventional wisdom would have been correct. That is, we should not have been urging people to ‘bug off.’ Interestingly, what we found is that the subsequent revenue raised turned out to be nearly identical across the ‘once and done’ and the standard letters” (loc. 2912). What’s more, the charity saved a substantial amount on letterhead and postage (since they no longer had to send letters to those who chose to opt out) (loc. 250). All in all, then, it was a win-win situation (loc. 250).
Just why did the ‘once and done’ approach work? Gneezy and List argue that the ‘opt out’ option acted as a kind of gift to potential donors. Having received a gift, the donors felt an inclination to respond in kind. As the authors explain, “by giving recipients the chance to opt out, Smile Train basically offered donors a gift. It relieved them from having to say no to future solicitations. Instead of merely asking for money, the charity basically said, ‘You scratch our back, we’ll scratch yours’… not all of us are selfish. Some of us, even some economists, are nice people who really do want to return a kindness with a kindness. Knowing this, appealing to people’s sense of reciprocity can work” (loc. 2922).
8. Raffles, Lotteries and Tontines
a. Raffles and Lotteries (and the Beauty Effect)
While appealing to people’s sense of reciprocity may be one way to attract more donations to charities, another tried and true method is to appeal to plain old self-interest.
And one of the best ways to do this is to give potential donors an opportunity to win something should they donate—say through a raffle, or lottery. The effectiveness of this approach makes intuitive sense, of course. However, Gneezy and List wanted to know just how effective raffles and lotteries are. So they decided to run an experiment.
In this experiment, Gneezy and List teamed up with the Natural Hazards Mitigation Research Center at East Carolina University (ECU) (loc. 2632). The research center was planning a door-to-door charity campaign, and were willing to allow the authors in to use their charity drive as an opportunity to experiment with a raffle.
The experiment ran like this: in half of the 5,000 houses that were visited, the solicitor just asked for money (loc. 2643). For the other half of the 5,000 houses, though, the potential donor was told they would be entered into a raffle. Here was the pitch in the raffle treatment: “To raise funds, we are conducting a charitable raffle. The winner will receive a $1,000 prepaid MasterCard. For every dollar you contribute, you will receive one raffle ticket. The odds of winning this raffle are based on your contribution and the total contributions received from other Pitt County households. The winner will be drawn at noon at the center on December 17 and will be notified and the results posted on the center’s website. All proceeds will fund the Hazards Center, which is a nonprofit organization. Would you like to make a contribution today?’” (loc. 2641).
Sure enough, the raffle worked like a charm. Specifically, as Gneezy and List explain, “we found that the lottery treatment (which we called ‘the lottery effect’ raised roughly 50 percent more in gross proceeds than the request for donations alone. We found that more people participated in the lotteries; roughly twice as many people opted to donate compared to the treatment where we simply asked for donations” (loc. 2647).
(In this experiment, Gneezy and List also controlled for the attractiveness of the solicitors, and, not surprisingly, they found that attractiveness did have an impact on the amount of donations that were collected—though the researchers were surprised as to just how big this effect was. As the authors explain, “observers rated solicitors like Jeanne on a scale of 1 to 10 for attractiveness. Jeanne was rated high at 8, and she raised about 50 percent more than an equally qualified woman who rated a 6. Perhaps not surprisingly, the women raised the most money when a male answered the door. Jimmy was also scaled to be much more attractive than Stan, and he raised more money than Stan; but women raised more than the men did. What was interesting to us was not that there was a beauty effect—it was the size of the beauty effect. We found that the ‘beauty effect’ was about as large as the ‘lottery effect.’ That is, just by changing the attractiveness from a 6 to 8, we could increase donations by as much as adding the lottery” [loc. 2661].)
b. Tontines
While raffles may work well in helping attract donations to a charity, Gneezy and List found that a slight variation on the raffle concept (called a tontine) may well have even greater potential.
There are 2 main differences between a raffle and a tontine. First, in a raffle there is one grand prize (and maybe a handful of smaller prizes), and everyone is entered into the same draw. In a tontine, by contrast, multiple prizes of varying worth are put for grabs, and the amount that you donate dictates which prize(s) you are eligible to win. So, for example, “if you give less than $20 you’ll be eligible for a few small trinkets, such as a bookmark and a water bottle. For between $20 and $50, you’re eligible to win a bottle of fancy wine. For $50, you might win a shopping spree; for $100, a weekend vacation at a resort; and for $200, you get a shot at a new Lexus” (loc. 2724).
Also, in a raffle your odds of winning the grand prize go up the more you give (since you earn more raffle tickets this way), though the specific odds that you will win are conditional on how much other people give (loc. 2638). In a tontine, by contrast, your odds of winning are fixed, and you are informed of these odds beforehand. So, for instance, your odds may be fixed at 25% (in this case, for each prize-category that is up for grabs, the number of donors entered into that category would be divided by four, and that number of names [representing 25% of the total] would be picked out of a drum, with each name drawn receiving a prize in that category).
Gneezy and List have not (as of yet) tested the tontine approach against the raffle approach in the field. However, they did conduct a lab experiment that indicates that the tontine may well be preferable. Specifically, the authors invited student subjects into the lab, gave them some money, and entered them into either a raffle treatment or a tontine treatment. In each treatment the subjects were given an opportunity to either keep the money or use it to play the raffle or tontine. The subjects were also told that all proceeds from the raffle or tontine would be given to a certain charity (loc. 2730). What Gneezy and List found is that the tontine approach did indeed end up earning more money for the charity (loc. 2738).
Gneezy and List argue that there are 2 main reasons why tontines have an advantage over raffles. One is that tontines offer more variety in terms of the prizes to be won, and thus they appeal to a wider range of tastes (and budgets) (loc. 2738). Second, by setting the odds of winning up front, tontines end up looking less like a gamble than an investment. Thus they reduce the element (or at least the appearance) of risk. And since we humans tend to be predominantly risk-averse, the tontine approach plays more into our nature (loc. 2740).
9. The Matching Effect Experiment
a. Prelude
Now, all of this talk of appealing to self-interest to help boost donations to charities may make us feel somewhat uncomfortable. After all, we may say, isn’t charitable giving supposed to be about the joy of helping others out?
Well, perhaps, but the fact is that there are many factors that motivate us humans to act, and there is no shame in drawing on as many of them as possible in the name of a good cause. And actually, on the topic of there being many factors that motivate us to act, Gneezy and List have discovered one more that may be of interest to charities.
The discovery was made in an experiment that Gneezy and List ran involving the charity strategy known as the matching effect. Matching is when a certain donor or organization promises to match any donation given with an equal donation of their own (or, in some cases, the matcher promises to double-up or triple-up donations etc.) (loc. 2559).
According to the conventional wisdom in the industry, matches are great for spurring donations, and the higher the match-ratio the better (loc. 2565). After all, if you are giving to a charity in order to help that charity (and its recipients) out, you should want to exploit any and all opportunities to maximize the amount of donations that it will receive. But Gneezy and List wanted to test this idea out empirically, so they decided to run an experiment.
b. The Experiment
For this particular experiment, the authors joined up with an (unnamed) liberal political organization and set up a fundraising campaign (through letters sent in the mail) that targeted 50,000 of the organization’s supporters (loc. 2570-73). There were 4 types of letters sent out. In the 3 experimental groups, 3 types of matches were used: a 1:1 match, a 2:1 match, and a 3:1 match (that is, the donors’ contributions were promised to be either matched, doubled or tripled) (loc. 2585). In the control group, no mention of a match was made (loc. 2577).
What Gneezy and List found is that offering a match did indeed spur more donations. As the authors explain, “we found that those people who received a match offer were roughly 20 percent more likely to send in money” (loc. 2586).
However, what the authors also found is that there was virtually no difference in the funds raised between the 3 different types of match. That is, “a 3:1 matching grant was no more effective than a 1:1 challenge. And the 2:1 challenge did about the same as the 3:1 and 1:1 challenges. In light of strong anecdotal evidence that higher match levels are better than lower match levels, this evidence from thousands of observations was shocking” (loc. 2591).
How can we make sense of these results? For Gneezy and list, what the experiment shows is that a big part of the motivation for giving to a charity may well be the sense of satisfaction one gets from the knowledge that they helped the charity (which tends to be triggered no matter how much one gives)—as opposed to the motivation being strictly wanting to help the charity for its own sake (since this would imply that people would be inclined to give more where their money stood to raise more funds in total). As the authors explain, “this notion of charitable egotism has a name: the ‘warm glow’ theory, made famous by our friend James Andreoni. The warm glow comes from feeling good when we donate; helping the local elementary school, supporting the food bank, saving the rainforest, or protecting harbor seal pups raises our self-esteem. Surely, a component of altruism motivates giving, but a warm glow (a.k.a. ‘impure altruism’) is also a motivator” (loc. 2615).
The lesson, then, is this: if you are raising money for a charity, then, by all means, appeal to people’s sense of wanting to help the charity itself (pure altruism). But, at the same time, don’t neglect to also make it clear how good donating will make them feel (impure altruism). Or, as the authors put it, “it’s important to appeal directly to people’s appetites for the warm glow by showing them how good they will feel after donating. When charities (and marketers) recognize that feature of human motivation, they’ll be able to come up with a hundred new and interesting ways to get Mr. And Ms. Citizen to open their wallets” (loc. 2627).
PART III: ECONOMICS AND BUSINESS: HOW CONTROLLED FIELD EXPERIMENTS CAN HELP MAXIMIZE PROFITS
Another arena wherein the experimental approach can pay big dividends is in the business world. But let us begin this section with a cautionary tale. The tale comes from the very successful company Netflix.
10. Netflix
Netflix burst onto the scene in the late 1990s with an alternative business model to the brick-and-mortar video rental store. As Gneezy and List explain, “Netflix was founded in 1997 on the basis of a great question: Would people pay a monthly subscription to have DVDs delivered to their doors (without late fees) instead of having to tromp down to the local video store (which made a lot of money on late fees)? The market responded with a resounding ‘yes.’” (loc. 3013).
As business boomed, Netflix eventually moved online, and began offering subscribers an opportunity to stream videos over the Internet (loc. 3016). With this move, the company scored another major success; and not only that, but the move also effectively ushered in the creative destruction of the movie rental store (witness the end of Blockbuster) (loc. 3016).
By 2011, Netflix had become the clear leader in the movie-rental industry, and “was a darling of the stock market, too: it was trading at nearly $300 a share by July 2011” (loc. 3019). However, it was also in 2011 that Netflix made a near-fatal business decision. Specifically, the company “told its customers in a lengthy, rather confusing email that it was breaking up its bundled mail and streaming service into two separate services. Customers already paid $9.99, $12.99 or $14.99 a month to rent one, two or three DVDs at a time, respectively, depending on their plan, and a limited number of streaming videos. But now, the company said, it would charge all customers $7.99 a month for one-at-a-time movies by mail and another $7.99 a month for streaming service, effectively raising its previous prices by 60 percent” (loc. 3022).
Now, on the surface of things, this may not sound like such a terrible idea. Successful companies often raise their prices to reap the rewards of increased demand for their products and services; and though a 60% increase may sound quite high, the move is not unprecedented. Still, Neflix’ customers responded with a near-revolt. Angry customers cut-off their subscriptions in droves, and the stock value of the company plummeted by 51% (loc. 3031).
Netflix took measures to stem the bleeding, but the measures only worsened the problem (loc. 3034). Yet more customers left, and the company stock dropped by an additional 7.4% (loc. 3037). By the time Netflix decided to cancel all of the changes, it had lost over 1 million of its 25 million customers, and the stock value was at an all time low (loc. 3043-46). Not surprisingly, “Neftlix was being universally slammed as a badly managed company. Even Saturday Night Live wound up making fun of it” (loc. 3042).
What could Netflix have done differently to prevent this whole mess? Here are Gneezy and List with their assessment: “rather than coming up with a national scheme to thrust upon customers, and instead of relying on rough-hewn ideas (based on the intuition of some very smart people on the board, maybe a few focus groups, or some expensive consulting firms), all Netflix had to do was run a pilot of their grand plan in a small portion of the country—say, San Diego—and then study its customers’ reactions. The small-scale experiment could have saved the company lots of money without cutting its value. Netflix might have lost a few customers in San Diego, but it would have had a chance to improve the plan (or maybe cancel it altogether) and remain the market leader. Even if this experiment had stirred up some negative attention, Netflix executives could have explained it was a local snag. The damage would have been much smaller and the experiment worth its weight in gold. Netflix has since recovered, and we expect that, given its product base and strong customer profile, the company will continue to do well, especially if it improves its performance by conducting field experiments” (loc. 3054).
Thus the lesson is this: when you want to tweak your business model (including fiddling with prices) your best to try out the changes in a controlled way in a limited market (say a given city) before adopting the changes across the board.
11. The Price of Wine Experiment
Gneezy and List have indeed helped out businesses employ this strategy to great effect. For example, in 2009 Gneezy and List were approached by a wine-maker in Temecula, California. The wine-maker (who is referred to as ‘George’) wanted help setting the price of his wines (loc. 3173). Unfortunately, there is no magic theoretical rule that can tell you exactly what price you should set for a product or service so as to maximize profits (loc. 3182).
So the authors suggested setting up a little experiment. The experiment took place at the winery itself, where, “as with other wineries in this region, [visitors] can taste different wines and subsequently choose to buy from the selection. Consumers typically come to Temecula for wine trips, going from one winery to another, sampling, and buying wine” (loc. 3190).
Gneezy and List chose one wine to experiment with: a 2005 cabernet sauvignon that had previously been priced at $10, and which had sold well (loc. 3190). For the experiment, the authors “manipulated the price of the cabernet to be $10, $20, or $40 on different days over the course of a few weeks. Each experimental day, George greeted the visitors and told them about the tasting. Then visitors went to the counter, where they met the person who administered the tasting and handed them a single printed page containing the names and prices of the nine sample wines, ranging from $8 to $60, of which visitors could try six of their choice” (loc. 3195).
Surprisingly, what Gneezy and List (and George) found is that raising the price of the wine from $10 to $20 not only increased profits, it also made the wine more popular! As the authors explain, “visitors were almost 50 percent more likely to buy the cabernet when [George] priced it at $20 than when he priced it at $10!… Using an almost cost-free experiment, and adopting prices accordingly, George increased the winery’s total profits by 11 percent… Since the vast majority of the winery’s clients are one-time visitors (this winery sells most of its wines in its store), very few people noticed the change in price” (loc. 3201).
12. Field Experiments in the Business World: Intuit
a. Prelude
Given the effectiveness of the experimental approach, you might think that