By Peter Murray & Steve Ma
The nonprofit organization Worldreader
launched in 2010 with a simple and clear mission:
to bring digital books to disadvantaged
children and their families. In just five years,
the organization has expanded its operations
to 54 developing countries. Today it offers
15,000 books in 43 languages, and it has
reached more than two million readers.
Worldreader is not just on the leading edge of international education
and technology. Its growth also reflects a new wave of nonprofit
organizations that employ a rapid experimentation method called
“lean.” First developed for use in the for-profit business world, the
lean method focuses on swiftly turning new ideas for products or
services into iterative experiments. Lean practitioners build simple
prototypes called “minimum viable products” (MVPs), move quickly
to get feedback on these MVPs from constituents, and then develop
iterations of their MVPs on the basis of that feedback.
The founders of Worldreader embraced a culture of lean experimentation
from day one. Instead of spending significant amounts of
time and money launching a full-fledged platform, they developed
the smallest-scale version of the platform that they could muster
(their MVP) and tested it in the field. In that experiment, which
began in March 2010, they introduced Amazon Kindle e-readers to
16 sixth-graders in Ayenyah, Ghana. The Worldreader team hypothesized
that the kids would embrace the e-readers, that they would
read more, and that their literacy rates would increase.
Focusing on a single school may seem terribly inefficient. But this
high-touch MVP approach (sometimes called a “concierge MVP” in
lean parlance) enabled Worldreader to find holes in its platform and
to troubleshoot problems before investing more time and resources
in the initiative. The Worldreader team saw, for example,
that the screens on the Kindles kept breaking because kids were sitting on
the devices during recess. “We taught students how to care for
e-readers,” says David Risher, cofounder and president of Worldreader.
“And we took the broken screens to the Kindle factory
and asked [people at Amazon] to make the next generation of Kindles
more durable—and they did.”
The most important test of Worldreader’s rapid experimentation
approach came in late 2011, when the organization faced a critical
challenge: The proliferation of basic-feature mobile phones—what
we now, in the era of smartphones, call “dumb phones”—across the
developing world created an opening for Worldreader to expand far
beyond its incremental, Kindle-based growth model. How could it
take advantage of that opportunity?
Some nonprofits, confronted with an opportunity of this kind,
might shift into a strategic planning mode. They would start by
conducting a series of internal debates about what the “right” strategy
is, and then they would focus on developing work plans, board
presentations, and funding proposals. Worldreader followed a different
course. Instead of launching a grand planning and development
process, the organization set up a small experiment to test a
critical hypothesis—the “riskiest hypothesis,” as lean practitioners
call it—of the proposed mobile strategy: Would children in the
countries targeted by Worldreader actually read books on a basic
phone? To answer this question, the Worldreader team partnered
with an app developer that had already created a basic-phone reading
app. The MVP version of Worldreader Mobile consisted of nothing
more than lists of books and a simple text reader. It had no book
covers, descriptions, ratings, comments, or bookmarks. But it had
just enough functionality to allow the team to test that hypothesis.
Almost immediately, as it turned out, thousands of users downloaded
the app and began using it. Only at that point—after the experiment
had verified the hypothesis regarding user demand—did
Worldreader enter a formal contract with its app developer and begin
to make improvements to the product. Today, more than 185,000
users read books on the Worldreader mobile platform every month.
Continuous rapid experimentation, along with a focus on building
solutions that work for children and their families, has made that
kind of growth possible.
The Art of Lean
In our work, we regularly interact with leaders of social purpose organizations.
When they first hear about lean, they often say, “Of course
I run a lean organization. I run a nonprofit.” But there is a world of
difference between being lean and being frugal. Most nonprofit leaders
believe that they have to be frugal: They pay low salaries, rely on donated
goods, and work in cheaply furnished offices. Being lean goes far
beyond cost-cutting, however. The lean process enables organizations
to speed up and focus experimentation in order to reduce wasted effort.
Many organizations spend a great deal of time and resources on
building solutions that don’t end up achieving their intended impact.
Lean accelerates the process of weeding out ineffective ideas and helps
quickly validate ideas that show real promise.
Today, the dominant mode of operation in the nonprofit sector
puts a premium on strategic planning. It emphasizes processes that
generate multi-year plans that cover—often in elaborate detail—a variety of tactics, roles, and outcomes. The old adage “Plan your
work and work your plan” captures the spirit of this approach.
Planning is important, of course. But by its nature, it discourages
experimentation and risk-taking. The emergence of strategic philanthropy
has reinforced this emphasis on planning. Under that
model, funders encourage nonprofits to propose specific tactics for
every desired outcome and to adhere to those tactics over multiple
years. The strategic philanthropy model works well for problems
with clear, proven solutions, but often it doesn’t work for problems
that require new approaches.
The lean model reinvents the traditional strategic planning process.
In effect, it offers a new adage to follow: “Plan your tests and test your
plans.” Lean practitioners don’t enumerate the precise tactics that
they will use because they don’t know in advance which ones will be
successful. Instead, they run many small tests and adjust their efforts
after discovering what works (and what doesn’t work). Done well, lean
helps organizations innovate more efficiently, build new services that
meet the needs of their constituents, and develop disruptive solutions
to seemingly intractable problems. Lean can be particularly effective
as a means of testing and validating revenue models that have the
potential to create sustainable, long-term funding streams.
The adoption of rapid experimentation has been slower in the
social sector than in the business sector (for reasons that we will
explore below). Yet there are a growing number of nonprofits that
use lean to support innovation in education, health care, international
development, and other fields within the social sector.
The Origins of Lean
To understand the lean method—and its applicability to the nonprofit
sector—it helps to understand its origins in the for-profit sector.
Part of a broad revolution in the business world, lean belongs to a
set of innovation and process improvement methods that also includes
Six Sigma, which managers at Motorola developed to enable
error reduction; Agile, a flexible and iterative approach to software
development; and Human-Centered Design, a solution-building
process created by leaders at the design firm IDEO.
Lean has two distinct strains: “lean production” (also known
as “lean manufacturing”), a structured method first developed
by Toyota more than 25 years ago that applies to complex processes
like manufacturing, logistics, and health services; and “lean
startup,” a set of principles and practices developed in Silicon
Valley over the past decade that help entrepreneurs and intrapreneurs
launch new products and services.1 Think of lean production
as a way to maximize the efficiency and impact of a good
idea, and think of lean startup as a way to figure out whether an
idea is worth pursuing in the first place. Although the two strains
developed separately and have distinct processes, they share a
commitment to identifying clear hypotheses, conducting rapid
experiments, and developing new product or service models in
response to experimental data.
Over the past decade, several developments—increasing global
competition, accelerated technological change, the emergence of big
data—have forced nearly every major company to adopt data-driven,
rapid experimentation methods in most aspects of their operations.
Today, when you buy a pair of stretch pants at H&M or download a
new iPhone app or make a purchase from Amazon or click a link on
Facebook, you are generating data for a series of experiments that will
inform how companies make their next strategic decision. Companies
that have incorporated rapid experimentation into their operations
range from large corporations like General Electric, Target, 3M, and
Xerox to high-growth start-ups like Dropbox, Etsy, and Upworthy.
The Elements of Lean
The lean process, as it applies to the business world, has several core
components. We have adapted those components to form a model
that suits the way that organizations operate in the social sector.
Here we will list the components in the order that they might occur
in a typical lean experimentation project. But keep in mind that lean
is more circular than it is linear, and the sequencing of components
in any given experiment will vary. (See “The Lean Experimentation
Process” below.)
Ideation and analysis | With your target constituents in mind (or,
better yet, with your constituents in the same room), generate ideas
for programs and solutions that you think might solve their problems
or help them achieve their aspirations. These ideas are what we
call “value hypotheses.” As you develop such ideas, analyze similar
programs and solutions that already exist, and figure out how your
approach might improve on those offerings. (In the business world,
that process is called “competitive differentiation”).
Constituent discovery | Get out of your office and listen to the
people you hope to serve. Through surveys and one-on-one conversations,
find out what your constituents truly need and want. Put
your value hypotheses in front of constituents, and observe how they
respond to those ideas. (In the business world, this process is called
“customer discovery.”) Done well, constituent discovery will bring
to light ideas that you hadn’t considered, and those ideas in turn
should lead you back to the ideation phase. Ideation and constituent
discovery should complement each other in a rapid feedback loop.
Building | Determine the one or two “riskiest hypotheses” that
apply to your idea. A risky hypothesis, in this context, is an assumption
that is critical to the success of your idea—an assumption that
may, however, prove to be invalid. In the lean process, you should focus
your attention on the riskiest hypotheses. To test those hypotheses,
develop an MVP (that is, a basic prototype of your idea). Also create
a rough financial model for your idea that covers cost estimates and
potential revenue sources. In many cases, your MVP will be a small-scale
version of your program or service. (One common lean tactic
is to customize and test pre-built products. This approach is widespread
in the technology world, where there has been a proliferation
of ready-to-use tools for developing apps, social platforms, and the
like.2) Another option is to build a “paper MVP”—a lean tool that dramatically
reduces the cost of testing demand for a program. A paper
MVP can take the form of a simple flyer about a not-yet-built program,
for example, or a basic online sign-up page for a prospective service.
Testing | Design a plan to validate (or invalidate) your riskiest
hypotheses. Then roll out your MVP to a group of constituents and
collect data on how they react to it. Be sure to test the MVP in a way
that will provide data on metrics that pertain to those hypotheses.
Avoid focusing on vanity metrics that might give you feel-good
results but don’t actually help you validate or invalidate an idea.
Responding to data | Analyze the results of your test. Did your
MVP appeal to fewer people than you had hoped it would? Did it
encounter unforeseen logistical challenges? Did you charge a price
for it that ended up being too high?
If your data show that you have a flop on your hands, hit the
reset button and begin the experimentation process again before
investing more resources in your idea. In the lean startup field,
that’s called a “pivot.”
If your data show promise, use feedback from the test to build a
better iteration of your idea. Then test that version of the idea, and
continue iterating and testing the idea until you have verified that it
will deliver its intended value. We call this process the “build-test-respond”
cycle. (It’s a variation on the “build-measure-learn” cycle
used in the lean startup model.)
Scaling up | Once you have an idea that works, use the data that you
have gathered during the constituent discovery and testing phases
to get buy-in—from your board, your staff, and your funders—for
implementing the idea more widely. As you scale up, continue to run
experiments on ways to increase efficiency and to create additional
value for your constituents.
The Practice of Lean
The Coalition for Humane Immigrant Rights of Los Angeles
(CHIRLA) provides a textbook case of how an organization can use
lean to identify promising service models. CHIRLA serves people
who confront barriers related to language, discrimination, undocumented
status, poverty, and limited access to technology. In 2014,
the organization was seeking to develop new services that would
meet the needs of its community, significantly increase its membership,
and provide financial sustainability. (In particular, it sought
to create services that would generate at least as much income as
they cost to provide). Instead of devoting large amounts of time and
money to implementing one or two ideas—ideas that may or may
not have worked—CHIRLA leaders launched a lean experimentation
process. Over the course of just a few months, they were able
to test the viability of more than a dozen potential services.
As part of an ideation and analysis phase, CHIRLA leaders drew
on their deep experience with serving constituents to gain a sense
of what those constituents might want or need. Using that insight,
they developed a list of more than two dozen offerings that they
thought had the potential to provide significant value in a financially
sustainable way. The list included financial services (such as
prepaid debit cards for unbanked immigrants), legal services, English
classes, prescription discount cards, low-cost international phone
cards, and health insurance products. The CHIRLA team then did
market research to learn about similar services that other organizations
were already offering.
A building phase came next. The CHIRLA team chose 14 of the
proposed services and developed paper MVPs for them. Instead
of building a full working version of any of those offerings, the
team developed flyers that described each potential service. For
most services, the riskiest hypothesis hinged on a simple question:
Would people actually sign up for them—and would they pay a price
that would make them sustainable? The flyers made the services
tangible and allowed the CHIRLA team to begin assessing how
much demand there might be for each offering.
In the following phase of its work,
the CHIRLA team engaged in both
constituent discovery and testing. The
team developed a survey that combined
general questions with MVPspecific
questions that focused on
determining
the viability of their ideas
for new services. Rosamaria Segura,
membership coordinator at CHIRLA,
led the constituent discovery process.
She delved into the lives of local immigrants
to understand their needs
and aspirations. In each constituent
interview, she also tested the riskiest
hypotheses for six to eight service
ideas. “Inviting our constituents to
help us discover what services they really
needed was a game-changer,” Segura says. “As
the data came in, we responded to the feedback,
reconfigured the surveys, and quickly
got a sense of whether our ideas were worth
pursuing and where our blind spots were.”
Ultimately, Segura completed more than 100 constituent discovery
interviews. She and her colleagues now had data on which services
people would or would not sign up for. The work of responding to the
data began almost immediately. CHIRLA leaders concluded that 10 of
the proposed services either didn’t have sufficient demand or would
require significant iteration before further testing could take place.
Four of the proposed services, meanwhile, had strong demand and
merited further exploration. In addition, a review of the interview
data led the CHIRLA team to explore several new service ideas.
One of those ideas involved offering classes to help people in
the CHIRLA community pass the written driver’s-license exam
in California. The state had recently passed a bill that would allow
undocumented immigrants to apply for driver’s licenses. According
to state records, however, 70 percent of those who take the exam in
a language other than English fail in their first attempt. CHIRLA
leaders, noting that many undocumented immigrants would fall
into that category, saw a new need that their organization could
fill. They developed a plan to offer classes on passing the exam and
moved quickly to test the viability of that idea.
Initially, CHIRLA staff members thought that they might need
multiple sessions to prepare immigrants for the exam. But instead
of building a curriculum around that hypothesis, they designed a
simple three-hour course—an MVP, in other words—and ran trial
classes for 60 constituents. After that single three-hour session,
nearly 90 percent of participants passed a mock version of the
driver’s-license exam. Clearly, a multiple-session course wouldn’t
be necessary. Through the MVP test, CHIRLA also learned that
demand was high for the classes and that people would pay to gain
access to them. On the basis of those findings, CHIRLA invested
resources in curriculum development, a train-the-trainer program,
and marketing materials for the new offering.
Lean experimentation enabled CHIRLA to identify a program
model that had three crucial features: high constituent demand,
demonstrated impact, and financial sustainability. Today, the organization
continues to improve the driver’s-license exam class, and it
plans to scale up the model in order to serve thousands of immigrants.
The Varieties of Lean
Organizations throughout the nonprofit sector have begun to apply
the lean method to their operations. Although lean can help organizations
to test and improve a wide array of programs and processes,
it is particularly effective as a way to optimize certain core activities.
Demand testing of new ideas | Lean can help an organization determine
whether anyone will take advantage of a given program or
service. Using lean, nonprofits can test assumptions about the pain
points, needs, and aspirations of their constituents.
GuideStar, an organization that gathers and shares information
about nonprofits, recently created a user advisory panel that
includes about 250 members. The purpose of the panel is to provide
rapid, actionable feedback that will help the organization decide
which innovations are worth exploring. In its
first two months of working with the panel,
GuideStar called on users to help test four
MVPs, along with 10 ideas that were at the
concept stage. Among the products tested
were a mobile app, a Charity Check widget, and a product to help
organizations prepare their US tax forms. After examining feedback
from the user panel, GuideStar is moving forward on the best ideas
and is altering or scrapping the others.3
Short-term outcome testing | Through lean, an organization can
rapidly test strategies for achieving clearly defined short-term outcomes—outcomes that relate to school attendance, reading rates, job
placement, health improvement behaviors, and the like. A-B testing,
in which an organization tests alternative approaches on randomized
samples of constituents, is a critical tool of lean outcome testing.
At a school in the Los Angeles Unified School District, investigators
conducted a rapid experiment to test the impact of parental
involvement
on student performance. In an experiment that involved
A-B testing, the investigators arranged to send some parents of
high-school students text and email messages to notify them that
their kids had missed an assignment. As it turns out, the students
whose parents received the messages experienced performance
improvements
that were much larger than the gains shown by students
whose parents didn’t receive such messages.4
Process efficiency improvements | Lean production, the process
improvement strain of lean, can help an organization improve the
flow of a process by identifying and eliminating waste. In that way,
lean can streamline a program and increase its impact.
The American Red Cross used practices from the Toyota Production
System (TPS)—a precursor to lean production—to improve in-the-field training for its disaster volunteers. Through that effort, the
organization was able to reduce the time required to register and
train volunteers from 3 hours and 45 minutes to just 30 minutes.5
Similarly, the Food Bank for New York City used TPS-based practices
to test alternative approaches to serving, seating, and line management.
As a result, the organization cut the average wait time for its
patrons from 1 hour and 30 minutes to 18 minutes.6
Revenue growth | Given its roots in the business world, the lean
method is particularly well suited to testing new revenue-generating
strategies. Lean, for example, can help an organization evaluate
its plans for fundraising optimization, membership growth, social
ventures, and program fee changes.
Environment America is a federation of state-based advocacy
groups. Before the start of each major campaign, a small team of canvassers
from the organization tests a variety of pitch messages. The
purpose of those messages is both to recruit supporters and to generate
income for the organization. Following that initial test, members
of the team analyze metrics that include the percentage of people who
listen to a pitch, the percentage of people who make a contribution,
and the average contribution amount. Using those data, they determine
which pitch is most effective, develop materials to support that
pitch, and then train hundreds of staff members to use it.7
Citizen organizing | Lean enables advocacy groups to experiment
with various campaigns and campaign tactics. Through lean, such
groups can rapidly test which media channels and which messages
actually move people to take action.
SumOfUs, a corporate watchdog group that organizes citizens
through online petitions, has more than five million members. Each
week, the organization conducts micro-experiments to evaluate
dozens of email-based corporate accountability campaigns. It ends
up shelving more than 80 percent of those campaigns because the
experiments reveal a lack of member interest in them. Then it focuses
its resources on the campaigns that its members clearly care about.8
The Challenge of Lean
In 2013, Steve Blank wrote an article in Harvard Business Review titled
“Why the Lean Start-Up Changes Everything.”9 It was a provocative
title, but it was accurate enough: Rapid experimentation methods
have permeated the business world. So why haven’t they spread as
widely within the social sector? Does lean not apply as directly to
social problems as it does to commercial situations?
There are, to be sure, limitations to applying lean in the social sector.
It cannot replace longitudinal research. No form of rapid experimentation,
for instance, can test whether an intervention aimed at kids in
preschool will affect high school graduation rates. Nonprofit leaders
also find it difficult to measure social impact using the kind of cold,
hard numbers that lean favors. It’s easy to measure revenue. It’s much
harder to measure (say) the effect that a given strategy might have had
on changing people’s minds about a social issue. Lean, moreover, can
be disruptive to existing programs and disorienting for staff members
who are comfortable with established approaches to pursuing social
impact. Perhaps most important, lean works best as a tool for testing
and improving discrete programs and processes. It cannot serve as a
master strategy, and it cannot answer fundamental questions about the
theory of change that governs an organization’s overarching approach.
In the business sector, companies have adopted rapid experimentation
methods partly in response to increased global competition
and accelerated technology change. For-profit companies that
don’t quickly adapt to the new environment will ultimately collapse.
Organizations in the social sector are generally less vulnerable to
such disruptive forces. But as these forces spread across the sector,
more and more organizations are likely to adopt the lean method.
For rapid experimentation to become widespread in the social
sector, funders will need to embrace new approaches to supporting
innovation. Traditional funding processes for nonprofits discourage
rapid experimentation by reinforcing risk aversion and an adherence
to top-down planning. The rules for submitting grant proposals
often require nonprofits to spell out every strategy, tactic,
and outcome in a detailed timeline. In addition, many funders have
adopted cumbersome grant amendment processes that inhibit efforts
to test new approaches.
A few pioneering foundations are trying to change this situation by
explicitly funding experimentation.10 Contests and prizes like those
funded by the Gates, Knight, and MacArthur foundations have opened
up space for experimentation. The use of prizes, however, works only
within very limited parameters and sometimes causes more harm than
good.11 Fellowships, meanwhile, provide people who have an entrepreneurial
mindset with an opportunity to develop and test new ideas.
But that kind of individual support is rarely enough to catalyze a culture
of rapid experimentation throughout an organization. Funders,
therefore, should make sponsorship of lean experimentation a larger
part of their ordinary grantmaking process.
The Power of Lean
All too often, the process by which nonprofit organizations develop
and launch new products and programs can stretch for months or even
years. The lean process, in contrast, enables teams to build and test
a new approach in a matter of weeks or even days. If that approach
is not effective, teams can pivot away quickly. If the approach
needs improvement, they can undertake new iterations rapidly. And if the
approach shows promise, they can cite data to prove its effectiveness
so that funders can invest in it with confidence.
Various tools are now available that will help nonprofit leaders
to engage in rapid experimentation.12 But at its core, the lean process
is simple. In 2014, at a Lean for Social Good Summit in San
Francisco, one of us (Steve Ma) saw just how quickly that process
can unfold. Dominique Aubry, who is now president of Lean
Leadership Inc., spoke about the lean process for an hour. She then
broke participants into teams and had them develop solutions to
specific problems. Next, after they had spent two hours refining
their ideas, Aubry told them to leave the conference facility, hit the
streets, and interview relevant constituents about their proposed
solutions. In just one day, participants went through ideation, constituent
discovery, building, and testing—followed by iterating,
testing again, and iterating again.
If you’re ready to make the leap into lean, start by testing it out.
You don’t need to hire consultants who are experts in lean. (We are
consultants, so trust us: You don’t need consultants.) You don’t need
to hold a board vote about implementing lean. And you don’t need
special grant funding or funder buy-in to run lean experiments. You
do need buy-in from your team to embrace rapid experimentation,
and you need to be willing to look at the data that you gather and to
change your approach accordingly. Once you’re ready, get out of your
office and talk to your constituents. Identify your value hypotheses,
build MVPs fast, and test them in the field. Then respond to the
results—and iterate.