Excerpts from "The Why Axis: Hidden Motives and the Undiscovered Economics of Everyday Life" by Uri Gneezy and John List
Chapter 11
Netflix: A Case in Point
Netflix, the movie delivery service, is the poster child for the need for experimentation in business. Because its product and client base is second to none, it was able to avoid bankruptcy following a series of breathtakingly, entirely avoidable, bad moves it made in 2011.
Netflix was founded in 1997 on the basis of a great question: Would people pay a monthly subscription to have DVDs delivered to their doors (without late fees) instead of having to tromp down to the local video store (which made a lot of money on late fees)? The market responded with a resounding “yes.” The feisty little Silicon Valley company delivered movies people wanted, delivered them fast, and in general did a fantastic job of playing David to the Goliath of video chains such as Blockbuster.
Later, Netflix also started offering streaming online videos, though the selection was far more limited, so that customers could watch films two different ways. In so doing, it effectively disrupted brick-and-mortar video rental outfits, including the giant Blockbuster, which was forced to shutter many of its stores. With twenty-five million happy subscribers, Netflix was a darling of the stock market, too: it was trading at nearly $300 a share by July 2011.
But then, the company did something strange: it told its customers in a lengthy, rather confusing email that it was breaking up its bundled mail and streaming service into two separate services. Customers already paid $9.99, $12.99 or $14.99 a month to rent one, two, or three DVDs at a time, respectively, depending on their plan, and a limited number of streaming videos. But now, the company said, it would charge all customers $7.99 a month for one-at-a-time movies by mail and another $7.99 a month for streaming service, effectively raising its previous prices by 60 percent.
Customers vociferously disapproved of the move, likening it to a “brain fart” on the part of management. Posting a comment on the Netflix site, one fellow named Greg (signing himself as an “ex-customer”) wrote the following:
Dear Netflix,
To say the least, I am shocked and appalled at your recent behavior. It seems like yesterday we were the best of friends. You informed me with your poignant documentaries; I always laughed at your corny B horror flicks. For four years you’ve been the gracious receptacle of my hard earned money, but alas, your current actions have forced me to reevaluate our relationship. Your nominal price increase, while unexpected, does not deter my loyalty. However, your mouthpiece Jessie Becker’s presentation of this upcharge—as an added choice for my own benefit—insults my intelligence and reveals the breadth of your arrogance. Had I been treated like an adult and informed of these changes in a straightforward, honest manner, perhaps we could rekindle our spark. Unfortunately, this course of action is no longer available; your condescending and manipulative tone has irreparably ruined our relationship.1
Netflix got so many complaints that it had to hire extra customer service employees. The company’s stock plunged 51 percent. Then, in September of 2011, CEO Reed Hastings apologized to customers and announced that Netflix was going to try to correct the situation. How? By splitting the company into two operations: one, called Qwikster, would be the mail-delivery business, run by a new CEO; the other would be the online streaming service, called Netflix.
This announcement made customers even angrier. Now, subscribers to both streaming video and DVDs would see two separate charges on their credit card statements and have to log on to two different websites. The stock dropped another 7.4 percent.
Realizing they’d made things even worse, “The Netflix Team” sent out the following email to customers in October 2011:
Dear [Customer’s Name Here],
It is clear that for many of our members two websites would make things more difficult, so we are going to keep Netflix as one place to go for streaming and DVDs. This means no change: one website, one account, one password . . . in other words, no Qwikster.
Netflix thought that some customers would leave; but they were shocked that close to a million customers would drop Netflix. By this time, Netflix was being universally slammed as a badly managed company. Even Saturday Night Live wound up making fun of it.
To see just how costly it was not to experiment, take a look at Netflix’s stock price, pre– and post–mess-up in 2011:
We tell you this story because the company could have avoided the loss of billions of dollars and the damage to its brand if it had run some simple field experiments. Rather than coming up with a national scheme to thrust upon customers, and instead of relying on rough-hewn ideas (based on the intuition of some very smart people on the board, maybe a few focus groups, or some expensive consulting firms), all Netflix had to do was run a pilot of their grand plan in a small portion of the country—say, San Diego—and then study its customers’ reactions. The small-scale experiment could have saved the company lots of money without cutting its value. Netflix might have lost a few customers in San Diego, but it would have had a chance to improve the plan (or maybe cancel it altogether) and remain the market leader. Even if this experiment had stirred up some negative attention, Netflix executives could have explained it was a local snag. The damage would have been much smaller and the experiment worth its weight in gold. Netflix has since recovered, and we expect that, given its product base and strong customer profile, the company will continue to do well, especially if it improves its performance by conducting field experiments.
When we discuss experimentation with business leaders, they usually reply by saying, “Tests are expensive to run.” After we point out that they are not, we turn the tables on them by showing how expensive it is not to experiment, as the Netflix example shows. We politely explain that every day that they set suboptimal prices, place ads that do not work, or use ineffective incentive schemes for their workforce, they effectively leave millions of dollars on the table.
Of course, many businesses do experiment, and often. Businesses always tinker with the machine and try new things. For instance, Apple’s Steve Jobs was constantly experimenting with design and with new ways to sell products. The problem is that businesses rarely conduct experiments that allow a comparison between a treatment group and a control group. Jobs’s launch of the iPod and of the iTunes music store revolutionized an industry. But for years, Jobs insisted that recording artists and record companies charge exactly 99 cents per song on iTunes. Defending any justification Apple can offer for this policy is difficult, however. The company never compared the impact of iTunes prices on its sales of songs and iPods. And in the absence of solid evidence, Apple executives turned to their intuitions. They did well with this strategy, but could they have moved “from good to great,” as author Jim Collins puts it, through experimentation?
Put differently, say that you have a serious illness. You go to your doctor, and she prescribes a new treatment regimen for you. When you ask what evidence she has for trusting this treatment, she says, “It’s my intuition.” In such a case, you’d probably leave and never come back, because you prefer to entrust your life to someone whose medical decisions are based on scientific evidence.
How does making the right business decisions differ from choosing the right medical treatment? You might say lives aren’t at stake, but executives who are paid millions of dollars a year to sign off on decisions can cost people their jobs and the economy billions. Business experiments are research investigations that give companies the opportunity to get fast and accurate data regarding important decisions. By manipulating various factors in the environment, companies can better understand the causal relationship between a change in strategy and a response in consumers’, competitors’, employees’, or other stakeholders’ behavior.
Field experiments in business are also different from other research efforts—say, focus groups—because participants make real-life decisions often without even knowing that they are part of a study. When designed properly, business field experiments can provide invaluable insights and reveal surprising results, which the company can then implement on a larger scale. In this chapter, we tell the story of two great executives who have guided their companies’ futures with field experiments. Along the way, we mix in experiments that we have conducted with these and other firms.
Innovation at Intuit
Intuit, the Silicon Valley–based firm famous for its QuickBooks and TurboTax software, has spent years building experimentation into the core of its being. “We used to make decisions through managerial analysis and opinion, and from the top-down,” says founder and chairman Scott Cook. “Now we let our small, rapid-fire experiments make the decisions for us.”
In the old days, Intuit was run like most large organizations. Product-development folks would come up with ideas. Managers of the business units would pull together data from focus groups and other research, run some analysis, stuff their findings into PowerPoints, disseminate the information to the rest of the company, and their higher-ups would decide whether to fund the project or not. But Cook began to understand that this way of doing work was like walking in concrete shoes. “I started to be convinced that experimentation was the solution to two problems,” Cook says. “The first was how to get a large, successful company to be agile and innovative, because the larger and more successful a company it is, the less innovative and entrepreneurial it can become. The second problem was that the decisions made in the old-fashioned way were often wrong.”
Intuit trained people in “design thinking,” a methodology for investigating problems (especially fuzzy, vague ones), gathering information, and coming up with creative solutions. Design thinkers use a holistic approach, bring creativity to their work, and then innovate new approaches to problems. A small group of design thinkers and executives trained 100 leaders in the organization to run experiments that tested assumptions and hypotheses; they gathered data and came up with solutions. And these leaders taught people who worked for them to do the same. In addition, 150 “innovation catalysts” throughout the organization work in all the firm’s departments to drive this culture of experimentation. Today, everyone is encouraged to toy with new ideas using the same, Galileo-like, scientific experimental methods that we use in our work.
In the old days, people in the Turbotax.com division ran seven experiments a year. Today they are running 141 rapid, low-cost experiments during tax season on a weekly cycle, beginning on Thursdays. They test the idea, run the experiment, read the data, tweak the experiment, and the following Thursday they test again. The rapid experimentation cycle “uncorks innovation and entrepreneurship,” Cook says.
As a company, Intuit frees its employees to spend 10 percent of their time working on projects of their own invention. Today, Intuit experiments whenever possible on a small, cheap basis as the core of the discovery process. Employees who come up with innovative ideas must prove that the concepts work by tracking results from real customers; the most promising ideas rise like cream to the top. In this way, Intuit developed SnapTax (which prepares taxes on a camera or a mobile phone); SnapPayroll (which enables employers to pay their employees via mobile phone); and an Intuit Health Debit Card, which offers health coverage to small businesses that cannot afford health insurance for their employees; and more.
Very often, such experiments result in new product features. For example, the development team used specific experimental questions about their tax situations; based on the answers, the software could recommend either the standard deduction or the itemized deduction. Testing showed that the feature could reduce the time it took to finish tax forms by 75 percent, so all subsequent versions of the product incorporated the new feature, called “Fast Path,” into its free “Federal edition” of the TurboTax software.
Intuit’s development teams also created an “Audit Support Center” that helped guide all customers through the audit experience, just as if they received an audit letter from the IRS. Testing confirmed that more customers started and completed their TurboTax filings when the feature was presented on the website. “Our customer conversion rate—the number of people who purchase the product after shopping around on the Internet—is up 50 percent in six years,” says Cook.
Employees are also encouraged to come up with solutions to serious social problems. In one instance, a team in India developed a service for Indian farmers called “FASAL” (“harvest” in Hindi). The team members had observed that farm families—comprising half of Indian society—were so poor that they didn’t have access to some of the most basic necessities. How, the engineers wondered, could they make these farmers’ lives better?
The team from Intuit conducted its own study, observing the poor farmers both in the fields and when they went to market. Most farmers had access to just one or two markets at a distance from each other, and they had to work through one middleman at each market to get a price for their produce. The middleman sat under a cloth and signaled the price through hand gestures. There was no transparent pricing, and the system worked against the farmers. But the farmers had one big thing going for them: they had cell phones.
So the engineers conceived of a cell phone texting application that let farmers know what the middlemen from a variety of markets were offering. In just weeks, the engineers tested the concept with a quick-and-dirty experiment, hand-typing text messages to 120 farmers that told them which markets could secure them better prices for their crops. The test worked, and farmers began adopting the application. Today, the FASAL service is helping 1.2 million farmers out of poverty.
“FASAL is not a charity. We run it as a business, so we can attack head-on one of the most pernicious problems of the developing world, rural poverty,” says Cook. “We go out and look for the biggest problems we can solve, and a number of them are social problems. We attack these by running low-fidelity, rapid experiments.”
Working with Intuit, we now have dozens of field experiments under way that promise to shed light on what works and why. We suspect many will help move Intuit’s bottom line. Intuit is a great company because the field experiment gene is built into their DNA.
Interventions at Humana
Another company that likes running field experiments is Humana, the giant health benefits firm that started out as a chain of nursing homes and hospitals. “I like to know what makes things hum,” says Mike McCallister, Humana’s affable, mustachioed chairman and CEO. Indeed, McCallister is one of those guys who is constantly thinking about better ways of doing things. In fact, he thinks a lot more like an entrepreneur—or even a field economist—than a CEO. Whereas others may trust their intuitions, he trusts his counter intuitions. “I try to find what is doable,” he says. “People assume that things are not doable, but who is to say they aren’t? Let’s find out!”
For example, in the old days before Humana was a health benefits provider, it owned hospitals and medical buildings, and McCallister was then in charge of the medical offices. The medical offices were money losers; but hospital pharmacies were money-makers. McCallister’s bright idea: attach some pharmacies to medical offices and see how they performed financially when compared to medical offices that didn’t have attached pharmacies. Lo and behold, the medical offices with the pharmacies proved more profitable. Evidence in hand, Humana expanded the pairing across its medical offices and made money. Nobody had ever tried this kind of thing before. It just wasn’t “done” at Humana, or elsewhere in the healthcare industry for that matter. Breaking the mold takes guts, we argue, and evidence from a field experiment that gives you confidence your idea is actually correct.
Once Humana switched to being a benefits provider and McCallister became CEO, he began experimenting with other policies. As an employer, Humana found that its own healthcare costs were out of control, in part because employees weren’t taking care of their own health. McCallister is a big believer in personal responsibility, so he told his employees they weren’t going to be told what to do. Employees had to work on the problem together. One approach was to run little incentivized experiments. Humana offered a weight-loss program that began and ended with a BMI (body mass index) measurement. Those who lost some of their girth had their names entered in a lottery for a hefty check for $10,000. Not surprisingly, this incentive created quite a bit of buzz around the firm—and, yes, some people lost weight.
The weight-loss experiment is a small one; but consider a large-scale experiment Humana is running today. Although McCallister believes all people should have access to affordable healthcare, he recognizes that the Medicare bureaucracy has very little incentive to invest in preventive care. This, McCallister says, leads to “fraud, abuse, and overuse of services.” In the face of a huge generation of rapidly aging baby boomers and ballooning healthcare costs, he thinks there’s a much better way of delivering patient care—one that focuses on patient wellness, which he believes saves both money and lives.
To that end, the company recently adopted a mantra: help people achieve lifelong well-being. But what works? To find out, he hired a consultant named Judi Israel to build a “behavioral economics consortium.” As part of this consortium, we helped design some field experiments and behavioral interventions. Our common goal was to see what kinds of interventions best helped patients improve or stabilize their health while managing costs.
For example, consider a senior citizen on Medicare who suffers a heart attack. She survives the attack, receives appropriate treatment, and goes home. But then she ends up back in the hospital within a month for some comparatively trivial issue, such as failing to take her prescribed medications. Each hospital readmission averages a $10,000 cost, not including “extras” such as prescriptions, rehabilitative services, and so on. Given that a whopping one in five patients on Medicare is readmitted to the hospital within a month of his or her first admission,3 these costs can be massive—and readmission is no fun for the patient, either. Humana, which covers the costs Medicare doesn’t, has a vested interest in addressing the situation.
So the firm did a little poking around in its databases and discovered that a substantial number of the two million Medicare-enrolled members it insures were being readmitted. The company chartered its analytic team to build a model to address this problem. Among other insights, the team found that members who suffered from chronic health problems (diabetes, obesity, heart disease, pneumonia, congestive heart failure, and so on) were at the top of the list. Accordingly, Humana made a point of following up with patients after they were released from the hospital. All patients receive an automated phone call offering help or advice via a toll-free number, but patients with chronic problems receive a call from a nurse who walks them through the steps of their rehabilitative care and makes sure they stay on track. And patients who suffer from several chronic problems at once receive a home visit from a nurse who monitors and coaches them along. More than 100,000 Humana Medicare members with multiple chronic illnesses receive this kind of help.
Through controlled tests, Humana has discovered that a proactive, low-cost, and simple intervention, such as sending a nurse to visit the patient, can save significant amounts of money while helping the patient. We continue to work with Humana using simple behavioral interventions that we trust will make significant bottom line advances.
From a business and healthcare industry standpoint, these moves all make sense. “Our industry has not been innovative,” McCallister insists. “This nation is productive on the back of technology, but there is no innovation in insurance or healthcare outside of products. We are trying to solve a big problem—to control healthcare spending and address deteriorating health at the same time. Maybe what we learn from our experiments here can spread.”
The Price Is Right
Field experiments focusing on products, services, and prices are not just the domain of big companies such as Intuit and Humana. They may, in fact, be even more crucial for smaller businesses, many of which teeter on the brink of bankruptcy daily.
In the summer of 2009, Uri and his wife Ayelet received a call from a fellow we will call “George,” a winery owner in Temecula, California, a lovely, languid town about an hour northeast of San Diego. George asked for their help with pricing his wines—clearly one of the most important business decisions he needed to make. They were delighted to take up the invitation to visit George’s winery, taste some of his products, and possibly help him in the process.
When Uri and Ayelet asked him how he’d chosen prices in the past, they heard about the usual suspects: George looked at how other wineries price similar wines, intuition, his last year’s prices, and so on. He expected the business professors to come over, look around, do some quick calculations—and come up with the magic numbers that would make him rich. You can imagine how disappointed he was when, after having spent some time with him (and his lovely cabernet), Uri and Ayelet told him they had no idea what the “right” price was, and that the magic number didn’t exist. He almost took away the wine he’d already poured for them.
In an attempt to save their drinks, Uri and Ayelet did offer him help, in the form of a method—no magic, no equations, and no superior knowledge—just a simple experimental design. Pricing wines is a particularly tricky task since quality is not objective. We automatically assume a connection between price and quality; all else being equal, if a laptop costs more because it weighs less, people think it’s better. And that’s how much of the world works—evidence that runs counter to this basic intuition is hard to find.
Is this also the case with wines? You’d assume so, since the price range for wines is so enormous—you can pay a few bucks for a bottle of rotgut, or $10,000 for a bottle of 1959 Domaine de la Romanée-Conti. Research suggests that even when evaluating the quality of a product is subjective (as is the case with wine, since people have different taste preferences), increasing its price may increases its attractiveness to consumers.
Visitors to George’s winery, as with other wineries in this region, can taste different wines and subsequently choose to buy from the selection. Consumers typically come to Temecula for wine trips, going from one winery to another, sampling, and buying wine. The wine with which Uri and Ayelet experimented was a 2005 cabernet sauvignon, a “wine with complex notes of blueberry, black currant liqueur, and a hint of citrus.” The price George had previously chosen for it was $10, and it sold well.
For the experiment, we manipulated the price of the cabernet to be $10, $20, or $40 on different days over the course of a few weeks. Each experimental day, George greeted the visitors and told them about the tasting. Then visitors went to the counter, where they met the person who administered the tasting and handed them a single printed page containing the names and prices of the nine sample wines, ranging from $8 to $60, of which visitors could try six of their choice. As in most wineries, the list was constructed from “light to heavy,” starting with white wines, moving to red wines, and concluding with dessert wines. Visitors typically chose wines going down the list, and the cabernet sauvignon was always number seven. Tastings took between fifteen and thirty minutes, after which visitors could decide whether to buy any of the wines.
The results shocked George. Visitors were almost 50 percent more likely to buy the cabernet when he priced it at $20 than when he priced it at $10! That is, when we increased the price, the wine became more popular.
Using an almost cost-free experiment, and adopting prices accordingly, George increased the winery’s total profits by 11 percent. Following this experiment, he happily adopted the results and changed the price of this wine to $20. Since the vast majority of the winery’s clients are one-time visitors (this winery sells most of its wines in its store), very few people noticed the change in price.
Be Creative
Finding the “right” price is important. But sometimes you need more. It’s not just about the price, but also about how it’s collected.
A few years ago, a graduate student at University of California, San Diego, Amber Brown, went to work for Disney Research—a to-die-for kind of job for a young psychologist. Disney has an in-house, interdisciplinary group of researchers that uses science to try to improve the company’s performance and explore new technologies, marketing, and economics. As is the case with Humana, this group understands the importance of using behavioral research to simultaneously improve both the customers’ experience and the company’s bottom line.
At about the same time Amber nabbed her job, we were becoming interested in an emerging behavioral pricing approach: pay-what-you-want. A famous example of this pricing is from the British band Radiohead. In 2007, the band released a CD as a digital download. It encouraged fans to log on to its website and download the album for any price they chose. Fans could get the album for free or pay as little as 65 cents (the cost of handling by the credit card company) or more. But would the fans pay for something they could get for free? And did they pay? Interestingly, hundreds of thousands of people downloaded the album from the band’s website, and many of them (around 50 percent) paid something for the CD. (By the way, as our friend, the recent Nobelist Al Roth likes to say, “Columbus wasn’t the first to discover America; he was the last.” After Columbus, everyone knew about the “new” continent. The same is true here. Radiohead wasn’t the first to discover this pricing strategy, but the group is famous enough to be the “last”—no one will ever need to “rediscover” it.)
This example shows that even in markets, people are not completely selfish. But the data from Radiohead’s model, and other companies who had used it, left many questions open. Clearly people paid more than they had to, but whether the pricing strategy had positive or negative consequences for the band remained unclear. Did the band make or lose money relative to a standard pricing scheme?
We decided to study the pay-what-you-want scheme in a field experiment.5 We thought a combination of a pay-what-you-want pricing strategy and charity might be an interesting way to go. We called this combination Shared Social Responsibility (SSR) because instead of the company alone deciding how much to give to the charity, customers could share in the donations, too. If people could pay what they wanted for an item, would they pay more if we appealed to the “better angels of their nature”?
So together with Disney Research, we designed a large field experiment that included over 100,000 participants to test the effect of pay-what-you-want pricing combined with charity. We set up our experiment at a roller coaster–like ride at a Disney park where people go on the ride and can afterwards buy a snapshot of themselves screaming and laughing.
We offered the photo either for its regular price of $12.95 or under a pay-what-you-want scheme. We also added treatments in which half of the revenue from selling the picture went to a well-known and well-liked charity. This experimental design resulted in four different treatments that we ran over different days during a month-long period.
The figure below shows the profits per rider:
As you can see, we found that at the standard fixed price of $12.95, the charitable component only slightly increased demand—raising the revenue per rider by just a few cents. But what happened when participants could choose their own price? The demand rates went through the roof. Sixteen times more people (8 percent instead of 0.5 percent) bought the photo. But since they only paid about a dollar on average, Disney didn’t make any money from them. (Remember: we are interested in running experiments in which we can find a win-win solution for both companies and their customers. That’s the best way to make changes that stick.)
And what, in the experiment results, were we most interested in? When we mixed the pay-what-you-want scheme with charity, 4 percent of the people bought the picture, but they paid much more (roughly $5) for it. Adding the charity option proved very profitable. In fact, the amusement park stood to make an additional $600,000 a year by offering the pay-what-you-want/charity combination just in this one location in the park. More generally, making this change also increased the benefit to the charity—and presumably to the customers, who felt they were doing something good.
An important takeaway from our experiment is that if you want your clients to act unselfishly, you need to show you can do the same. When Disney agreed to experiment with the new pricing, the company signaled to its customers that it cared about charitable causes and, more importantly, was willing to share the risk of acting on that concern. More generally, we learned that being creative with your pricing strategies proves you can do good while doing well (as we discussed in Chapters 9 and 10).
How Can We Get You to Respond?
As we mentioned in the previous chapter, we’re all used to the piles of junk mail with offers that sound too good to be true (probably because they are not so good or not so true). Many of us never open such mail—we just “file” it in the trash without even looking at it. Those who do open it usually ignore the content or requests. Knowing this, how can a company get your attention through direct mail (or social media)?
Imagine you open a direct-mail plea and a $20 bill falls from the envelope. Whoever sent the mail likely has your attention now. Curious, you read the enclosed letter. The company is asking you to complete and return a short survey. Would you do it? What if there was only $10 enclosed? Or just $1?
Earlier, we showed how charities like Smile Train and Wonder Work.org have successfully used reciprocity, the basic principle that if someone does something nice for you, you should do something nice in return. But what if you’re not a charity?
In this direct-mail case, the company sweetly sent you cash and is asking you to do something for them in return. Let’s say you’re the chief marketing officer of a big chain store, and you ask yourself whether it makes sense to try to appeal to people’s sense of reciprocity when asking them to respond to a direct-mail pitch. Your company has lots of experience in sending surveys and is also good at collecting data. But it isn’t very good at figuring out which kind of incentives work best when it comes to direct mail.
With our colleague, Pedro Rey-Biel (of the University Autonoma of Barcelona), we analyzed the results of a large field experiment, comprising 29 treatments and 7,250 “club members” who were already registered customers of a big chain store.6 The company sent letters asking club members to complete a fifteen-minute survey. The company was interested in the question: Which is better—paying customers ahead of time to respond to a direct-mail plea, or promising to pay them after they’ve responded?
Put another way: Would more people respond—and would the study be more cost-effective—if the company used the reciprocity angle and sent people money with the survey in hopes that they would fill it out? Or would it be smarter to do things the old-fashioned way? That is, would it be better to treat people more like employees and make the reward contingent on having done the work? Or should the company forget the whole incentive thing and just send out surveys without a reward?
In one treatment, the company sent letters with cash, ranging from $1 to $30 (we called this the “social” treatment, since reciprocity is a social phenomenon), to about half the addressees. In another treatment, the company promised to send 3,500 people checks (with the same amounts as in the other treatment) if they filled out the survey (we called this the “contingent” treatment). In the control treatment, the company just sent the survey to 250 people and asked them to respond. The chart below shows the response.
This chart shows the “breaking point” was around $15. Up to $15, we found giving people money up front made them feel like reciprocating and therefore more likely to return the survey, even for small amounts such as $1. In fact, significantly more people responded when we told them: “You’ll get a dollar if you’ll fill out the survey and send it back.” But after $15, more people responded to the contingent, fill-out-our-survey-and-then-
Importantly, the contingent was less expensive than the paying-up-front approach. Makes sense: after all, sending money only to those who send the survey back is cheaper than paying everyone regardless of whether they respond. The average cost of a returned survey in the social treatment was $45.40, more than double the cost in the contingent treatment ($20.97). As a result, the total cost in the social treatments was almost three times higher than in the contingent treatments ($38,820 vs. $13,212).
What can companies that send out direct mail learn from this exercise? If your budget allows you to only pay $1 for a returned survey, put the buck in the envelope. People (at least the nice ones) will be happy to get it and reciprocate in kind. But if you can spend enough money per person, you’ll be better off paying only people who send the survey back. You might, of course, sample different people in the two cases; our bet is that you’ll get more people who think like economists when you make the payment contingent and more noneconomist thinkers when you don’t.
A Trip to China
In Chapter 4, we talked about the way framing a bonus as a gain or loss affected teacher and student performance. Framing can be an important tool for businesses, too. Let’s say that you are the marketing manager of a product called Sunny Sunscreen SPF 50 Lotion, and you are deciding what kind of spin to put on a campaign. Your “gain-framed,” or positive message might go like this: “Use Sunny Sunscreen to decrease your risk of getting skin cancer” or “Use Sunny Sunscreen to help your skin stay healthy.” Alternatively, a “loss-framed,” or negative message could be “Without Sunny Sunscreen, you increase your risk of developing skin cancer” or “Without Sunny Sunscreen, you cannot guarantee the health of your skin.”
Similarly, a manager can tell employees, “If we boost production by 10 percent this year, we will all be in for a bonus!” Or he could say, “If we don’t boost production by 10 percent this year, none of us will get a bonus.” Which kind of framing do you think is the better motivator?
To find out, we took a trip with our colleague Tanjim Hossain (of the University of Toronto) to the vibrant, modern city of Xiamen, in Fujian province on the southern coast of China, not too far from Hong Kong.7
Xiamen is home to lots of large factories—such as Dell and Kodak. The site of our six-month experiment was a 20,000-employee Chinese high-tech firm that produces and distributes computer electronics. The company—Wanlida Corporation—produces and distributes cell phones, digital audio and video products, GPS navigation devices, small home appliances, and so on, which are exported to more than fifty countries.
Our goal was simple: we wanted to see if we could increase productivity at the plant using simple framing manipulations. So we sent two different letters to two different groups of employees.
Imagine for a moment that you are a twenty-one-year old woman—we’ll call you Lin Li—working for Wanlida, and your job is to inspect PC motherboards. You come into the factory on Monday morning, sit down at your desk, and turn on a magnifying light like the kind dentists or surgeons might use. You pull on a pair of lightweight gloves, take a motherboard in your hands, and go over every chip, nook, and cranny, looking for defects. You do this for nine hours a day, six days a week, and, of course, receive a salary for your work.
One day, you receive a letter from management. “Dear Lin Li,” the letter says, “You will receive an RMB 80 bonus for every week in which the weekly production average of your team is above 400 units per hour.” RMB 80 is about $12, which is a pretty nice weekly bonus for a blue-collar worker in China. Because the average salary of workers in China is between RMB 290 and 375, RMB 80 represents more than 20 percent of the weekly salary of the highest paid worker. None of the 165 workers involved knew they were part of an experiment.
Feeling invigorated, Lin Li goes back to work, smiling. Another young employee—we’ll call him Zi Peng—receives a different letter: “Dear Zi Peng, You will receive a one-time bonus of RMB 320. However, for every week in which the weekly production average of your team is below 400 units per hour, the salary enhancement will be reduced by RMB 80.” Zi Peng isn’t quite sure how he feels about this arrangement, but he goes back to his desk and takes up his work with gusto.
Now, this kind of framing might remind you of the incentives that we tried on teachers and students in Chapter 4, when we said that they would lose their money if they didn’t perform well. And as you also probably noticed, this kind of framing combines a carrot (“you will receive a bonus”) with a stick (“if you don’t produce enough we’ll take your bonus away”). The message is clearly—and intentionally—mixed, because we wanted to see the effects of what social scientists call “loss aversion” at work in a real factory scenario.
When we feel we “own” something—say, social media privileges (if you are a preteen), our 1960s-era LP-album collection, our car, home, job, and yes, our bonus paycheck—the prospect of losing it makes us pretty darn unhappy.
So back at the factory, which individuals and teams performed the best? Those who, like your fictional self, Lin Li, received the carrot letter? Or those like your fictional colleague Zi Peng, who received the stick letter? Before you venture a guess, ask yourself what motivates you more: “gain-framing” or “loss-framing”? And if you work on a team with other people, knowing that the performance of each member affects the entire team’s bonus, do you work harder under the reward or the punishment framing?
Here’s what we found: just having a bonus incentive in place improved productivity. The effect was in the neighborhood of 4 percent to 9 percent for workers in groups and 5 percent to 12 percent for individual workers. These are sizable effects, considering the magnitude of our bonuses. But, more interestingly, although individual workers were not influenced significantly by the loss frame, people who were working in groups increased their productivity by some 16 percent to 25 percent above the workers in the reward framing. And, guess what? Errors and defects didn’t rise.
Overall, we found that Wanlida could effectively use simple framing to increase overall team productivity.
Would these results eventually wane over time? Would the workers slow down or stop responding to the punishment incentive? The answer was “No.” Week after week, for six months, the punishment framing increased productivity.
Clearly, the fear of loss motivated the workers more than the prospect of gain. In other words, carrots may work better if they look a bit like sticks. But who wants to work for a company that gives employees this kind of dual-handed, carrot-and-stick treatment? Well, losses are a fact of life; someone has to bear them. We believe losses are a powerful motivator. Businesses have used the threat of layoffs or firing to encourage productivity, but outside of those large-scale threats, companies rarely use loss-framing.
Of course, if you are a manager, you don’t have to use incentives as devilishly designed as the ones used in this study. Remember: it has to do with framing. If you give workers a stake in their production and then focus on the losses that could come from their lack of production, you should achieve the effects described above, without scaring employees through manipulative incentives.
So What’s the Big Problem?
So why don’t businesses experiment more? A number of barriers make implementing experimentation in firms difficult. One barrier, as Scott Cook pointed out to us, is that the people in power like to hold onto their PowerPoints, and they don’t want the little guys pointing out that the emperor has no clothes, or that he might run his empire in a different way.
Another is sheer bureaucratic inertia. For example, in the summer of 2009, we recruited some students to help us with a field experiment on incentives in a large company. The company came over to San Diego to meet with us, explained the simple problem they were facing, and agreed to run the experiment within a couple of months. Four years later, the study is still buried somewhere in the big organization, waiting for management approval.
Other times, managers are intimidated by the uncertainty involved in a change and the unknown. Going the traditional route without introducing new methods is familiar, and as long as it works, it seems safer (“if it ain’t broke, don’t fix it”). Managers also feel they’ve been hired to provide solutions and make tough decisions to enhance the firm’s performance. In other words, they feel they are expected to have ready answers for the challenges the firm faces. Opting for experimentation may appear to imply that they don’t, and could compromise the appearance of their expertise—making it look as if they have failed to do their jobs.
One could overcome these barriers in two distinct ways: top-down and bottom-up. First, the company’s managing team would need to overcome the typical “short-term earnings first” mindset and encourage (and even reward) experimentation that could improve the firm’s performance, as Cook and McCallister have done. This approach requires hiring and training people to design and run experiments, analyze the data, and draw conclusions. Under a bottom-up approach, lower-level managers could conduct smaller-scale field studies and then present the results to the management, providing them with the costs and benefits associated with running the research.
Changing a tried—if not so true—mindset is no small feat. In the end, it takes a combination of fearless leadership, training, and hands-on experience to develop an experimental culture. If companies succeed at doing that, they can reshape their industries altogether.
We have seen too many top executives fall in love with their own ideas and then unleash these on an unsuspecting world, generating a big backlash, as Netflix (and other companies before and since) did. We’ve seen business leaders apply carrots and sticks in an effort to raise productivity, to no avail. We’ve seen companies try to figure out the right price for a product, without having any idea what it’s worth to consumers. These costly mistakes occur all the time, and they are utterly preventable.
By contrast, businesses small and large that do run field experiments are making more money and attracting more customers. Intuit has expanded its market by testing out small ideas, and expanding on the good ones. Humana found that by actively helping senior citizens with their prescriptions and self-care, older people could stay out of hospitals and the company could save millions of dollars in the process. A big technology company like Wanlida learned that offering employees a bonus and threatening to take it away raised productivity dramatically. A small vintner in northern California, experimenting with pricing for his wine, discovered that he had been charging half of what customers were willing to pay. And Disney learned that letting people pay what they wanted for a photo taken at the end of a ride worked especially well when half of their donations went to a charity.
The bottom line for business is this: Do you want to make more money? If yes, then run field experiments. Do you want to go down in the annals of great companies? If you do, then run field experiments.
No comments:
Post a Comment