Big data can blind us to the long term

FT.com

December 9, 2013 4:52 pm

By Andrew Hill
The numbers look good but without technical expertise and intuition do not add up to much


Before long, “everything that computes will connect, and everything that connects will compute”, Abhi Ingle, who spearheads innovation at AT&T, told last week’s FT Innovate America conference in Silicon Valley.

As connected networks yield their riches, so companies are testing changes to the design, line-up and nature of their products, based on rapid, accurate and relatively inexpensive experiments. Farewell, guesswork and gut instinct; hello, big data and empirical proof.

I have warned about the danger that boardrooms will catch the big data bug and assign big teams and bloated budgets to it. But there is another pitfall: in interrogating the data for an answer to their questions about products, marketing and even strategy, chief executives may favour the search for short-term, numbers-driven answers over the messier quest for better, longer-term solutions.

Susan Athey of Stanford’s Graduate School of Business calls this tendency “easy-to-measurism”. Prof Athey, who won the John Bates Clark Medal for US economists in 2007 and also consults for Microsoft as the group’s chief economist, is rightly excited by big data’s “potential for [encouraging] vastly better decision-making”. She practises what she preaches, using data to analyse seemingly intractable microeconomic questions, such as the impact of online social media on the type of news people read and news organisations produce.

But, she told me last week, “you can still make some really fundamental and basic mistakes by getting seduced by either the rigour of an experiment that misses some things, or the complexity of a machine-learning algorithm that seems to be doing amazing things, but has some blind spots”.

This is an algorithmic mutation of the plague of distorted incentives and short-term priorities that have blighted companies since stock options and shareholder valuetook hold in the 1980s. Big-data analysis is a tool with greater promise than either of those concepts. But an engineer considering two legitimate data experiments – one with a quick pay-off and the other requiring analysis of the longer-range implications for customer behaviour – could still opt for the former to please the boss. Worse, he may not even propose the latter, closing off what could be a route to innovation that would be more pleasing for customers and more profitable for the company.

There are three ways of vaccinating companies against this infection.

The first, and hardest, is education. If schools promote computer and statistical literacy, eventually even executives who do not pursue such courses at university should reach the boardroom with at least a basic understanding of the language of big data.

The second is communication. Prof Athey was an avid mathematician from an early age, but even she confesses she found the statistics courses she took deeply dull, because they made no connection between theory and practice. “In business meetings, people aren’t spending hours writing statistics on the chalkboard,” she points out. “They’re discussing their application.” Managers who combine technical knowledge with the business intuition to understand where data fall short, and the communication skills to explain how to apply the results, are currently so rare that Prof Athey has described them as “rock stars” – and they command salaries to match.

The final and critical ingredient is that companies should focus on the benefit of data analysis for the user. It sounds obvious, but each time I visit Silicon Valley I am struck by how companies stress that the biggest incentives for engineers and scientists to join and stay are the complexity and importance of the challenges they will encounter. Big data help meet those challenges, which is why Google is currently the biggest stage for technology’s “rock stars”, but customers keep Google in business. As Sunil Chandra, who heads the company’s hiring team, told Innovate America, it is still important for Google to remember that “our products aren’t used by engineers, even though they are built by engineers”.

Prof Athey calls the promise of big data “awe-inspiring”. It is. But managers who become too beguiled by easy-to-measurism will eventually find themselves on the wrong side of a simple formula: a bright idea minus long-term user satisfaction equals failure.

andrew.hill@ft.com

Twitter: @andrewtghill