Top 10 List for Big Data

The top marketing executive at a sizable US retailer recently found herself perplexed by the sales reports she was getting. A major competitor was steadily gaining market share across a range of profitable segments. Despite a counterpunch that combined online promotions with merchandizing improvements, her company kept losing ground.

When the executive convened a group of senior leaders to dig into the competitor's practices, they found that the challenge ran deeper than they had imagined. The competitor had made massive investments in its ability to collect, integrate, and analyze data from each store and every sales unit and had used this ability to run myriad real-world experiments. At the same time, it had linked this information to suppliers' databases, making it possible to adjust prices in real time, to reorder hot-selling items automatically, and to shift items from store to store easily. By constantly testing, bundling, synthesizing, and making information instantly available across the organization-from the store floor to the CFO's office-the rival company had become a different, far nimbler type of business.

What this executive team had witnessed first hand was the game-changing effects of big data. Of course, data characterized the information age from the start. It underpins processes that manage employees; it helps to track purchases and sales; and it offers clues about how customers will behave.

But over the last few years, the volume of data has exploded. In 15 of the US economy's 17 sectors, companies with more than 1,000 employees store, on average, over 235 terabytes of data-more data than is contained in the US Library of Congress. Reams of data still flow from financial transactions and customer interactions but also cascade in at unparalleled rates from new devices and multiple points along the value chain. Just think about what could be happening at your own company right now: sensors embedded in process machinery may be collecting operations data, while marketers scan social media or use location data from smartphones to understand teens' buying quirks. Data exchanges may be networking your supply chain partners, and employees could be swapping best practices on corporate wikis.

All of this new information is laden with implications for leaders and their enterprises. Emerging academic research suggests that companies that use data and business analytics to guide decision making are more productive and experience higher returns on equity than competitors that don't. That's consistent with research we've conducted showing that "networked organizations" can gain an edge by opening information conduits internally and by engaging customers and suppliers strategically through Web-based exchanges of information.

Over time, we believe big data may well become a new type of corporate asset that will cut across business units and function much as a powerful brand does, representing a key basis for competition. If that's right, companies need to start thinking in earnest about whether they are organized to exploit big data's potential and to manage the threats it can pose. Success will demand not only new skills but also new perspectives on how the era of big data could evolve-the widening circle of management practices it may affect and the foundation it represents for new, potentially disruptive business models.

Five big questions about big data

In the remainder of this article, we outline important ways big data could change competition: by transforming processes, altering corporate ecosystems, and facilitating innovation. We've organized the discussion around five questions we think all senior executives should be asking themselves today.

At the outset, we'll acknowledge that these are still early days for big data, which is evolving as a business concept in tandem with the underlying technologies. Nonetheless, we can identify big data's key elements. First, companies can now collect data across business units and, increasingly, even from partners and customers (some of this is truly big, some more granular and complex). Second, a flexible infrastructure can integrate information and scale up effectively to meet the surge. Finally, experiments, algorithms, and analytics can make sense of all this information. We also can identify organizations that are making data a core element of strategy. In the discussion that follows, we have assembled case studies of early movers in the big data realm (see "Seizing the potential of 'big data'" and the accompanying sidebar, "AstraZeneca's 'big data' partnership.")

In addition, we'd suggest that executives look to history for clues about what's coming next. Earlier waves of technology adoption, for example, show that productivity surges not only because companies adopt new technologies but also, more critically, because they can adapt their management practices and change their organizations to maximize the potential. We examined the possible impact of big data across a number of industries and found that while it will be important in every sector and function, some industries will realize benefits sooner because they are more ready to capitalize on data or have strong market incentives to do so (see sidebar, "Parsing the benefits: Not all industries are created equal").

The era of big data also could yield new management principles. In the early days of professionalized corporate management, leaders discovered that minimum efficient scale was a key determinant of competitive success. Likewise, future competitive benefits may accrue to companies that can not only capture more and better data but also use that data effectively at scale. We hope that by reflecting on such issues and the five questions that follow, executives will be better able to recognize how big data could upend assumptions behind their strategies, as well as the speed and scope of the change that's now under way.

1. What happens in a world of radical transparency, with data widely available?

As information becomes more readily accessible across sectors, it can threaten companies that have relied on proprietary data as a competitive asset. The real-estate industry, for example, trades on information asymmetries such as privileged access to transaction data and tightly held knowledge of the bid and ask behavior of buyers. Both require significant expense and effort to acquire. In recent years, however, online specialists in real-estate data and analytics have started to bypass agents, permitting buyers and sellers to exchange perspectives on the value of properties and creating parallel sources for real-estate data.

Beyond real estate, cost and pricing data are becoming more accessible across a spectrum of industries. Another swipe at proprietary information is the assembly by some companies of readily available satellite imagery that, when processed and analyzed, contains clues about competitors' physical facilities. These satellite sleuths glean insights into expansion plans or business constraints as revealed by facility capacity, shipping movements, and the like.

One big challenge is the fact that the mountains of data many companies are amassing often lurk in departmental "silos," such as R&D, engineering, manufacturing, or service operations-impeding timely exploitation. Information hoarding within business units also can be a problem: many financial institutions, for example, suffer from their own failure to share data among diverse lines of business, such as financial markets, money management, and lending. Often, that prevents these companies from forming a coherent view of individual customers or understanding links among financial markets.

Some manufacturers are attempting to pry open these departmental enclaves: they are integrating data from multiple systems, inviting collaboration among formerly walled-off functional units, and even seeking information from external suppliers and customers to cocreate products. In advanced-manufacturing sectors such as automotive, for example, suppliers from around the world make thousands of components. More integrated data platforms now allow companies and their supply chain partners to collaborate during the design phase-a crucial determinant of final manufacturing costs.

2. If you could test all of your decisions, how would that change the way you compete?

Big data ushers in the possibility of a fundamentally different type of decision making. Using controlled experiments, companies can test hypotheses and analyze results to guide investment decisions and operational changes. In effect, experimentation can help managers distinguish causation from mere correlation, thus reducing the variability of outcomes while improving financial and product performance.

Robust experimentation can take many forms. Leading online companies, for example, are continuous testers. In some cases, they allocate a set portion of their Web page views to conduct experiments that reveal what factors drive higher user engagement or promote sales. Companies selling physical goods also use experiments to aid decisions, but big data can push this approach to a new level. McDonald's, for example, has equipped some stores with devices that gather operational data as they track customer interactions, traffic in stores, and ordering patterns. Researchers can model the impact of variations in menus, restaurant designs, and training, among other things, on productivity and sales.

Where such controlled experiments aren't feasible, companies can use "natural" experiments to identify the sources of variability in performance. One government organization, for instance, collected data on multiple groups of employees doing similar work at different sites. Simply making the data available spurred lagging workers to improve their performance.

Leading retailers, meanwhile, are monitoring the in-store movements of customers, as well as how they interact with products. These retailers combine such rich data feeds with transaction records and conduct experiments to guide choices about which products to carry, where to place them, and how and when to adjust prices. Methods such as these helped one leading retailer to reduce the number of items it stocked by 17 percent, while raising the mix of higher-margin private-label goods-with no loss of market share.

3. How would your business change if you used big data for widespread, real-time customization?

Customer-facing companies have long used data to segment and target customers. Big data permits a major step beyond what until recently was considered state of the art, by making real-time personalization possible. A next-generation retailer will be able to track the behavior of individual customers from Internet click streams, update their preferences, and model their likely behavior in real time. They will then be able to recognize when customers are nearing a purchase decision and nudge the transaction to completion by bundling preferred products, offered with reward program savings. This real-time targeting, which would also leverage data from the retailer's multitier membership rewards program, will increase purchases of higher-margin products by its most valuable customers.

Retailing is an obvious place for data-driven customization because the volume and quality of data available from Internet purchases, social-network conversations, and, more recently, location-specific smartphone interactions have mushroomed. But other sectors, too, can benefit from new applications of data, along with the growing sophistication of analytical tools for dividing customers into more revealing microsegments.

One personal-line insurer, for example, tailors insurance policies for each customer, using fine-grained, constantly updated profiles of customer risk, changes in wealth, home asset value, and other data inputs. Utilities that harvest and analyze data on customer segments can markedly change patterns of power usage. Finally, HR departments that more finely segment employees by task and performance are beginning to change work conditions and implement incentives that improve both satisfaction and productivity.

4. How can big data augment or even replace management?

Big data expands the operational space for algorithms and machine-mediated analysis. At some manufacturers, for example, algorithms analyze sensor data from production lines, creating self-regulating processes that cut waste, avoid costly (and sometimes dangerous) human interventions, and ultimately lift output. In advanced, "digital" oil fields, instruments constantly read data on wellhead conditions, pipelines, and mechanical systems. That information is analyzed by clusters of computers, which feed their results to real-time operations centers that adjust oil flows to optimize production and minimize downtimes. One major oil company has cut operating and staffing costs by 10 to 25 percent while increasing production by 5 percent.

Products ranging from copiers to jet engines can now generate data streams that track their usage. Manufacturers can analyze the incoming data and, in some cases, automatically remedy software glitches or dispatch service representatives for repairs. Some enterprise computer hardware vendors are gathering and analyzing such data to schedule preemptive repairs before failures disrupt customers' operations. The data can also be used to implement product changes that prevent future problems or to provide customer use inputs that inform next-generation offerings.

Some retailers are also at the forefront of using automated big data analysis: they use "sentiment analysis" techniques to mine the huge streams of data now generated by consumers using various types of social media, gauge responses to new marketing campaigns in real time, and adjust strategies accordingly. Sometimes these methods cut weeks from the normal feedback and modification cycle.

But retailers aren't alone. One global beverage company integrates daily weather forecast data from an outside partner into its demand and inventory-planning processes. By analyzing three data points-temperatures, rainfall levels, and the number of hours of sunshine on a given day-the company cut its inventory levels while improving its forecasting accuracy by about 5 percent in a key European market.

The bottom line is improved performance, better risk management, and the ability to unearth insights that would otherwise remain hidden. As the price of sensors, communications devices, and analytic software continues to fall, more and more companies will be joining this managerial revolution.

5. Could you create a new business model based on data?

Big data is spawning new categories of companies that embrace information-driven business models. Many of these businesses play intermediary roles in value chains where they find themselves generating valuable "exhaust data" produced by business transactions. One transport company, for example, recognized that in the course of doing business, it was collecting vast amounts of information on global product shipments. Sensing opportunity, it created a unit that sells the data to supplement business and economic forecasts.

Another global company learned so much from analyzing its own data as part of a manufacturing turnaround that it decided to create a business to do similar work for other firms. Now the company aggregates shop floor and supply chain data for a number of manufacturing customers and sells software tools to improve their performance. This service business now outperforms the company's manufacturing one.

Big data also is turbocharging the ranks of data aggregators, which combine and analyze information from multiple sources to generate insights for clients. In health care, for example, a number of new entrants are integrating clinical, payment, public-health, and behavioral data to develop more robust illness profiles that help clients manage costs and improve treatments.

And with pricing data proliferating on the Web and elsewhere, entrepreneurs are offering price comparison services that automatically compile information across millions of products. Such comparisons can be a disruptive force from a retailer's perspective but have created substantial value for consumers. Studies show that those who use the services save an average of 10 percent-a sizable shift in value.

Confronting complications

Up to this point, we have emphasized the strategic opportunities big data presents, but leaders must also consider a set of complications. Talent is one of them. In the United States alone, our research shows, the demand for people with the deep analytical skills in big data (including machine learning and advanced statistical analysis) could outstrip current projections of supply by 50 to 60 percent. By 2018, as many as 140,000 to 190,000 additional specialists may be required. Also needed: an additional 1.5 million managers and analysts with a sharp understanding of how big data can be applied. Companies must step up their recruitment and retention programs, while making substantial investments in the education and training of key data personnel.

The greater access to personal information that big data often demands will place a spotlight on another tension, between privacy and convenience. Our research, for example, shows that consumers capture a large part of the economic surplus that big data generates: lower prices, a better alignment of products with consumer needs, and lifestyle improvements that range from better health to more fluid social interactions. As a larger amount of data on the buying preferences, health, and finances of individuals is collected, however, privacy concerns will grow.

That's true for data security as well. The trends we've described often go hand in hand with more open access to information, new devices for gathering it, and cloud computing to support big data's weighty storage and analytical needs. The implication is that IT architectures will become more integrated and outward facing and will pose greater risks to data security and intellectual property. For some ideas on how leaders should respond, see " Meeting the cybersecurity challenge."

Although corporate leaders will focus most of their attention on big data's implications for their own organizations, the mosaic of company-level opportunities we have surveyed also has broader economic implications. In health care, government services, retailing, and manufacturing, our research suggests, big data could improve productivity by 0.5 to 1 percent annually. In these sectors globally, it could produce hundreds of billions of dollars and euros in new value.

In fact, big data may ultimately be a key factor in how nations, not just companies, compete and prosper. Certainly, these techniques offer glimmers of hope to a global economy struggling to find a path toward more rapid growth. Through investments and forward-looking policies, company leaders and their counterparts in government can capitalize on big data instead of being blindsided by it.

About the authors

Brad Brown is a director in McKinsey's New York office; Michael Chui is a senior fellow with the McKinsey Global Institute (MGI) and is based in the San Francisco office; James Manyika is a director of MGI and a director in the San Francisco office.

In the last few decades, statisticians and computer scientists have produced a dazzling arsenal of extremely powerful tools to help managers translate data into business decisions.

Having access to a wide array of versatile solutions is not ordinarily considered a problem in the world of business. But the rise of "big data" has also brought along with it the explosion of mathematical models made possible by today's low-cost computing and storage platforms. Ironically, this poses a number of substantial challenges to managers trying to making sense of ever-growing quantities of information.

For example, says, Wharton PhD student Eric Schwartz, managers may be tempted to, as he put it, "flex their data-science muscles" and use a statistical model that is simply too complicated for the task at hand. The result, he notes, might well be that the model produces bad advice.

Alternately, managers may waste time trying to figure out which of several dozen possible models would be the most precise fit for the data set they are using. But the time it takes to play statistical guessing games after the analyses would be better spent running their businesses, Schwartz says.

"Wouldn't it be nice to be able to know, just from looking at the data, how complicated a tool you should use with it?" he asks. Such a "recipe," Schwartz adds, would have two benefits: It would allow managers to pick the "golden model", one that was neither too complicated nor too simple. And it would let them do so quickly, before having to undertake a lot of the more complicated analytical work.

That was the genesis of " Model Selection Using Database Characteristics: Classification Methods and an Application to the 'HMM and Its Children,'" which is currently under review in the premier academic marketing journal. (The "HMM" in the title stands for "Hidden Markov Model," a widely-used statistical modeling technique.) Schwartz's collaborators on the paper are Wharton marketing professors Eric Bradlow and Peter Fader, his dissertation advisors, who are also co-directors of the Wharton Customer Analytics Initiative.

"No one in the real business world has time to run a bunch of different models on a data set to see which one is best," Bradlow says. "We've come up with a way of picking the winner that is quite sophisticated in its science but quite simple in its practical application."

Some background for non-statisticians: A data set or a database can be anything from a retail chain's sales figures to the donor list from a charitable organization. Managers consult a data set when they need to make a decision, like whether a product should be discounted or whether a group of customers should be targeted with a special promotion.

The complexity of data sets has grown in parallel with progress in the development of tools to extract information from them. Lately, as computers have become more powerful, the number and sophistication of those modeling tools have skyrocketed. Some are simple functions built into every copy of Microsoft Excel. More complicated are the Hidden Markov Models of the paper's title, which require their own computer program.

For their paper, Schwartz, Bradlow and Fader took 64 different data sets that were representative of numerous real-world situations, from retailing, online media and elsewhere. With each one, they then ran four different models ranging in complexity from the simplest - a "Beta-Geometric Beta Bernoulli" model, which can be run in Excel and on an average laptop - to the most complicated, a full-blown Hidden Markov Model, which generally requires the use of a specialized programming language and takes much longer to run.

A typical data set might have two years' worth of information. For each of the 256 (4 X 64) variations of data sets and models, the computer was fed, for example, the first year of data, and then told to use the model to predict the numbers for the second year. The results were scored and sorted by accuracy. In all, the number-crunching required 24,000 hours of computing time if a single CPU was used. But because it was run in parallel across many different machines on Amazon's Elastic Computing Cloud (EC2), in connection with a research grant from the company, the job took just two days to complete. "Without Amazon's generosity, we would be a few years older by the time the analysis was completed," Bradlow says. "However, with the use of cloud computing, this shrinks down to two days. Amazon's investment in our work, the first major use of the cloud for large-scale marketing academic purposes, is not to be underestimated."

The different permutations of data sets and business models sound like something requiring a massive database just to keep track of. But the major surprise of the paper - indeed, the reason the authors believe they can help lighten the workload of every manager who works with data - is the great deal of simplicity beneath all of the apparent complexity.

Specifically, the data sets ended up clustering into a small number of different groups, each with easily identifiable characteristics. In one group - for example, talking about a retailer's sales - the database would be characterized by a steep decline in aggregate sales over time. This suggests a model in which customers might buy a product a few times, but then stop purchasing it altogether. Another group of databases can be characterized by a slight decline in aggregate sales without an extreme purchase concentration (e.g., less than 80% of sales come from the top 20% of customers). This suggests customers may keep switching back and forth between being frequent and enthusiastic purchasers and being less active buyers.

The good news was that each of those patterns can be linked with one of the four models tested by the researchers. In business life, it's relatively easy for managers to have an intuitive sense of which of the four main groups their data sets belongs to; but in this work, that information can be gleaned by looking at a simple chart. Armed with that knowledge, managers will now be able to use the results from the paper to pick with confidence which statistical model is the most appropriate.

"When people who work with data read our paper, I want them to think to themselves, 'Wow, I didn't realize that just by quickly summarizing my raw data, I could figure out which tool from my analytical toolbox was the right one,'" Schwartz notes.

The decision process involved in actually matching a particular data set with a particular model is described fully in the paper, and is easily accessible to someone with a basic background in statistics, Schwartz adds.

Picking the right model for a given data set has significant implications for business decisions. Schwartz says that choosing the wrong model can degrade accuracy of sales forecasts or behavioral targeting. The other bit of peace of mind delivered by the paper is that managers with relatively simple data sets can choose relatively simple modeling tools without worrying that they might be missing something in their analysis.

"There are some people who spend too much time worrying about complex models without thinking about the business value," Bradlow notes. "And there are some people who don't worry about it at all, but who should, because their fundamental business value depends on it."

Knowledge@Wharton: George and Dave, thanks for joining us. George, your paper is called "Closing the Marketing Capabilities Gap." What exactly is that gap and why is it so important that companies address it?

George Day: As I reflect on the title, I am beginning to think a better title would be "Narrowing the Marketing Capabilities Gap." The gap is a consequence of two things. One is what you might call a data deluge, the information explosion, where we have a doubling of the amount of data - not information, necessarily, or wisdom - but the amount of data stored every 18 months. This is an exponential increase in the data available to companies. At the same time, our best guess on the ability of companies to actually use this data ... is that it is growing at about two percent a year. That's the gap I'm looking at here. So we're overwhelmed, we're confused by the consequences of a term I love - which is the "splinternet" - [meaning] the fragmenting media, and by the tremendous decomposition of mass markets into little niches. And, of course, media choices are proliferating rapidly. It's overwhelming companies and also individuals. We're all struggling with this tremendous overload.

Knowledge@Wharton: Dave, what are the traps that companies fall into trying to manage all this data?

David Reibstein: One of the things that I think is really interesting is thinking about the history of how this gap happened. What George was just referring to, which is absolutely right, is this huge proliferation of choices that customers have, options that companies have, and then the abundance of data. But let's back up just a little bit and think about who was in marketing and who has been doing this in the past. Much of it has been people who have risen up through sales and have a sales orientation. They were very good interpersonally, very good at working and talking to customers. Or they were people who could identify and feel what the customer felt. But what's happened is we don't need to rely on the "pearls of the tongue," or the empathy that the marketer needs. We have information.

So the new marketer today is very different from the marketer of the past. I think one of the traps is falling into doing things the old way and trying to unlearn the way that we used to do marketing. Today it is so much more data driven.

Knowledge@Wharton: There are many new developments in knowledge sharing technology, like micro-targeting and communities of users and all the social networking tools. What are the best examples of these technologies and who is using them in the most advantageous ways?

Day: Let me set the frame for a broader way of thinking about how companies cope, and then see how they're able to take these technologies. I have two ideas. One is that technology is creating the problem. It's also going to be the solution to the problem. So social networking, knowledge sharing, mini-Googles that companies are creating for their own data bases are obviously the paths to the solution. But they will only work if the organization is inclined to use them and is prepared to use them.

So I am looking at three different capabilities that companies will need in order to effectively use all this new technology for sorting through, sharing and interpreting data. The first one is what I call "vigilant market learning." That is the ability to see things sooner, to capture the right information, as opposed to treating all information as equally important. It is sorting out which signals you really should pay attention to. That's a significant capability - it requires a [certain kind of] leadership, highly networked and so forth. Second is the notion of adaptive market experimentation. This is a capability companies have to master. And where it's B to B [business to business], B to C [business to consumers], it's companies like Quicken doing 600 experiments every year with its customers, and learning systematically from it. So we have a lot of interesting tools on predicted analytics and so forth that our colleagues Peter Fader and Eric Bradlow are working on here to try to unlock the lessons from those experiments.

But the other thing I think is important about companies that use these tools well is they don't try to do it all themselves. What I've been fascinated by is what I'm now calling "open marketing," which is a direct lift from "open innovation." This is the notion that if you want to do search engine optimization or predictive analytics, it's probably not a good idea to try to build that capability in-house. Rather, you find a really good partner who is able to help you with this, to apply this - and has the right kind of talent. The reason you have to partner in part is that the talent is very hard to find. In an era of high unemployment, there's a desperate need for qualified, creative, analytical people - kind of the two ideas coming together. But we have a talent crisis, and companies are struggling to find people to do that. The answer, I think, is to find the right partners.

Knowledge@Wharton: Who are these partners? Are they other companies? Are they individuals that you hire?

Day: There are at least 12 kinds of partners, anywhere from specialists ... all the way to one of the big three integrated advertising agencies which have a host of creative shops, marketing research firms with the specialized talent built into them that they can spread around and build up experience in lots of different situations.

Reibstein: So you asked, Robbie, specifically who's doing a good job of trying to use the new tools and trying to take advantage of them. George mentioned Quicken. Let me add to that. I would say Amazon falls in that category. They're constantly experimenting, and the web page that you get and the web page that I get might be very, very different - not just based on our history, but because we're coming in and they're going to try something different. It's a whole test-and-learn mentality. I'm going to test something out, see what works, deploy it. And that could happen all within the same day. In fact, you could be doing hundreds of these every day. Say I buy a blue sweater and you buy a blue sweater. You come back and they're going to give you a yellow sweater and see if you [go for] it. That's what it is that is being featured. And for me they're going to give me - what would go well with a blue sweater? Black slacks. And they're going to see which works better. Do that quickly with a couple thousand people, learn from that, deploy.

Knowledge@Wharton: In fact, in your paper, George, you quote [Amazon CEO] Jeff Bezos as saying, "Rather than ask what we're good at and what else we can do with that skill, ask who are your customers, what do they need? And then say we're going to give that to them, regardless of whether we have the skills to do so. And we will learn those skills." But Amazon has the luxury of being able to do these millions of tests to find out what consumers need, even before consumers themselves know what they need. Other companies don't have that capability.

Reibstein: There's an interesting property that Amazon has and other companies have that are taking advantage of this ability - which is dealing with customers individually. So there are lots of companies that sell through distributors, who sell to other distributors and are not really in touch with the end customer. Amazon is in touch with exactly who their ultimate customer is. Certainly online businesses have that ability. But actually, if you think about it, cellular companies have that ability as well, because they know exactly who that individual customer is. Banks have that capability. [They are] companies that have the individual level data, which takes us back to the earlier problem - this abundance of data. The question is, who's able to really take advantage of it.

Let me go to the other extreme, which is to take a company like Unilever. Unilever sells through ... distributors to retailers or directly to retailers, who end up then selling to consumers. They don't have that individual consumer level data.

But what they have started doing, and doing very effectively, is working on a lot of social media. The whole Dove campaign that got tons of publicity was something that was primarily created online. They created ads that were never shown on broadcast television. It was really by spreading [the message] through YouTube and through other social media that [they were able to] reach a massive audience. It has been a very clever [strategy] for a company that doesn't have that individual level data.

Knowledge@Wharton: George, there's a follow-up question to that. So consumers are obviously increasingly vocal, chatty - they get online and they're constantly spreading their opinions about products and services. How can companies figure out which comments are important and need immediate action, and which are just noise?

Day: There's a lot of interesting technology to, first of all, capture these blog comments, and then edit them and see if there are patterns. So if you go on to Google, you can see patterns of recurring use of a term. What you're looking for is literally clearing out all of the noise and concentrating on what's the signal that we really want to look at. It might be a complaint about product performance. I remember one of the very earliest ones was an experience Procter & Gamble had as it brought out - remember Febreze?

Knowledge@Wharton: Yes.

Day: Everybody knows Febreze. It almost failed because, early in its launch, a woman got on, blogged and said, "Febreze killed both of my canaries." This spread through the Internet. P&G had a very good way of capturing that. They figured out who she was, because they sent an e-mail saying, "What can we do?" They rushed a team out, talked to her and after a while discovered - and she agreed - that the canaries just died of old age. This meant they were able to stop that really threatening message going out over the toxic effects of Febreze. They actually turned it into an opportunity on the Internet. And it's that ability to capture these signals, understand them and act on them quickly. But we've got lots of examples of blog messages that have been ignored for way too long and just go viral. It doesn't take very long.

Reibstein: Which takes me back to your question. I want to build on what George said related to your question. You said, "How do you determine which ones to respond to?" I think part of the message ends up becoming, "Every customer becomes important," in part because, today, every customer has a microphone, or a megaphone, I guess I should say, because it is easier to broadcast your message. You could have said, "There's not a massive number of people claiming their canaries are dying from Febreze. Let's ignore it." But, as George points out, it goes viral and suddenly it starts becoming this big publicity.... If we ignore it, then it could be very, very dangerous. So every customer starts becoming important, and you need to be addressing all of them. That requires a whole different skill set.

Day: So this gets back to the issue, "How do you build the skill set, the technologies?" Most companies would be ill-advised to try to do it themselves. It's an extremely sophisticated kind of analysis. That's where partners come in. Finding the right partners, building a relationship with them - those are difficult management challenges. The good companies will do it.

Broadly, this leads us into the question of how do you then think about this marketing capabilities gap? The answer is, most companies are not going to be able to close it. All their aspiration should be is to close it faster than their competitors. Get ahead. Line up the right partners. Build a capability to do these kinds of test-and-learn experiments. Stay ahead of the competition. By the way, just a side-bar to this: One of the things that I'm fascinated by is that there are only so many good partners out there. I think what we're going to see in this open marketing arena is that the smart companies are going to get there first. They're going to lock up the good partners, and the later entrants who say, "Oh, this is a good idea," won't have the pick of the partners.

Knowledge@Wharton: Should we worry about consumers' short attention spans? I mean, they're trying to absorb these thousands of messages thrown at them. At some point, consumers just tune you out. How do companies avoid that? How do they avoid being ignored by potential customers?

Reibstein: That is always a threat. It's been a threat for a long time. But because we have so many different channels to try and reach customers, it's become more of a threat. The responsiveness to banner ads is down, just because the number of them has gone up. I think part of the answer goes back to really customizing the message in your communications to the individual customer so that what you're saying resonates with them. Historically, we're going to post it out there for a large number of people. Our message comes from the history again of mass media. We're going to put out a mass message versus talking specifically to Robbie, or talking specifically to individuals about what it is they might need.

Day: But at the extreme, not only is the message tailored, but the whole product offering.

Reibstein: Absolutely.

Day: So you get exactly what you want. Now customers will pay attention because they understand that you're solving precisely their problem. Why does Zappos do so well in an otherwise crowded category? Because they are very good at customer service, and they give the customers exactly what they want. Customers can solve their problems very easily on Zappos.

Knowledge@Wharton: Right. Weren't they one of the first to offer free shipping back and forth?

Day: Yes. So you can order three pairs of shoes to get the one that you really want to keep.

Knowledge@Wharton: Exactly. And not pay any penalty. George, you sent me a study from IBM called "From Stretched to Strengthened," in which IBM talked to more than 1700 chief marketing officers worldwide. At one point, the study advises CMOs to understand individuals as well as markets, to establish customer intimacy, to focus on relationships and not just transactions. How do you do that in such a huge global online marketplace?

Day: That's where technology becomes our friend. We can build deep insights into customers, if they choose to let us. They may not. They may want a transactional relationship. But by and large - let's just take Zappos because it's a very familiar example - Zappos knows a lot about its customers and it reaches out to them. Let's take another company which I think has done a masterful job of addressing that, and that's Tesco, with their Club Card in the U.K. They have 14 million customers who have given Tesco permission to capture all the data about every transaction, all their demographics, background information. So Tesco knows an enormous amount about these people. It can then tailor messages precisely to their geographic area and to them individually. So a newly wed couple gets a certain set of offerings. A family with a new baby gets offers.

Knowledge@Wharton: But at some point, don't you think customers are going to stop giving that permission?

Reibstein: There's a concern about too much intrusion. "I want to guard my privacy." I think we're going to go through some waves of people saying, "No, I want to be private" and other people saying, "Wait a minute. I'm being better served by companies knowing some information about me." My belief is that the only people who should receive coupons for baby diapers are people with babies. Right? You could go through the whole list of the only people who are going to get specific offers are ones who are in need of it. We see the phenomenon of Groupon going on right now. The number of promotions that I get for Brazilian blow-outs is just amazing. And for tooth-whitening and for things that I'm not particularly interested in right now. That's just wasted messaging going on. I wish they knew more about me so that I wouldn't get all the noise.

Knowledge@Wharton: So would you give them that information so they could, in fact, learn more about you?

Reibstein: As I said, we're going to see these waves of people discovering, "I want to be private, but on the other hand I am much better served, I get much less of this noise if people know more about me." I think there's going to be a bifurcation of those who say, "Bring it on." And others who say, "I want to protect." Those who are asking for protection of their information are going to be observing the others. I think we will see an evolution towards more disclosure, not less disclosure.

Day: Just to put a wrapper around this, what's a relationship? It's a perception of mutual understanding and mutual benefits. So customers will only give the company permission if they think it's in their best interests.

Knowledge@Wharton: So they have control over this?

Day: They have control. Why does Tesco get permission to collect all this information and to use it to target, as Dave was suggesting, very specific messages? The answer is a rebate. What's neat about Tesco is that every quarter, [customers] get a rebate based upon a percentage of the value of the purchases they have made in that quarter. That's a reminder that this is a very good two-way relationship.

Knowledge@Wharton: I have two more questions. Dave, the IBM study also quotes CMOs saying that return on marketing investment is the primary measure of effectiveness. This is nothing new. It's something you've talked about a lot. But how do you prove that value? How do you come up with some numbers? Or are numbers not the answer?

Reibstein: People are stuck on this return on marketing investment. They do that, in part, because it integrates with the rest of the financial communications of the company because you've got an ROI that is measured in all sorts of investments. So it's a basis for a comparisoning across. But I think what we need to do - and what our current accounting systems are not very good at doing right now - is capture the long-term contributions of marketing. So we have things like a brand, which only shows up on our books if we acquire a brand. If we build it, it doesn't show up officially on our books. But that doesn't mean that we can't measure it and can't measure what the value of that particular brand is. Again, there are partners. There are various companies that assist in helping with that particular measurement.

But maybe even better than brand is looking at the long-term value of a customer. That's not saying, "Just what is it that they bought right now and how much should I make in that transaction?" But [rather], "I want to think about that relationship - that relationship I have with that customer that is today and hopefully out into the future." So when someone buys a BMW, the value of the customer who bought the BWM is not what I just sell him right now and the service that he is going to have, but it's also what is the probability that he is going to buy a BMW out into the future? What companies are going to move more towards looking at that longer term? I raise this in the context - I haven't forgotten your question, the return on marketing investment which is sort of looking backwards. "How much did I make from that?" It's not looking enough forward towards that future revenue stream that's going to come by having gotten this customer, solidified my relationship, increased the probability that [he or she] is going to be buying out into the future.

Knowledge@Wharton: George, I'll throw this out at you. The IBM report at one point uses a very strong image. It says, "Just as X-rays transformed medicine by letting doctors see through human tissue, so the new information and communication technologies are revolutionizing business by letting customers and citizens peer through corporate walls." What is the one single best thing a company can do to turn consumers' ability to see through corporate walls into a competitive advantage?

Day: To essentially create transparency, to focus largely on the building of relationships based upon mutual benefits, mutual understanding. The company has to really invest to understand those customers and then create offers that are exactly tailored to their requirements, so that they feel they are being listened to and that their [needs] are being met. This is this notion of mutual commitment based upon mutual rewards. I don't mind you getting a benefit from this, but it's got to be worth my while.

Knowledge@Wharton: Thank you both for coming.