ROUNDTABLE

Are some types of marketing activities more accountable than others?

Marilyn Franck: Marketing activities that are meant to influence and change behavior are much harder to measure. For example, the impact of a loyalty program can be hard to measure because of its long time horizon.

With advertising, while you can measure awareness and perception changes, nobody's found the magic bullet to tie that specifically to revenue or share changes. There is nobody who can tell you with absolute certainty that a specific percentage change in awareness is going to drive a specific percentage change in revenue for a brand.

Scott Relf: The easiest marketing activities to measure are those that are the smallest and shortest in duration. It's easiest to optimize the little stuff and hardest to optimize the really big stuff.

A company might spend several million dollars on a handful of rebate-type promotions, and another 50 or 100 million dollars on television advertising. It's much easier to measure the rebates. So, you really have to force yourself to figure out how to measure the big, long-term investments effectively and stick with it long enough to measure and adjust this spending, as needed.

Sunil Garga: Certain types of multicultural efforts -- say Hispanic marketing -- have been harder to read because companies, by definition, execute them in smaller geographies, across smaller sets of products.

However, you also need to be careful about short-term measurement. Gauging the success of a two-week promotion or coupon offer may be valid, but trying to measure ROI from a two-week advertising campaign is problematic, since ad campaigns tend to have longer-term visibility or branding goals in mind.

Pricing is an area where great strides have been made. Marketers have been able to assess the impact pricing changes might have on sales, as well as in terms of attitudes towards the product or purchase intent.

"Over the next 24 months, more change will happen in the marketing R.O.I. arena than in the last 20 years." SUNIL GARGA

Mike Duffy: We have actually held off measuring some marketing programs within Kraft because we know that premature measurement can kill a good idea before it has a chance to take hold. A good example is the Internet. Four or five years ago, we were afraid of measuring systems that did not capture all the benefits that the Internet provides. Now we understand that that may not have been a good decision.

Our measurement systems are doing a fairly decent job in fairly broad areas including mass communications, advertising, consumer promotion, sweepstakes, coupons, refunds, trade promotions, displays and circulars. But, we still find we're much more comfortable measuring the relative performance of our activity -- it's better this year than last, or it's more efficient on this brand than that brand.

What have you learned about the effectiveness of integrated, offline-online marketing programs?

Franck: At Universal, we've seen enough to know that there's great value in integrated programs. When we advertise and have a direct call to action to our Web site, Web activity and online sales increase dramatically.

When we did a Halloween Horror Night down in Florida, we had a basic brand message out about it while using the Web and other tools to inform consumers about the program, giving them a visual glimpse and letting them know when the program sold out.

The Web also allowed us to time-shift interested attendees to other days so we could expand total attendance, without affecting the quality of the guest's experience.

Duffy: We're learning more even as we speak. Kraft wants to make it easy for our consumers to contact us in whatever way they prefer, not try to force them online or to see us o n TV. We're trying to better integrate our call center, Web site and direct marketing activities.

One of the more interesting things we've done recently was to launch a magazine, called Food & Family. The magazine offers solutions -- as opposed to just telling consumers about our product benefits. The issues are seasonal and focus on family topics, like what to feed the kids when they're off for the summer. It's a direct communications vehicle to our key mom-and-dad audiences, in which we build-in Kraft products as integral components of family meal solutions.

We identify households that are buying Kraft products and send them the magazine. With the help of Nielsen and IRI, we can gather sales information from stores in our readers' area and gain a quick, accurate read on sales lift -- just nine days after issue distribution.

Relf: Initially, our view was that the Web was its own, unique, place in the world, almost another planet. But there did come a time last year where it became clear that our customers were seeing the Web as being clearly aligned with the rest of Sprint. They wanted to see consistency between our stores, our Web site and our call centers.

From a pure headcount basis, for every person who actually consummates a purchase online with Sprint, there are ten or twenty who take that information and go to one of our stores -- or one of our retail partners like Best Buy -- to buy the phone. Those folks want the information and the offers to be consistent. The last thing that a customer wants is to go to the Web site, see a special offer or sales price on a piece of equipment, only to learn at the retail location that they can't get the product or the offer.

Garga: Traditional and online media can work very well together. If I'm a food products company or detergent maker, a large part of my Web marketing efforts may revolve around providing valuable information about the products -- say recipes or new product uses.

An apparel retailer, for example, might send an e-mail message to their whole customer database or advertise on their Web site that jeans are five bucks off. And if you print out a coupon and bring it to the store, you'll get a five-dollar discount. You have the option of redeemin

"I like to think of new media as being vehicles that are under our control as opposed to the Internet, a magazine, or some other new smokestack." MIKE DUFFY

g the coupon online or at the store.

How will the advent of new media change the analysis of ROI?

Relf: The biggest challenge may be to figure out the diagnostic information. What do I change about the new media to improve it? If the verdict comes back that it's working, how do I make it work better? If it comes back that it's falling short, how do I turn it around?

The treatment of the patient is a lot harder than the diagnosis. And to some degree that requires trial and error by the people making the decisions, to hone their expertise and to create the fodder for correlation analyses to work.

Duffy: I like to think of new or emerging media as being vehicles that are under our control as opposed to the Internet, a magazine, or some other new smokestack. Again, Kraft's magazine, Food & Family, satisfies that definition. We know who reads it, we have a general sense of where they live and we know roughly where they shop. But, perhaps best of all, just nine days after we drop each issue, we get a fairly clear read of any sales lift for Kraft's products that may have resulted.

We're turning the traditional longer-lead ROI measurement model on its head. We don't have to wait for data to accumulate or rely on a six-month or even once-a-year snapshot. Instead, we've created a very powerful means for providing ROI measurement on a nearly real-time basis.

Franck: New media inroads are helping spark proliferation in the number and nature of consumer touch points. We're seeing consumers take a far more active role in their relationships with brands. We're seeing changes in the depth and frequency of the potential contact that consumers have with us. A great example is what Amazon has done with personalizing offers.

CRM has become a more powerful tool because of the low costs of touching consumers. We're also experiencing a rise in powerful analytic tools, which give us a stronger link to different information sources. We now need to look at quick-stream data and link it to behavioral data, demographic data, sources of awareness and offers made.

With the mounting types of data available, you can literally drown in all this information. The real challenge for all of us is to pull back and ask not "what?" but "so what?" Where do we go with it? What does it mean? And, where does all of this take us?

Garga: Over the next 24 months, more change will happen in the marketing ROI arena than in the last 20 years. The media landscape has become so fragmented, with the advent of new and emerging channels, interactive TV and TiVo, that it's having a transcending effect on the media options marketers can pursue.

The only way for a marketer to decide where to allocate spending is through a fact-based budget setting. One issue that's delaying this process is getting the data on new media options, like advertising on PDAs. We're starting to include these analyses in our work. Marketing ROI analysis will take a center stage in overall planning, especially in planning the media mix.

Overall, what are the major obstacles to measuring the effectiveness of marketing programs?

Duffy: One of the greatest challenges is simply the healthy level of skepticism at the management level. They think, "Okay, we're going to take an advertising campaign that represents the blood, sweat and tears of a large number of folks who've agonized over it for months and try to boil it down to some numbers to calculate the ROI. And then once we have the numbers, what do we do with them?"

One of the biggest mistakes is made by analysts who don't recognize the limitations of the data and focus purely on the appropriateness of the model, on statistics and technique. The low-hanging fruit for improvement is a better understanding of the data.

"I'm a soccer mom, so I liken the offline/online mix to a sports team. You have to have an offense, a defense and a mid-field. Everybody has to rely on one another to do their part. " MARILYN FRANCK

Garga: I would say the lack of data is the greatest obstacle. For example, a major consumer packaged goods company was printing special labels on their products with an educational message and wanted us to quantify the value. There's no way for us to quantify that because we didn't know how many labels were printed or when they went into distribution -- and there was no explicit value associated with the program.
Sometimes the marketer just didn't engage in a particular marketing activity long enough to measure its effectiveness. They'll ask: "I ran a flight of television advertising for two weeks and could you tell me the impact of it?" The answer is "no."

Franck: If there's a lack of true clarity about what the impact of the program is meant to be, there's no way you can ever measure ROI. You need to understand the distinction between driving immediate sales and building loyalty longer term. You need to be crystal clear about what you're measuring and how many different activities need to be evaluated.

Other barriers that we've run across are co-branding, co-marketing and product placement deals. The whole issue of what ROI is measuring becomes far more complex because the ROI analysis you have to do needs to be calculated two ways -- for you and for your partner.

Relf: The biggest obstacle is whether you really want to measure it. At Sprint PCS, we're willing to test unproven ideas, but will only invest heavily and over time in those areas where we have data and facts to support measurement. It's important to establish what you think are the right cause-and-effect relationships so that you know what data you need to do the analysis. If you have a new product that you want to put into test market, you need to be able to do a quick analysis to decide whether broadening the test or expanding the offering makes sense.

Sometimes the math involved in analyzing marketing results is beyond the capabilities of your internal resources. It makes sense to turn to firms that specialize in very complex regression models to address these challenges.

What are the keys to the successful analysis of marketing ROI?

Duffy: There are really three key areas: managing outward, managing inward and managing forward. Managing outward is achieving organizational buy-in prior to the analysis. Managing inward means having an internal owner. It has to be thought of as continuous tracking -- versus project-based -- so that the results are incorporated over time as opposed to once a year. Managing forward means using prior learning as is the starting point for future plans, or as a basis for a revision to current plans.

Franck: Another important issue concerns the degree to which the organization is willing to learn. If you view marketing initiatives as singular successes or failures, then the internal dynamics will prevent successful analyses. The environment around you is constantly changing, as are consumers' expectations. It's the linkage of marketing and sales activity. It's the linkage of understanding the financial metrics, and having clear metrics across the entire range of the consumer experience.

Relf: It is really important to sustain the discipline over time. If you had a certain consumer offer -- let's say a rebate -- it's logical to evaluate its impact on a stand-alone basis. But then you need to ask: Did that rebate actually drive incremental sales and were the profits sufficient to pay for the rebate offer and contribute to the bottom line?

"It's much harder with the Web to create that test environment in the traditional sense, because you can't limit the Internet geographically." SCOTT RELF

To get the biggest benefit, you have to expand your analysis to include all marketing activities. You need to evaluate activities quarter after quarter, year after year. You need to evaluate how all of your various marketing activities interact with each other. It could be that the extra product that you thought you sold thanks to a marketing campaign was actually just the result of moving product from one distribution channel to another.

Garga: In addition to creating a culture that values ROI analysis, companies also need to treat ROI analysis as a cross-functional effort. They should assign ownership internally while also recognizing that there are limitations to both the model and the data. They need to understand that you can't quantify or assign values to every nugget of data.

And, certainly, you need to think of marketing analytics as part of an ongoing planning process -- not on a project-by-project basis or as an isolated event. The leaders are doing ROI analysis on an ongoing basis and view it as one of the pillars to planning. They're constantly measuring, implementing, re-measuring, learning and then re-analyzing. They continuously seek to validate their models.