Market Research posts

“When The Gauntlet didn't do as well as hoped, Warner's became concerned that Eastwood was making the wrong choices. He always had a streak of Burt Reynolds redneck humor about him, and when he wanted to play opposite an orangutan in Every Which Way But Loose, Warners' did some market research that indicated a negative reaction to the title, to the orangutan, and even to the idea of Dirty Harry in a comedy. But Eastwood doesn't have much truck with market research and went ahead anyway. It cost about $8 million and grossed about $85 million (about $150 million in today's dollars*), making it his biggest film.”

--Peter Biskind, in the article “Any Which Way He Can,” in his book “Gods and Monsters.”

* The article was written in the early 1990s. So that $85 milliion in 1978, which made it the no. 4 box office hit of the year, which was $150 in 1993 dollars, is approximately $289 million today.

What's wrong with Market Research? Plenty according to IT consultant, author and developer Isseki Nagae (in translation from the Japanese):

Large advertising companies have long invested huge sums of money in conducting studies based on research, customer surveys and interviews. I myself have participated in such studies. At the big advertising agencies they round up users and market to them as reps from the manufacturing companies watch, sometimes through one-way mirrors. I’ve heard of some companies that go so far as to base their product planning on the opinions of schoolgirls. Will this type of planning seriously result in a hit product? I don't think so, and ... neither did Steve Jobs. I’ve actually been to many of these marketing focus groups, and I can tell you that no great ideas come of them.

He then quotes Steve Jobs on the matter:

It’s not the consumers’ job to know what they want.

And:

It’s really hard to design products by focus groups. A lot of times, people don’t know what they want until you show it to them.

See also:

• Pomona's reaction to “The Magnificent Ambersons.”
• Market research's initial reaction to “All in the Family” and “The Mary Tyler Moore Show.”
• Audience test scores and “The Office.”

“I find they are pedestrian people. Their consensus is cliche. Something is good because they've seen it before.”

--Herbert B. Leonard (1922-2006), producer of “Naked City,” “Route 66,” and “The Adventures of Rin Tin Tin,” and known as a writer's producer, as quoted in Susan Orlean's biography, “Rin Tin Tin: The Life and The Legend.” The “they” he is referring to are studio executives.

“We do not need trouble pictures, especially now. ... Make pictures to make us forget, not remember.”

“People like to laff, not be bored to death.”

“The picture was a masterpiece with perfect photography, setting and acting. It seemed too deep for the average stupid person. I was disgusted with the way some people received this picture.”

These comments, taken from David Thomson's book, “Rosebud: The Story of Orson Welles,” pg. 215, are among the response cards (53 favorable, 72 unfavorable) for the March 17, 1942 screening of Orson Welles' original version of “The Magnificient Ambersons,” at the Fox Theater in Pomona, Calif. It was this audience survey, this market research, that led to some of the most infamous cuts and perhaps the most horribly re-shot and re-edited ending in movie history. Add it to the list.

Way to go, average stupid person.

“The magnificence of the Ambersons was as conspicuous as a brass band at a funeral.” Unless, of course, you lived in Pomona.

I'm late to the Maclolm Gladwell parade. I read his stuff in the New Yorker but didn't check out any of his books until I had to read "Outliers" for work—I interviewed M&A lawyer Joseph Flom, who is the subject of that book's fifth chapter, "The Three Lessons of Joe Flom"—and was particularly impressed, not only with the Flom chapter, but with the first chapter, in which Gladwell dissects the success of youth hockey players in Canada and the puzzle over the preponderence of early-month birthdates among them. Lots of January, February and March babies playing in the NHL. Why? January 1 is the cut-off date for youth hockey, so at an early age a January 1st kid will be competing against a December 31st kid and have a year advantage in growth and coordination. That January kid will play in more tournaments, and get more coaching and practice, and what began as an accident of birth will become a self-fulfilling prophecy: He'll be better. We're never the meritocracies we think we are.

"Blink" isn't quite as good but I did enjoy the chapter, "Kenna's Dilemma," for its confirmation of my own thoughts on audience test scores. Two years ago, when this blog was a baby, I wrote how "The Office" (both versions) got some of the lowest audience test scores in their respective networks' histories, as did "Seinfeld." I asked:

If you don’t recognize Seinfeld and The Office and The Office for what they are, or what they might be, what good are you? How many other Seinfelds are you turning into something ordinary and short-lived? How many millions are the money-people blowing?

Thanks to Gladwell, here are a few more names to add to the list:

In the late 1960s, the screenwriter Norman Ler produced a television sitcom pilot for a show called All in the Family. ... All in the Family scored in the low 40s [out of 100, in market research]. ABC said no. Lear took the show to CBS. They ran it through their own market research program... The results were unimpressive. The recommendation of the research department was that Archie Bunker be rewritten as a soft-spoken and nurturing father. CBS didn't even bother promoting All in the Family before its first season. What was the point? The only reason it made it to the air at all was that the president of the company, Robert Wood, and the head of programming, Fred Silverman, happened to like it...

That same year, CBS was also considering a new comedy show starring Mary Tyler Moore. ... The [market research] results were devastating. Mary was a "loser." Her neighbor Rhoda Morgenstern was "too abrasive"...

"Archie Bunker [should] be rewritten as a soft-spoken and nurturing father." That's one of my new favorites.

In case the lesson isn't obvious, Gladwell drives it home:

The problem with market research is that often it is simply too blunt an instrument to pick up this distinction between the bad and the merely different.

I wrote much the same a year ago January, regarding a Tad Friend New Yorker piece about audience testing, in which it was mentioned that "Pulp Fiction" received some of the lowest test scores in its studio's history and "Akeelah and the Bee" received some of the highest.

"Pulp Fiction," "All in the Family," "The Mary Tyler Moore Show," "Seinfeld," "The Office," "The Office."

Others?

A friend once wrote a song called “Mr. Time,” which, in its overall sense of losing everything (inch by inch) while waiting for something, anything, to happen, I’ve always, unfortunately, identified. One stanza in particular hits home:

Tooth by tooth
You put on a smile
And stuff in a word for yourself
But every word on your own behalf
Is just an ad for something that no one needs

There’s doing and there’s selling. The great myth of America is that it’s all about doing (Horatio Alger, bootstraps, etc.), while the great reality of America is that it’s all about selling. I’m not a bad doer but I think I’m one of the worst sellers in the world. I can sell nothing, particularly myself, because of what’s articulated in “Mr. Time.” Every word on my own behalf does feel like an ad for something that no one needs.

This means, yes, I’m still thinking about Tad Friend’s New Yorker piece on Tim Palen and Hollywood marketers. Particularly these lines: “Publicity is selling what you have... Marketing, very often, is selling what you don’t have...” These are people so good at their craft they can sell what doesn’t exist. Remarkable. God, I hate them.

I do want to mention one area where I agree with marketers. It comes two-thirds of the way through the article and involves test audiences. Friend writes:

Yet testing is fraught: it rewards comedy, narrative, and familiar stars or plot elements, and often undervalues the new. Executives’ testing stories take divergent paths to the same punch line. Either they decided not to tamper with a “Pulp Fiction,” despite testing results invariably described as “the lowest scores in the studio’s history,” or they were confounded when an “Akeelah and the Bee” faltered commercially despite “the highest scores in the studio’s history.” In both scenarios, the numbers lied. “Testing is a sham,” one marketing consultant says. “All you’ve learned is what people thought of a movie they didn’t have to pay for. It does not mean they’re going to go pay for it.”

Ex-motherf---ing-actly. Particularly the line about undervaluing the new. It was the same for “Seinfeld” and the British “Office” and the American “Office”: low, low audience test scores. People didn’t get these shows. They didn’t get “Pulp Fiction.” I’ve never seen anything like this before so it can’t be any good. In this way, test audiences are actually like marketers, who, according to Friend’s article, have trouble selling the new because there’s no playbook for it. It takes a lot of luck for a “Seinfeld” to get through. One wonders how many “Seinfeld”s — and thus cash cows — get killed in the process.

So that’s the area where I agree with marketers. Here’s the area where I don’t get marketers. These are people who supposedly can sell anything — including something that doesn’t exist. They can sell crap and make us think it’s pudding. But they can’t sell quality.

The best films are sold on a limited basis, in select cities, and might, if carefully nurtured, make it into most big cities and most states. But that’s if it’s lucky and the zeitgeist is right. Otherwise, not.

I know marketers take their orders from someone else, as we all do, but some marketers, as Friend tells us, are now running the studios. Universal, run by a former marketer, is one of the worst culprits. Unless they know something I don’t, unless there’s a strategy here that I don’t see, they’re in the process of killing both “Frost/Nixon” and “Milk.”

There’s an assumption out there that people don’t want quality. There’s an assumption out there that people want (the same old) crap. I’m hardly a pollyanna but, more and more, I’m assuming the opposite.

That’s the unanswered question from Friend’s article. It’s the unasked question of marketers and admen everywhere: How good can you be if you can’t even sell quality?

I’m not the first guy to not get fashion magazines. You take the world’s best-looking people, give them the world’s best make-up artists and hairstylists and photographers, airbrush out what imperfections remain...and half the time they still look like heroin addicts. But Patricia subscribes to a few of these things and sometimes they’re worthwhile. The W magazine with Charlize Theron on the cover includes an article on Ricky Gervais of The Office fame, horribly titled “Tricky Ricky,” in which we get the following:

Before The Office premiered on the BBC in 2001, Gervais recalls, the show received the lowest audience test scores in the network’s history, but he defended every word in the script. It was a similar story with the American version: Gervais remembers getting an e-mail from producer Greg Daniels saying the series had scored abysmally. “I sent back a message: ‘Brilliant, so did we,’” he says. Now, he points out, The Office is NBC’s highest-rated sitcom. “All the things I’ve ever loved, I hated at first,” Gervais adds. “the best things are an acquired taste.”

For the writer, Paul Quinn, the point of this story is that Gervais’ apparent self-assurance, “rooted in defensive smugness or genuine confidence,” helped save his greatest creation. Here’s the lesson to me: Audience testing sucks. Seriously. It was the same story for Seinfeld, which became one of the most successful sitcoms in TV history. But the initial test scores reflected an audience distaste. People didn’t like it because they didn’t get it. It wasn’t familiar.

One wonders if testers are attempting to fix this obvious problem with innovative shows and movies. Forget aesthetics for a moment. Just think of the money. These things are cash cows. Cash cows with long fucking lives. And the money people, whose job it is to find such cash cows, when confronted with them, actually try to turn them into something else.

If you don’t recognize Seinfeld and The Office and The Office for what they are, or what they might be, what good are you? How many other Seinfelds are you turning into something ordinary and short-lived? How much money are the money-people blowing?

All previous entries