Data-Driven (Bad) Decision-Making

This is the first in a series of posts on the benefits and pitfalls of relying on data to drive your marketing decisions.

What could go wrong with this clever use of user browsing data? Well, it turns out that people who browse wedding dresses, locations, cakes, and the like are not, in fact, all immediately getting married. In fact, many are wedding guests, planners, and daydreamers that are quite single. Some of these individuals aren’t too happy about being congratulated on their nonexistent weddings, and they’re also active on social media.

It turns out that blindly using data to target your customers is not always the best idea. Let’s explore where Pinterest went wrong:

Using a finding in your data without confirming the finding qualitatively

A digital marketer at Pinterest could have avoided this situation with about an hour of work. A quick survey blast to a sampling of individuals who’d been identified as “likely brides” asking whether they were currently planning a wedding could have uncovered that the bride assumption was faulty. This bias toward quantitative data is extremely common in modern marketing, and it’s especially so among tech companies.

This faulty belief is that you can learn everything you need to know about your customer simply by monitoring their behavior. The most common manifestation of this belief comes in the form of a typical A/B test. Most A/B tests are conducted as if you were monitoring particles in a accelerator. If the test beats the original design, then the assumption is made that the test is preferred over the original by most users. The next test then pits the winner of the first test against a new test, typically based on a hypothesis generated in a silo, and the process continues. Unlike particles in an accelerator, humans have the amazing capacity to communicate the reasons for their actions. They often don’t communicate their reasons very well though, so we need to validate them by using quantitative methods. Survey and feedback forms combined with behavioral data work better together to create more informed hypotheses and more valuable testing outcomes.

Using only a single signal to target

If you go to the gym every day, then you’re probably a healthy person. What if you go to the gym every day and follow it up with a Big Mac while smoking heavily? People can rarely be defined by a single signal, and yet often campaigns target people based on very simple criteria. We look for users who have viewed certain items on our site or are interested in similar products online. It isn’t problematic to target these users in an attempt to reach those who do fit your target, but making blanket assumptions that show up in your messaging is very dangerous.

In the case of Pinterest, the bold assumption that these wedding-pinning users were brides turned out to be too broad. But could this targeting have been improved? If the above qualitative survey had been done, we could look for additional correlated activities or signals that could help remove false positives. We could look at user history to see if this interest were something new or something that they’d been engaged in for some time. In any case, going beyond a single criteria could have done quite a bit to limit the number of mistaken assumptions and reduced the strength of the tweetstorm.

Giving away how much you know in messaging that isn’t opted in to

Despite the obvious failings of mocking your customer base in a tweet, it also revealed that Netflix knows pretty much everything about your viewing behavior. Netflix customers who picked up on this launched a PR nightmare, starting with a tweetstorm and ending with a segment on The Late Show with Stephen Colbert.

Both Netflix and Pinterest made the same mistake of using messaging that is far too specific for a highly targeted ad. While it may be tempting to customize your messaging to speak directly to an audience, in most cases it will backfire. If your targeting isn’t perfect, you will create obviously irrelevant messaging that can insult your users. Even if you get the targeting correct, there is a very real chance you’ll cross over the “creepy” line and cause your customers to turn off. If your messaging is opted in to and your customers have self-selected to provide you with the information, then you can be as specific as you’d like, but derived information should be kept to a minimum. Instead, focus on general suggestions and questions. For example, had Pinterest started that email with “Planning a wedding?” or “You might be interested in…” rather than an assumption of knowledge about the user’s life, they would likely have avoided the entire situation.

Your user data is a powerful tool for customization and personalization. It can be used to help create better web experiences and ensure that people only see relevant and interesting advertising. Like most powerful tools, it should be used with caution or someone will come and take it away. Recent backlash over privacy and a push toward do-not-track flags have been accelerated by unfortunate decisions like these made by companies with too much data on their hands. Being responsible and thoughtful about how you target is important for the future of the industry, and it keeps your brand safe from tweetstorms.