Breaking Down AI: How to Work Side-by-Side with Data, Algorithms, and Automation Systems
When preparing my speech for Revolution, I found myself at the movie theater, excited to see Blade Runner 2049.
Surely a movie about robots and AI would drive creativity for my upcoming speech about understanding artificial intelligence.
While the movie didn’t inspire my speech per my original intention, it did make me realize that those types of movies — ones with robots and AI taking over the world in a far-fetched fantasy world — are the reason it’s important for me to address the hype surrounding AI.
AI is not science fiction. It’s a real, tangible technology that’s been driving our world forward for years. Unlike the movies, AI is more than science, numbers, charts, and calculations.
Unlike the movies, #AI isn’t sci-fi — it’s more than science, numbers, charts, & calculations,” says @merklik CLICK TO TWEET
It’s increasingly important for engineers, marketers, and consumers to understand the basic concepts of AI. And when you understand AI, you can ask the right questions, make your own conclusions, and start working comfortably side-by-side with the machine.
So, this post aims to break down the hype around AI, provide you with a functional understanding it as a technology, and help you see how it can help you get back to doing the marketing work you love: strategy, content, and creative.
It’s All About (Quality) Data
We often hear about how much the world has changed over the past few decades, but the way we think about “big data” may be misaligned with reality. Algorithms we’re using today to fuel machine learning have been around for decades — but we didn’t have the power to act on them on a daily basis.
When we finally had the ability, we were faced with not having enough data to teach the machines. But now, we almost have more data than we know what to do with.
Having data is great, but what the machines really need is quality data.
When you’re working in an automation center, you’re able to create campaigns without thinking about all of the heavy lifting and data processing that is happening in the background to make it successful.
For example, our automation center offers a visual representation of how the dots of a customer journey are connected — bridging data points: events, subscribers, consumer actions, and your marketing activities.
When you’re able to use AI in conjunction with automation, the machine uses the data in your customer data platform to learn buyer behaviors and deliver results.
If the data is inaccurate the machine learns suboptimal, ineffective habits rather than learning from the habits of individual buyers.
For example, the first few campaigns for one of our pilot clients for Send Time Optimization were disappointing, as we didn’t see the results we were expecting. Once we dug into the data, we realized the client gave us a list of inactive customers, rather than engaged customers, to test on. Because these customers weren’t opening emails or engaging with the brand from the beginning, so the results were different than what we would have expected from a random group.
Incoming data, that is, what you put into the machine, has a huge impact on your end results. In order to be successful, you must have a constant flow of quality inbound data.
AI is really nothing more than some data going in the machine and some other data coming out. That’s it. What makes AI mysterious is when the resulting data is combined with automation and execution.
How to use Automation for Personalization
Now that we’ve covered data collection, storing, and processing, it’s time to understand how to feed your data to the machine.
As consumers first and marketers second, we understand that personalization is the future. It’s not going away, and will require us to become more granular. So, in our businesses, we must find ways to make it possible to deliver personalization at scale.
|How we do it: When we started developing our AI-driven product Smart Insight, we started with a general approach, looking at all of a client’s purchase data and at different clustering algorithms. From there, we started to cluster their consumer base by buying behavior. This enabled us to identify meaningful groups of consumers for our clients. We named them, “charted” them, and visualized them. It was a great first step in taking data and turning it into meaningful information. And for our clients, it was the first time they saw their consumer base broken down into significant groups based on buying behavior — and understanding the value these groups bring – or could bring – to their business.|
Your intent should be to create as personalized of an experience as possible for each customer. We want to have classification of buyer behavior on an individual level, rather than based on averages from a consumer base. When you base your personalization on averages, you may find yourself doing more harm than good.
Customer data in action
Let’s say we have two different shoppers:
- Allen loves fancy shoes and buys a new pair every month.
- László hates shoe shopping, so he buys two pairs of the same shoe once a year.
If Allen hasn’t purchased shoes from you in two months, you can reasonably conclude he’s defecting and buying somewhere else. On the other hand, if László hasn’t purchased from you in two months, he’s simply within his normal buying cycle, and should be considered an active customer.
At this point, treating these two customers the same with a “We Miss You” email based on averages would be confusing to László, as he just purchased shoes and will be back in six months. He knows he hasn’t defected and it would be off-putting if a brand he’s loyal to doesn’t know his purchase pattern.
Personalization on an individual level can earn you customers, but failing to do it can just as easily cost you someone’s business, too.
How to Use Algorithms to Personalize
Even armed with the ability to customize content, personalization refers to how well your platform understands individual buying journeys, and acts on them at scale.
We need to dive deeper… but how?
How to drive “deep personalization”
For our team, it was important to explore the problem from different angles. Our data science team needed to test different algorithms and methods to uncover how we could predict the likelihood of someone purchasing or buying again, and then optimize communications to them.
The best way to do this is to test different algorithms against one another (over a large data set) to see which one would perform best, which would be most feasible to implement, and which can most easily be customized.
In this case we ended up with the Logistic regression model. This is made up of 3 steps:
- Analyze all incoming data to pinpoint the exact data points that would help with prediction. These data points include clicks, time since last purchase, frequency of purchase, and monetary value.
- Tell the machine what to focus on and find the exact correlation between these factors and the probability of someone purchasing.
- Let the machine do its thing — once the exact correlation is calculated, the model can be used to calculate the buying probability for every consumer individually.
We train our predictive models separately for each client, so this process ensures that the model will be tailored to the characteristics of your business.
A/B testing isn’t the answer
Why do I think this deep level of personalization is the way to go?
First, I’m going to challenge you to reflect on the way you’re testing and scaling personalization now. Chances are, you’re adding names to emails and A/B testing different messages.
But the reality is you can’t A/B test your way into personalization. It just doesn’t work.
40% of companies do some kind of A/B testing, and some experts say it should actually be 100% of companies. But it should be 0%. A/B testing is not the right way to go about customer engagement and customer experience.
Let me explain. A/B testing is a tool, a good tool at that, but it’s really hard to execute it properly with the amount of customer data available now. A/B testing originated from an era when you had very little information on your website visitors. Consumers might come to your site and never come back, and even if they did, you had no idea who they were.
Without having any information, A/B testing allowed us to find little tricks that worked among anonymous visitors and buyers.
But lack of information is no longer a problem. We have tons of data coming from a dizzying array of sources, so you can no longer use the excuse of not knowing your buyers.
You can no longer say, “I don’t know who you are, so here is Version A of our marketing offers.” You know who is interacting with your brand — and if you don’t, you need to more data to identify and understand them.
“Wait. More data? Earlier you said we have enough data — actually too much from too many channels. How can we possibly need more data?”
The reasoning here is that we sometimes need more quality data to better serve our customers — to give them the personalization they demand. The data needs to be more granular, more indicative of why someone is doing what they’re doing — of their user intent.
How to Develop Best Send Time Using Bayesian Bandit
If you want to determine the best time to send emails to your customers, don’t base it on the mean open time — and don’t send at the same time for every customer.
If you base send time on average customer engagement, you’ll end up picking times that work based on averages, not individuals. Your system would designate one time to send to each segment, a time you likely tested and optimized to get the best customer engagement overall, because it has no other data to learn from.
The problem with this model is that it, again, relies on averages, not individual preferences. Not everyone engages with an email at the same time every day. Imagine if you could put an email in front of someone at the exact moment they are most likely to not only open it, but also click through to your products.
This is what we want, but we need to generate more data around email engagement so the machine can learn from it.
To do so, we use a method called the Bayesian Bandit. This algorithm originally derived from the idea of a gambler trying to weigh the odds of winning on a row of slot machines. They would have to play each machine several times, track the rewards, calculate probabilities, and figure out which one is the most lucrative.
Applying this algorithm to customer data, the machine can actually calculate best send hour for one contact over time, not a segment. For example, it starts testing emails at 8 a.m., but then quickly realizes that there is no response and begins testing other times to find the best moment of engagement.
Even after the machine has identified the best send time, it will continue to test other times to make sure it is not missing a more opportune time and to ensure the behavior of the person hasn’t changed.
The Bayesian Bandit algorithm continues to learn individual behaviors to ensure your brand stays connected with your customer even as their shopping habits evolve. This is the best way to optimize send times for individuals across millions of database contacts.
How to Optimize Incentive Recommendation
Not everyone views incentives the same. For instance, I grew up in the suburbs of Budapest, so if you give me a discount above 30%, I’m going to be convinced that it’s either broken, stolen, or fake. And if you’re going to go higher, say 50 or 55% I might even report you to the police.
The machine can identify what incentive is the most effective, and most profitable, for each of your individual customers.
The final recommendation is ultimately the result of many other predictive models, calculations, and factors that optimize per individual.
In this model, your job shifts from sorting through data and making best guesses about what to send each segment to adjusting the overall strategy.
This process — using algorithms to optimize offers for individuals at scale — is AI in action. We collected the data, we processed it, we understood it, and we figured out how to feed it to the machine.
This is how you can use AI to drive the best incentive for each of your customers.
Make no mistake about it, AI is here. Unfortunately, it’s also incredibly full of hype.
But AI is real — and it allows us to solve real problems for real people… like a marketer trying to deliver a much better customer experience.
While machines may be great at analyzing data and predicting buyer behavior, they are not great at content creation. Even with the power of a machine, human marketers still need to spin creative webs, wield the power of persuasion, and turn it into concrete campaigns. AI exists to help amplify and augment the execution phase of your integrated marketing mix.
I believe humans make machines better, not the other way around.
It’s up to you leverage the power that machines offer. It’s up to you to make your work life more fulfilling, and make your customers’ experience better. AI is the bridge to mending that existing gap, and I hope that 2018 will usher in a new age of both happier marketers and happier customers.
► To watch László’s full-length, 25-minute presentation on AI, algorithms, and automation, click here.
- Revolutionizing Big Data with Artificial Intelligence
- You’re a Marketer, Not a Data Aggregator
- Building Your Ship: A (Step-By-Step) Guide to Enterprise AI Marketing Implementation
László Merklik is the Chief Product Officer at Emarsys. He lives in Budapest, Hungary.
Connect with László: