3 Questions to Ask When Someone Says Their Chatbot is “AI-Powered”.

By Anush Clive Fernandes / In Chatbot analysis, Sales / March 3, 2020 / 5 min read

In a Narrative Science Report released in 2017, 61% of businesses said they’ve implemented AI in some capacity. That’s up from just 38% in 2016.

It’s 2020 now and I’m entirely certain that self-proclaimed number is now closer to ~80%.

Catchy headlines like these make it easy to wonder if your enterprise firm is doing enough to keep up with the competition.

But how do you cut through the noise? AI is an incredibly broad and evocative term. It’s sole purpose in media today is often simply to rally the masses in one direction or the other.

So let’s take it from the top. Let’s start off with that report.

Where are these enterprise companies applying this AI?

According to the same report, conversational automation and technologies related to it led the charge when it came to enterprise AI application. In fact, just four subsets accounted for nearly 60% of implementation.

Machine Learning,
Voice Recognition, and
Personal Assistants.

how to choose a chatbot


So when a sales rep says their chatbot or any product that they’re trying to pitch you is powered by AI – a few intelligible questions take you a long way.

With that in mind, your first question is as follows.


Let’s start off with the big disclaimer: Any AI product is only as powerful as the model that powers it.

Like we mentioned earlier, there’s a lot of eye-catching headlines in the world. But in (an admittedly oversimplified) reality, terms like AI, ML, Predictive Analysis and Neural Networks are just variations of the same core ideology that’s existed since the 1950s.

The “AI” that most people throw around in conversation today is less about an all-purpose assistant – but more about number-crunching math.

And that’s where statistical models kick in.

Decision trees are often the most common ones around – and should be your first line of questioning.


Decision trees are produced by algorithms that identify various ways of splitting a data set into branch-like segments. These segments form an inverted decision tree that originates with a root node at the top of the tree.

A decision tree is fairly easy to understand and is beloved for that very reason. An empathetic data scientist could easily walk you through the braches of his work.

A ‘random forest’ or ‘neural network’ requires a more scientific background. But asking whether it’s a decision tree model (and how the model works) and is often enough to stump most people who puffy-chested salesmen.

Book A Demo Today!

Talk To The Team


Machine Learning is based on two sets of data: Training data and test data.

Understanding what each one is and what they do gives you incredible leverage in conversations with braggadocious salesmen who boast about their AI-powered products that give “100% accuracy and precision”.


Assume you’ve taken half the population of a small town as a sample set. You’ve collected all necessary demographic data-name, age, gender, income, ethnicity, religion, and family structure.

Your goal is to try to figure out how many of them grow up to like the color blue.

Since you have all the required details – name, age, etc., you can build a model to identify correlations and trends. You find that a certain demographic group is more likely to like blue when compared to their peers.

This sample set that you’ve used to teach the model is called training data.

“Training data” is an initial set of data used to teach a program the necessary correlations within the dataset.

You now have a model to predict who likes blue. Time to put it to the test.


You now use the other half of the town’s population. This is the test data.

“Test data” is trying to see if the algorithm can implement the correlations it observed during training on a separate, similar dataset.

If the model can successfully predict who likes blue amongst the second half of the town’s population based on what it learned from the first half during training, the model is doing its job well.

This part of the process is where most marketers and salesmen get their “close to 100% accurate” claim from. But there’s a lot that can go wrong with this number. Case in point – overfitting.


A chatbot’s end-goal is to simulate human conversation with its customers, clients, and visitors.

Any AI used in a chatbot is simply trying to predict what a human is saying. It predicts this based on large sets of conversational training data it’s been fed.

This makes the kind of training data that is used extremely important to a chatbot’s success.

Taking our analogy into consideration – while gender and age are likely important qualifiers to determine whether someone will go to college, it is unlikely that a variable like hair color or preference in soft-drink has any standing on higher education.

You want to remove all potential noise from your training data to ensure clean results. Not doing so is called “data dredging”.

It’s also important to note that the predictions that an AI-powered tool presents are correlation – not causation.

Talk To The Sales Team.

Schedule A Demo



Anush Clive Fernandes

Content and Marketing, Verloop.io

Love Canines, Conversational Automation and Curry - Steph and otherwise.