Do your chatbot users speak English? The answer is probably no.

Out of the 4.72 billion Internet users, only 25.9% speak English. In other words, 3 in 4 users DO NOT speak English. So how are you going to engage your audience?

To avail the benefits of a chatbot, you need to make sure your users (i) can interact with the chatbot and (ii) the chatbot can understand them. That means, your chatbot should speak your user’s language and decode the right intent.  

You need a multilingual chatbot that’s localised to meet your users’ expectations to improve your chatbot engagement. But there are challenges with being multilingual. Going local isn’t easy. 

To understand the challenges associated with the localisation of AI chatbots, we will look at:

• How localisation is more than translation?
• Why is it challenging to localise AI-powered chatbots?
• What game plan you can use to overcome AI localisation challenges?

How localisation is more than translation?

While localisation and translation are commonly substituted, there is a vast difference between them. 

Translation is the word-to-word rendering of sentences or phrases from one language to another. It rarely considers emotions, tenses, regional dialects or context. Of the two, translation is less complex, not time-consuming, and cost-effective.

In contrast, localisation is adapting to the cultural preferences of the region. In addition to translating the language, you create systems that decode customer habits, emotions, regional regulations, and more. Comes as no surprise, that it takes more time, money, and effort to localise.

Netflix demonstrated the importance of localisation in their expansion strategy. In 2010, Netflix was available primarily in the U.S. By 2017 it expanded to 190 countries. Today, out of 130 million subscribers, 73 million are outside of the U.S. How did they achieve this? 

Early on, Netflix realised that its international subscribers weren’t fluent in English and preferred local language programs. In phase-wise progress, they collected data and analysed it to learn about international markets. Based on this research, they invested in content geared towards the preferences of new geographies. Some of the things they did were:

• Marketed based on local preferences
• Offered new content in regional languages
• Added local languages for subtitles and dubbing
• Optimised personalisation algorithms for the global library
• Expanded support for a range of devices, operations, payments, etc. 

Similarly, if you want to provide a delightful customer experience and tap on the local market, you need to localise your chatbot, and not just translate it. Especially because the AI in the chatbot reads the little nuances in the user’s conversations, understands the intent based on this data and personalises their experience accordingly. 

So, why is it challenging to localise AI-powered chatbots?

As seen above, localisation concerns conform to local or regional preferences. And when it comes to AI, a major challenge includes imitating human communications, which includes, variations in the context interpretation, inflection in languages, hybrid languages, subtle changes in the meaning of words due to dialects and other such nuances involved with languages.

And it’s not just limited to language. It’s also their preferences in payment mode, the channel of communication, etc. We’ve identified six such challenges a business faces in AI localisation. 

1. Preference of your local audience
2. Limited resources and quality data
3. Multi- and hybrid languages
4. Colloquialism and slang
5. Dialect and word choice
6. Inflection language

1. Local audiences’ preferences

First and foremost, you need to understand what your customers prefer. For example, if you are an international company and have been using PayPal for your payments, your users in India will be reluctant to complete payments using this method. You will need to provide them with local UPI payment options such as Paytm, Google Pay or Razorpay payment gateways. 

Similarly, your users in Asian countries would prefer chatting on WeChat and Viber over websites and maybe even WhatsApp. 

Hence, it’s really important you research and understand your audience thoroughly. And for this, it’s best if you collaborate with local third parties who understand your audience better. 

2. Limited resources and quality data

Probably one of the biggest challenges in localising your AI chatbot is the availability of data. For years, data scientists have been using English to train AI. But now that businesses want to focus on their customers and their preferences, they are trying to train the AI in local languages to understand the users better. Unfortunately, they are struggling to find datasets with sufficient utterances in these languages.

To address this issue, companies purchase data from third-party vendors, a common practice these days. However, many times, these datasets lack quality in either labelling or annotations. This requires companies to spend extra time and effort in clearing data and training the AI.

3. Multilingualism and hybrid languages

As we move towards globalisation, we are also moving towards multilingualism. That means, we tend to speak two or more languages on average. What happens next is we mix words from one language to another and create a sentence of mixed languages. This is called a hybrid language. Examples include Hinglish (Hindi and English) and Spanglish (Spanish and English). 

You cannot simply solve the hybrid language issues by training your chatbot in two languages. It will just not be sufficient. Your users are switching context between two languages and the AI needs to be trained on this context switch, which is a rather difficult task to achieve.  

Second, you need to label each script with the correct language and train your AI on all the possibilities this hybrid language can have. This takes time and a talented team of AI experts. 

4. Colloquialism and slang 

Every country has words that are casually used in conversations and do not mean their dictionary definition. These contextual words have different meanings in different scenarios. And simply translating them won’t translate the message behind the sentence. 

For example, in South Korea, the word Oppa translates to elder brother. But Korean women tend to use it to refer to older males in general and these days to their boyfriends. The context depends on who is talking to whom and what is their relationship. Similarly, in Tamil, the word macha means brother-in-law, but in local slang, it means friend (something like dude or buddy). 

When localising a chatbot, getting the right meaning of these words is challenging but if done right, you can build a personal relationship with your audience. 

5. Dialect and word choice

People in different countries have different vocabularies for objects and emotions. Even within a country or region, people have different dialects, which constitute differences in pronunciation, vocabulary and grammar. 

Getting these aspects right is critical to conveying your message correctly. For instance, at a country level, US and UK English use different words for things (see the table below).

American English	British English
bill	banknote
attorney	barrister, solicitor
cookie	biscuit
hood	bonnet
trunk	boot

When going for a multilingual chatbot, you need to consider word preferences for your that region. Otherwise, your users will not understand your chatbot.

6. Inflection in languages

Inflection is the process of forming or modifying words based on different grammatical categories such as tense, case, voice, person, number, gender, mood, etc. Some languages have more inflection than others. English has relatively less inflection when compared to Hindi, German, Arabic etc. 

Let’s understand this with an example. 

English: Mary is washing clothes. 

Hindi: मैरी कपड़े धो रही है। (Mary kapade dho rahee hai)

Now if we change the subject of this sentence to a man, say Kumar, the verb will not change in English, but it will change in Hindi. 

English: Kumar is washing clothes. 

Hindi: कुमार कपड़े धो रहा है। (kumaar kapade dho raha hai.)

In languages with more genders, such as Spanish, German and French, there are more variations in the verb. 

Languages are very complex and if you want your AI to replicate human conversations, you need to train it on different aspects of the language. This not only helps in creating human-like conversations but also helps your AI to understand the user and their intent accurately. 

What game plan you can use to overcome AI localisation challenges?

Netflix focussed on their customers. You can do it too. 

You need to develop a game plan for your AI chatbot to help your users who don’t speak English. To ensure your chatbot is localised for your users’ preferences, you need to:

• Collaborate with local companies to get relevant datasets
• Research and understand your customers’ preferences
• Design your conversation flow based on your research
• Analyse past interactions and train your chatbot
• Train your AI in local or regional languages
• Partner with local service providers for payments, delivery, etc.

Wrapping Up

Research shows that 55% of global customers buy only from brands that speak their language. And with conversational commerce on the rise, not providing multilingual chatbots can cost you a large section of your user base. 

Verloop.io is a leading provider of conversational AI in India, the Middle East and Africa region and the Southeast Asia region. We’ve trained our AI on billions of intents and utterances in English, Hindi, Arabic, Indonesian and more so that it can accurately understand the user in these regions. ADIB, Kanmo Group, and Flyin are some of the companies that are using our multilingual trained AI.

Talk to our conversational experts and get more details on the multilingual chatbots for your location.