Why Chatbots Fail: 5 pitfalls and how to avoid them
Have you ever had a bad experience with a chatbot? Of course you have! Perhaps the chatbot completely missed the intention of your message, or you got stuck a never-ending loop cycle of “I didn’t understand, can you try rephrasing your question?” Perhaps you spent 10 minutes talking to a bot to achieve something you could have done yourself in five. Or perhaps the bot said something rude and did not pick up on your disapproval — we’ve all had bad chatbot experiences.
The hype cycle of chatbots was in full force; the fanfare and promise of chatbot armies automating service desks was upon us, and companies scrambled to implement a chatbot for fear of being left in their competitors’ AI dust. As we’ve come down the other side of the hype cycle, that promise has not materialised, and we’ve seen a significant number of chatbots fall short of their intended value. In other words, a large proportion of chatbots have failed.
In my experience building, tinkering, and playing with various chatbots, a large number of bots do not achieve their aspirations. Why does this happen time and time again? For the most part, organisations are too hasty to implement a chatbot; they see this technology as an easy way to say “hey, we’re digital too!” More specifically, if this is broken down, I see five key causes of chatbot failure. These are:
1. User value is not well understood
It is imperative in the design phase to establish what user value will be achieved by implementing a chatbot. Too often, development teams charge ahead, caught up in the hype, and deploy something that users don’t actually need. As Ali Loghami asserts, great chatbots reduce friction in the user experience (UX). Too many chatbots in production increase the number steps or time to complete an action, thereby increasing UX friction.
The other side of this issue is that user value is often not well understood by the users themselves. There is no point building an earth shattering solution if the value of the chatbot is not communicated to users, who may be content with their existing way of working.
Some questions to consider to avoid this pitfall:
· What friction will the chatbot remove from the process?
· Are we addressing a real problem, or are we building a chatbot for the hell of it?
· What value is this delivering to the user/customer?
· How is value being communicated to the user before they use the chatbot?
2. Chatbot personality is not defined
One of the often-overlooked components of a successful chatbot is the bot personality — and I don’t just mean telling a joke! Personality is the combination of behaviour, motivations, characteristics, and qualities that form a being. Research by Tuva Lunde Smestad has found that personality plays an important part in how users perceive chatbots. In fact, it can be the determining factor in whether the user interacts with the chatbot again, or not. For this reason, every chatbot my team and I build has a well-defined and documented persona, developed iteratively from numerous design workshops with end users. This persona is central to the development of conversation flows, and informs every design decision made. A misaligned, generic, or undefined chatbot personality has a detrimental effect on user experience, and research shows it can affect user perception of other non-personality aspects of the bot, even with all else equal.
Some questions to consider to avoid this pitfall:
· What personality traits does our bot have, and how will these resonate with the user?
· What persona must the chatbot assume to delight the user, and maximise engagement and retention?
· How will the chatbot personality play out in the context of this use case?
· How will the chatbot personality govern the interaction between bot and user?
3. Conversation flow is too unstructured
Despite the hype, Natural Language Processing (NLP) is not good enough yet at a commercial level for authentic, humanistic flowing conversation. As I’ve learnt first-hand, we can never truly predict what users might say in the real world, and user behaviour will continue to surprise us. While a degree of free-flow, unstructured conversation can be effective in certain situations, an end-to-end free flow chatbot will cause frustration to users who type something unexpected and are misunderstood or not understood at all. No chatbot is perfect, and error messages are crucial for when “I don’t quite understand what you’re trying to say”. However, if users end up stuck in error loops, it is likely because the conversation is not sufficiently structured. Sacrificing some of that ‘cool’ free flow conversation in favour of UI elements such as buttons may pay off in the form of task completion rates and successful user outcomes.
Some questions to consider to avoid this pitfall:
· Is there a logical conversation path for users to follow?
· Is it clear what is expected as an input at every step of the conversation?
· How well are the limits of chatbot capability established?
· How will the chatbot bring the conversation back on track when this limit is breached?
4. Adoption strategy has not been considered
As chatbot developers, we forget the chatbot hype is not a phenomenon experienced by everyone. Unfortunately, it is not just a case of ‘build it and they will come’. As Casey Phillips explains, humans are creatures of habit, and convincing them to use a new tool can be difficult, especially when they already have some way of achieving the objectives your chatbot supports. As such, a comprehensive adoption strategy is imperative. This will ideally include components of a communications strategy and change management, and will be unique for each organisation. Typically, my teams will conduct an assessment to understand staff experience with chatbots, and willingness to use chatbots in the workplace, the results of which will inform the adoption strategy developed.
Some questions to consider to avoid this pitfall:
· How are we going to generate awareness and communicate the launch of the chatbot?
· How will misconceived expectations and other concerns be managed?
· How will we position the chatbot to maximise adoption?
· How will we ensure users are sufficiently prepared and skilled to utilise the chatbot?
· How will we measure adoption?
· Through what channels are users able to provide feedback or raise issues?
5. Analytics and insights aren’t considered
The four pitfalls above talk to the bot building and preparation phases. However, your job doesn’t end at Go Live! As mentioned, user behaviour is unpredictable. No amount of testing will prepare you for how your chatbot will be used in the real world. Further, no chatbot will please everyone! However, we see a lot of chatbots deployed and left to sink or swim. This is a critical pitfall to avoid. The success and value add of your chatbot should be closely monitored to drive continuous improvement. Conversation data, usage, and user feedback should all be analysed to identify areas of improvement. Platforms like Watson Assistant allow you to track basic KPIs and train the NLP model using real user data. Conversations should be analysed to understand success/completion rates, and where users typically get stuck. Further, and particularly valuable in the weeks following Go Live, user feedback should be gathered directly from the user themselves through surveys and other engagement channels.
Some questions to consider to avoid this pitfall:
· How will we capture and analyse conversation data?
· How will we drive continuous improvement of the NLP model?
· How will we capture and address user feedback?
· How will we use insights to improve and refine the bot moving forward?
Chatbots have the potential to fundamentally disrupt customer service and engagement
Artificial Intelligence is improving every day. So too is our collective chatbot capability and understanding of user behaviour and requirements. To this effect, we will see the proliferation of chatbots, and the value they provide, continue to rise to the point where most interactions with companies will be with full service artificial intelligence of some description. But we are not there yet; there is a long way to go. However, this should not stop us creating virtual assistants that can add real value to users if we heed the common pitfalls above.
If you’ve had similar or different experience and would like to continue this discussion, please connect with me on LinkedIn.
