Write once, chat anywhere: the easy way to create chatbots with Drupal

Historically, we have many ways of serving content digitally: through a website, through mobile apps, via social media, RSS feeds, RESTful APIs (allowing content to be consumed by other apps, websites etc), and email. Now we have a new player in the game: chatbots and personal assistants. Conversational interfaces promise a more natural way for users to interact with your organisation’s content, but without the need for humans on the other end.

In this post I’ll walk through the basic concepts involved in creating conversational experiences based on your already existing content and user journeys. It’s based on a talk I delivered at DrupalCamp London earlier this month: “Hi user, I am Drupal. How can I help you?” Here’s the recording:

And here’s the slide deck.

Conversational experiences FTW

Don’t think of this as something you need to add to your website, increasing complexity. Think of it as a new way for users to interact with your brand/content, in a more natural, conversational manner. They’re perfect for scenarios where a user doesn’t have time to hunt for an answer to their needs on your website but where the query is straightforward enough that human interaction can be avoided.

For example, how many times has your organisation received an email of the type “I’m on your website but can’t work out how to do x”, where you’ve ended up sending a link in response? This is exactly the kind of enquiry that bots can automate, saving time for both you and your user.

But of course, bots can go even further than this. Because they have access to all your content and data they can help the user by surfacing more than just links. Text, images, video, forms, can all be embedded in a conversation in response to queries. Bots can even edit the user’s data (e.g. profile info, preferences etc) on the fly. In theory, anything that a user can do on your website can be done via a conversational interface.

• Answer FAQs
• Provide media-rich search results
• Provide access to services

How do chatbots and personal assistants work?

The magic is done through natural-language processing (NLP), a technique which relies on machine learning to work out the intent of a user’s query, what entities it concerns, and understand and manage the context in which the query is made.

Let’s consider a query from a housing association resident, made to the association’s website: “How can I get my boiler fixed?” A common enough case.

Intent

The intent here is a request for maintenance. But this could be expressed in many different ways to the phrasing “How can I get my boiler fixed?” e.g.

• Can you fix my boiler?
• I need my boiler fixed.
• I need you to fix my boiler.
• How can I get my boiler fixed?
• How can I fix my boiler.
• My boiler needs to be fixed.
• Fix my boiler!
• Etc

NLP engines, like those that power Alexa, Dialogflow, Watson, Conversable etc rely on being fed many examples of ways in which an intent can be phrased to accurately work out what the user is trying to accomplish.

Entities

Now that our NLP engine knows that the intent is a maintenance request, it needs to work out what entities the request concerns, so it can point the user in the right direction. NLP engines are pretty good at identifying entities already, so when programming them we can use entity placeholders, e.g.

• Can you @task my @appliance?
• I need my @appliance @task
• Etc

Where @appliance could be ‘boiler’, ‘microwave’, ‘fridge’, ‘front door’ etc, and @task could be ‘replace’, ‘fix’, ‘install’ etc.

Of course, each possible value of the entity will have multiple synonyms (fix: repair, mend, sort out, see to) which the NLP will have to deal with.

Context

The context in which the user’s query is made will often be critical to determining the desired response of the bot or assistant. In our example with the faulty boiler, the user probably needs a repair appointment, but before the bot can book that appointment for them it might need to understand what kind of property the user lives in, their address, how urgent the request is, an account number, time of day etc.

The NLP engine might be able to extract some of this data from the user’s account, if they’re signed in, or from other sources (e.g. time of year, availability of repair crews) or it might need to ask follow up questions.

Each conversation will have its own context, which evolves as the conversation progresses and which is cleared when the conversation ends.

Designing conversational experiences

We can think of the NLP engine as the brains behind the conversational experience, while the chatbot is the face, which actually interacts with the user. This could be a bot in a messaging app (either embedded in your site or in an external app like Facebook Messenger), an Alexa skill etc.

From behind the scenes then, the conversation is three-way: the user, asking the question; the NLP which understands the query and translates it into machine-friendly language; and the chatbot/assistant, which performs the required logic (sending an email, providing a link, booking an appointment etc).

Does this mean that, to surface your website content via conversational interfaces, you need to write new code to sit between your website and each bot? That’s exactly what we found when we were asked by a client to make an Alexa skill for surfacing event listings from their Drupal website, and then a Google Home app, and then a Facebook Messenger bot, and then a website chatbot. Writing new code to sit between Drupal and the bot each time seemed excessive, when the interactions we were programming were essentially the same. That’s why we created the Drupal Chatbot API.

Chatbot API for Drupal 8

Chatbot API is an extra layer that sits between your Drupal installation, your NLP and your various chatbots and personal assistants. It makes your website chatbot-friendly so you don’t have to write new code each time you translate your conversational experiences from one interface to another.

Chatbot API works with all these systems:

• Dialogflow
• Alexa
• Slack
• Cisco Spark
• Microsoft
• Twilio
• And soon, Botkit

The video embedded at the top of this post contains additional material aimed at developers which walks through creating a conversational experience with Chatbot API. I’ll follow up with a more developer-oriented post soon…