Handling deviations from the happy path

Video



Another important aspect of good conversational AI assistants is the ability to handle conversations that deviate from the “happy path”. Let’s dive into how those situations occur and how Conversation Repair feature in CALM enables assistants to handle such situations effectively.

Intro to Conversation Repair

Deviations from the happy path happen when the user says something unexpected based on the general conversation flow. For example:

  • When asked to provide a specific detail, the user says something else instead.
  • When user switches the subject of the conversation in the middle of the ongoing flow.
  • When the user changes their mind about something they said earlier in the conversation.

Gracefully handling these situations is crucial to provide users with a natural conversational experience while making sure the user achieves their goal.

This ability is called Conversation Repair and is natively supported by CALM. This means the flows designed to handle the most common Conversation Repair cases don’t need to be included in your assistant’s project directory and are supported out-of-the-box. Those cases are:

  • Digressions - when the user shifts from one flow to another in the middle of a conversation. For example, when the user first asks to make a specific money transfer and then switches to another request, for example, asking what is their current account balance mid-way through the money transfer flow.
  • Corrections - when the user updates the information about something said earlier in the conversation. For example, when the user corrects the name of the person they would like to send the money to.
  • Cancellations - when the user halts the flow mid-conversation. For example, when the user starts the money transfer request and the cancels it.
  • Chitchat - when the user engages in the conversation with the assistant that doesn’t affect the actual flow. A typical example of that, is when the user asks the assistant if they are a bot.
  • Completion - when the user concludes the flow by either achieving their initial goal or by abandoning the conversation. For example, when the user asks for a specific amount of money to be transferred and then abandons the conversation when the task is finished.
  • Clarification - when the user’s request cannot be easily identified and matches multiple flows. Very common for short one-word interaction like this. When the user sends an assistant one-word answers it can be difficult for an assistant to confidently evaluate what exactly the user is asking. Those kind of situations would be extremely challenging for NLU-based assistants. LLM-powered assistants are much better at handling those situations by asking the follow-up questions to better understand user’s intent.
  • Internal errors - when the user’s request cannot be handled due to internal system issues.
  • Human handoff - when the user requests human assistance or when the assistant is incapable of handling the user’s request.

Conversation Repair flows

All of these Conversation Repair cases listed above are implemented in CALM out-of-the box. These flows come with the following names:

  • pattern_continue_interrupted for digressions
  • pattern_correction for corrections
  • pattern_cancel_flow for cancellations
  • pattern_chitchat for chitchat
  • pattern_completed for completion
  • pattern_clarification for clarification
  • pattern_internal_error for internal errors
  • pattern_human_handoff for human handoff

The details of each flow can be found in the reference of Rasa's documentation. For example, here’s how the default pattern_human_handoff flow looks like:

pattern_human_handoff:
    description: human handoff
    name: pattern human handoff
    steps:
      - action: utter_human_handoff_not_available

Customizing a Conversation Repair flow

What happens if you want to customize a specific Conversation Repair flow? You can easily do this by modifying the flow in your flows.yml file and adding necessary responses or implement custom actions where necessary. Let’s see an example.

Let’s say we would like to modify what happens when an internal error occurs. First, we will update the flow inside of the flows.yml file as follows:

flows:
  pattern_internal_error:
    description: Internal error
    steps:
      - action: utter_internal_error_response

Next, let’s update the domain.yml and add the action utter_internal_error_response:

responses:
  utter_internal_error_response:
    - text: "There was an internal error. Please, try again later."

Finally, to make sure the flow has been overridden, let’s retrain the assistant: rasa train

Customizing the human handoff flow

Let’s have a look at one more customization example. Human handoff is a feature commonly implemented in conversational assistants. By default, CALM implements human handoff flow by sending the user a response that says, “At the moment, it’s not possible to speak to a human.” If you’d like to actually implement the human handoff, you can do it as follows:

  1. Update the flow. This flow checks the slot for confirming the human handoff action and, if confirmed, performs the human handoff action:
flows:
  pattern_human_handoff:
    description: Human handoff implementation
    steps:
      - collect: confirm_human_handoff
        next:
          - if: slots.confirm_human_handoff
            then:
            - action: action_human_handoff
              next: END
          - else: 
            - action: utter_human_handoff_cancelled
              next: END
  1. Update the responses and actions. Here the custom action action_human_handoff will be your actual implementation for the human handoff. This can result in a live conversation with a customer support agent, a customer support ticket, etc.
slots:
  confirm_human_handoff:
    type: bool
    mappings:
      - type: custom


actions:
  - action: action_human_handoff


responses:
  utter_ask_confirm_human_handoff:
    - text: "Do you want to be connected to a human agent?"
      buttons:
        - payload: "yes"
          title: Yes
        - payload: "no"
          title: No
  utter_human_handoff_cancelled:
    - text: "Ok, I understand you don't want to be connected to a human agent. Is there something else I can help you with?"
      metadata:
        rephrase: True
  1. Retrain your assistant with rasa train.

Conversation repair is a very powerful tool to make your assistant robust against unexpected interactions with the user. Generally, you can treat conversation repair patterns as flows that can either be stored in separate flow files or you can put all of your pattern flows in one file called patterns.yml.

2016-2024 © Rasa.