Bixby Developer Center

This guide covers the different ways that you can steer a conversation within Bixby. In an Input Moment or a Confirmation Moment, you can use prompts. In a Result Moment, you can encourage users to continue conversations in Bixby with conversation drivers or follow-up questions.

You should follow the specific Bixby conversation flow when interacting with users, which is defined and explained further in the Design Guides.

Prompting Users

Bixby takes user-supplied input values and your capsule's models to construct a program that reaches a specific goal, such as displaying tomorrow's weather conditions, or presenting a list of local Italian restaurants. However, a user utterance might not include all the information that Bixby needs to reach its goal. In this case, Bixby might interrupt execution to ask the user for more information. Bixby then has an intermediary step with a recovery goal: it uses the user's response from the prompt to create the type of information needed to continue execution and reach the final goal.

If the user asks "how much are flights to New York City next Friday," the utterance includes neither an arrival nor a departure airport. Bixby can use location services to find the closest major airport to the user and assume that it is the departure airport. It could also use previous preference learning to know that, for example, the user lives in Fremont, California, and prefers to fly out of Oakland rather than San Jose. But there are two airports in New York City: La Guardia and John F. Kennedy. Unless it knows a previously-expressed preference from the user, Bixby will need to prompt the user to choose which airport.

Bixby prompts users when a cardinality violation has occurred: a required input has no value available to it, or an input that can take only a single value has multiple values available to it (such as the airport example above). In general, Bixby tries to prompt users as infrequently as possible. An input could have a default-init that yields a value, it could have selection rules or selection learning that yield a value, or it could have learned a user's preference for a value. The user is prompted for a value when Bixby cannot resolve a cardinality violation on its own.

You can add training examples specifically for prompts.

Types of Prompts

There are three common types of prompts Bixby might give a user:

• A value prompt occurs when a required input is missing a value, and Bixby needs that value to continue execution. In the dice capsule from the Quick Start Guide, if the user asks "Roll dice" and the RollDice action doesn't provide a default for NumDice, Bixby will prompt for the number of dice to roll.

  

• A selection prompt occurs when a max cardinality One input has multiple possible values and the user must select the correct one. This can happen, for example, if the user searches for hotels by city name, and the search action returns multiple possible cities with that name.

  

• A confirmation prompt occurs before Bixby executes a particular action. Generally, it's used before executing actions that have side effects, such as completing a purchase or sending a message.

Confirmation prompts are explicitly generated by your capsule by the confirm key. Value prompts and selection prompts are generated by Bixby when cardinality violations for inputs occur and cannot be resolved without user input:

• When a min (Required) input has no values available, the user will be prompted for that value.
• When a max (One) input has multiple values available, the user will be prompted to select a single value from the available values.

In addition, the min-preferred and max-preferred keys can be used to affect when Bixby prompts for inputs:

• An input with min (Optional) cardinality and min-preference (Required) indicates that an input does not require a value, but having a value is preferred. This will cause Bixby to prompt the user the first time the input is used in an action and has no value. If no value is provided, the action continues without it.
• An input with max (Many) and max-preferred (One) indicates that an input allows multiple values, but prefers just one. If only one value is available to the input, the user will not be prompted. If there are two or more values, Bixby will prompt the user to select one or more items from the available values.

Prompt UX Customization

Your capsules should take advantage of Bixby's personalization and learning abilities to minimize the number of prompts when possible. For example, when the user asks for the weather but doesn't specify a location, it's reasonable to assume they want the weather for their current area; a ride-sharing service capsule could remember that the user always asks for the cheapest available ride, and so might present that as its default choice. However, in some cases—especially involving actions that might lead to purchases, such as booking a ride or a hotel—your capsule should prompt the user for confirmation.

When Bixby does prompt users, though, you should ensure the prompts are clear, easily understood, and presented in the most logical order.

Make sure any text you write for Bixby follows the dialog best practices as well as the Writing Dialog Design Guide.

Here are a few ways to customize prompts:

• Customize dialog
• Set prompt order
• Modify prompt behavior

Dialog Customization

Bixby can generate default dialog for selection and value prompts, but you can refine its dialog by creating elicitation and selection dialog events.

An Elicitation Event generates dialog for a value prompt. The "I need a number of dice to continue" prompt message comes from the default template for a missing input with single value cardinality:

I need #{concept(this, 'Indefinite')} to continue.

You could provide your own dialog event for it to override the default:

dialog (Elicitation) {
  match: NumDice
  template("How many dice do you want to roll?")
}

A Selection Event generates dialog for a selection prompt. The default template for a selection is to present a concept fragment for the input's concept (see the dialog event reference for details). This template can also be overridden with a custom dialog event:

dialog (Selection) {
  match : viv.email.EmailAddress (this)
  // Which email address?
  template("Which #{concept(this)}?")
}

This dialogue uses the concept Expression Language formatting function to format viv.email.EmailAddress as a concept fragment.

Prompt Order

If Bixby needs to prompt for more than one piece of information during execution, the order is determined by the order of the input blocks within the action's collect block. For example, this is part of the RollDice action from the Dice example capsule:

action (RollDice) {
  collect{
    input (numDice) {
      type (NumDiceConcept)
      min (Required)
      max (One)
    }

    input (numSides) {
      type (NumSidesConcept)
      min (Required)
      max (One)
    }
  } 
  output (RollResultConcept)
  type (Calculation)
}

View master on GitHub

An utterance with no values, such as "Roll dice," will cause Bixby to prompt first for the number of dice and then the number of sides for the dice. An utterance missing only one value, such as "Roll 3 dice," will only cause one prompt.

Prompt Behavior

By default, Bixby will not prompt the user unless a cardinality violation has occurred. There are some circumstances, however, in which you might might want to always prompt the user. For an example, the sample basic shopping cart capsule creates a shirt store. The shirt's Size input should always be confirmed, even when the size is mentioned in the utterance. You can change prompting for a specific input through the prompt-behavior key. To always confirm the shirt size, set prompt-behavior on the size input to AlwaysSelection:

action (GetSize) {
  type (Constructor)
  collect {
    input (size) {
      type(Size)
      min(Required)
      max(One)
      plan-behavior (Never)
      prompt-behavior (AlwaysSelection)
      //Lists all available options
      default-init {
        intent {
          goal: Size
          value-set: Size { Size(ExtraSmall) Size(Small) Size(Medium) Size(Large) Size(ExtraLarge) }
        }
      }
    }
  }
  output (PromptedSize) {
    evaluate { $expr(size) }
  }
}

View master on GitHub

This setting ensures the user will always be prompted to either supply a missing value or confirm it. Typically, this is the only behavioral mode you will use in input blocks other than default, which will only prompt the user to supply values when there is a cardinality violation (there are no values supplied, or there are two or more values supplied for a max (One) input).

Confirmations

As described in Enriching the Conversation, Bixby offers three kinds of confirmation prompts:

• Persistence Confirmation: Asking the user if Bixby should remember a value and use it as a default in the future
• Action Confirmation: Confirming the user wants to take a specific action
• Input Adjustment: Confirming Bixby's understanding of the user's input values and allowing refinement

As with selection and value prompts, you can override Bixby's defaults by supplying new templates for dialog events.

A Storage Event generates dialog for a persistence confirmation prompt, asking the user if they wish to store a value.

dialog (Storage) {
  match: viv.payment.CreditCardPaymentSource (this)
  template ("Would you like us to save this #{concept (this)} for future use?")
}

A Confirmation Event generates dialog for an action confirmation prompt, asking the user if they wish to proceed with an action.

dialog (Confirmation) {
  match: viv.paymentService.MakePayment
  template ("Ready to send?")
}

Actions can be set to explicitly require confirmation through the use of the confirm key, which calls an appropriate confirmation-view that you create. You can use confirmation layouts to create more complex confirmations or to override the default layouts. If all you need to do is create a simple confirmation, you can choose to render nothing.

For the more complex interactions, such as presenting a restaurant booking screen that lets the user verify and, if necessary, change the reservation date, time, and party size, you can render a layout within the confirmation view.

If you are in a confirmation prompt, you can set the modal-prompt key in the confirmation view to true to ignore any off-topic responses and re-prompt for the confirmation.

Permission Prompts

Bixby can also prompt a user to grant a capsule permission. Users can respond both by tapping "Yes" or "No", or by speaking a response. Permission prompts are not customizable.

Continuing Conversations

When a user starts a conversation with Bixby, the conversation might continue based on Bixby's response. For instance, after asking for nearby restaurants, the user might want to see the restaurants in a map view rather than in a list. After the user has selected a specific restaurant, they might want to make a reservation, call the restaurant, or get directions to it.

Bixby provides ways for your capsule to drive the conversation forward from a result list. Conversation drivers let you offer shortcut buttons that work with continuations; follow-up questions allow Bixby to ask yes or no questions after result views, and take specific actions based on the response.

When you have Bixby continue conversations with the user, you should follow the appropriate Dialog Design Patterns to prompt users, and continue to follow dialog best practices.

Conversation Drivers

Imagine that you have a capsule that provides restaurant results.

When users search for nearby pizza places, your capsule could show a list with results. However, you could also provide a "View map" button at the bottom, allowing users to quickly see the results on a map. If the user selects a specific restaurant, the capsule could then provide buttons in a similar fashion to reserve a spot at the restaurant or get directions to the place.

To offer users convenient shortcuts to related actions like this, you can use Conversation Drivers.

You add Conversation Drivers to views using the conversation-drivers parent key. In this example that allows users to book a space resort, the conversation drivers provide a quick way for users to go to the booking flow after looking at the details of a particular space resort:

result-view {
  // This view is used to show the SpaceResort details when the user select a space resort from a summary list. This follows the design paradigm to go from Summary to Details
  match {
    SpaceResort (result)
  }

  render {
    // We know the size is always 1 because this view is only attainable when drilling into a single item to see the details
    // Lists of space resorts are handled in the ViewAll_Result and Input files
    if (size(result) == 1) {
      layout {
        macro (space-resort-details) {
          param (spaceResort) {
            expression (result)
          }
        }
      }
    }
  }

  conversation-drivers {
    if ("size(result) == 1") {
      conversation-driver {
        macro (MakeReservation)
      }
    }
  }
}

View master on GitHub

The resulting button appears at the bottom of the screen.

When users tap on this button, Bixby effectively runs a new utterance using the new template text. These utterances can be trained as continuations.

You can use conversation drivers in input-view and result-view, as well as detail-view in transactional workflows under the state parent key. In order to ensure that your user is in capsule lock while they are in a result moment, you need to add a conversation driver in the result view.

Follow-Up Questions

A followup allows Bixby to ask a yes or no question after a result view is rendered, specifying behaviors for when the follow-up is confirmed (a "yes" answer) or denied ("no").

Let's continue the example of a capsule that searches for restaurants, and the user has made a request such as "show me the closest coffee shop". After Bixby shows the result, several things could happen next to continue the conversation:

• The user gives a new utterance trained as a continuation, such as "call the shop" or "give me directions".

• The user taps a button for a supplied conversation driver to take an action.

• The user does nothing, and the conversation ends.

However, you might want Bixby to simply ask the user, "Would you like directions to the shop?"

• If the user says "yes", Bixby gives directions.

• If the user says "no", the conversation ends.

"Yes" and "no" utterances can't be trained as continuations, so this can't be implemented as a conversation driver. Instead, it can be implemented as a follow-up.

result-view {
  match: Business (this)

  message ("I found #{value(this.name)}")

  render {
    layout-match (this) {
      mode (Details)
    }
  }

  followup {
    prompt {
      dialog (Would you like directions?)
      on-confirm {
        intent {
          goal: NavigateTo
          value: Business$expr(this)
        }
      }
      on-deny {
        message (Okay.)
      }
    }
  }
}

This result view first renders, with layout-match, a layout for the Business concept. The follow-up begins in the followup block. This block has only one optional child, prompt, which specifies the prompt dialog and its possible behaviors.

• dialog provides the actual message of the follow-up question: Would you like directions?

• on-confirm can provide either an intent to execute or a message to display if the follow-up is confirmed (the user says "yes"). In this example, a new intent is created with a goal of NavigateTo and a value of the current Business model. (The value uses the $expr() type coercion construct; for more information, read the Expression Language Reference.)

• on-deny provides either an intent or a message to display if the follow-up is denied (the user says "no"). In this example, a message is provided.

  The on-deny block is optional. If it's not provided, then a denial of the follow-up will by default do nothing.

While follow-ups prompt the user for action, they are not prompts in the sense Bixby often uses the term: there is no learning, the prompts are not modal, and so on. The user could respond to a follow-up prompt with an utterance that triggers a continuation, or an entirely new request.

Capsule Lock and Modality

Being locked into the current conversation with the capsule is known as capsule lock, and it ensures that subsequent utterances are always sent to the current capsule for a short amount of time. When a user is at a prompt, utterances will be locked to the type being prompted for, and the user must answer the prompt to continue. Also, if there are follow ups (such as conversation drivers or followup questions), the user is locked into a conversation with the current capsule for a short amount of time.

How long capsule lock lasts depends on the moment and if users are in hands-free mode mode.

• If users are in a result view:
  • In hands-on mode, capsule lock is ~15 seconds.
  • In hands-free mode, capsule lock is ~5 seconds.
• If users are at a prompt:
  • In hands-on mode, capsule lock is ~10 seconds.
  • In hands-free mode, users are re-prompted after ~5 seconds. If they don't respond within ~7 seconds of the re-prompt, capsule lock ends.

When Capsule Lock Is Active

Users know they are locked into a capsule on a mobile device when they see the message "You're now talking to %capsule-name%" in the action zone of the screen. For example, users are locked into a conversation with the Meme Generator capsule below:

Capsule lock is triggered under the following conditions:

• If there is a followup or conversation driver in the result view
• If the user needs to answer a prompt in an input or confirmation view
• If the user is in a list navigation using hands-free mode

Capsule lock ends when users do any of the following:

• Exit the UI by tapping the X
• Close the Conversation View
• Navigate away from Bixby (for example, by pressing the home button)
• Do not interact with the capsule even after a second prompt (hands-free mode only)
• Say an utterance that specifies a capsule through named dispatch, even if users are at a prompt

Off-Topic Utterances

During capsule lock, if a user's response to a prompt cannot be interpreted as a valid response to the prompt (that is, if it cannot connect a value in the utterance to the current goal), the response will be classified as "off-topic". Bixby will then try to re-interpret the off-topic utterance in the context of the current capsule, considering all queries and relevant continuations again. For instance, if a weather capsule prompts the user to select between multiple options ("Do you mean Portland, Oregon or Portland, Maine") and the user instead says "what's the weather", Bixby re-interprets the utterance and classifies it as a weather intent that is locked within the capsule. After the capsule lock expires, utterances are treated as "brand new" and will no longer be limited to the weather capsule.

You can force Bixby to re-interpret an utterance non-modally using the flag-as-off-topic effect.

action (SelectSearchRegion) {
  ...
  output (SearchRegion) {
    on-empty {
      if (!exists(regionCode)) {
        flag-as-off-topic
      }
    }
  }
}

Override Capsule Lock

You can opt out of capsule lock by overriding the result-view-capsule-lock flag in your runtime version and setting it to false. If you need to opt out of capsule lock only during specific result moments, you can use the capsule-lock key in that particular result view.

If your capsule is on the Marketplace and you haven't used the result-view-capsule-lock flag to opt out of capsule lock, conversation drivers in result views only display if the device itself also supports capsule lock.

If you do opt out of capsule lock, conversation drivers will still appear on the result view page, but they might not behave as expected. This is true even if the device does not support capsule lock.