Use slot filling in fulfillment

December 5, 2018

Overview

Slot filling simplifies your agent by letting you obtain multiple required parameters within a single intent. If any parameters are missing from a user utterance, your agent will automatically prompt users for the required parameters—either dynamically through your webhook or statically for one missing parameter at a time.

This post walks you through creating a slot-filling agent that constructs an Adlibs story using a country, year, and color the user specifies. You’ll learn all the necessary steps and considerations to have your agent use slot filling, with or without fulfillment.

Here, however, we will more closely examine slot filling with fulfillment.

The basic steps to slot filling include the following:

  1. Specifying annotated training phrases that include the parameters you want to extract from a user utterance. These parameters are the "slots" that you want users to "fill" for a particular intent.
  2. 1. Providing a few static prompt phrases for each parameter (not using fulfillment for parameter collection). Dialogflow uses one of the prompt phrases for each parameter to ask users for missing slots, individually.

    OR

    2. Having your fulfillment handle the missing slots— this lets you set up customized logic depending on which combinations of slots are missing, minimizes the number of responses back to the user, and allows you to create dynamic prompt phrases, which enables you to capture multiple missing parameters.
  3. Note: This sample follows B2. You can use option 1 for simplicity but option 2 provides a better user experience in most cases.
  4. Returning a final response only after all the required slots are filled (the final response can also be defined in your webhook).

Benefits of using a webhook for slot-filling

Instead of letting Dialogflow handle the prompting of parameters, you can route any slot filling logic in your fulfillment, which enables you to return dynamic prompts. Consider the following benefits of using fulfillment to handle slot filling:

  • You can access the values of required parameters from user utterances through the automatic setting of output contexts field in the fulfillment request JSON object.
  • You can use the incomplete data that the slot-filling feature provides to customize the response to the user in order in order to obtain the outstanding parameters.
  • You can control slot-filling behavior to end the conversation gracefully if the user doesn’t provide all the necessary information within a reasonable amount of attempts. (By default, Dialogflow repeatedly asks users to fill missing slots, regardless of how many failed attempts.)
  • You can re-route users to another piece of logic if they are having trouble filling the required slots.

Quick deploy option

If you’d prefer to skip the step-by-step instructions and additional information, you can go ahead and use our pre-made template below to configure your slot-filling agent. You can elect to do this option and follow along, but ensure that the Firebase function is deployed from the Fulfillment section of the agent.

Intents setup

In this section, we’ll go over how to configure and customize your agent’s intents.

Customize Default Welcome Intent

First, you need to initialize a custom welcome response inside the Default Welcome Intent that tells users what information our agent needs to proceed. To do so, follow these steps:

  1. In the Responses section of the Default Welcome Intent, add the following: Hey there, welcome to Adlibs! All I need is a country, year, and color for a great story.
  2. Click SAVE.

Create makeAdlibStory intent

You’ll now create and modify the main intent that will be doing the slot filling work, which contains the agent’s training phrases and parameters for collecting the required information. To create this intent, follow these steps:

  1. Click on the + next to Intents in the Dialogflow console.
  2. Name the intent makeAdlibStory.
  3. Click SAVE.

Add training phrases

Next, you need to add some training phrases for the agent. These are utterances you would reasonably expect from users. You can add as many training phrases as you see fit; the more training phrases you add, the more your agent will understand. It is recommended to use at least 10-15 phrases. To add training phrases, follow these steps:

  1. Navigate to the Training phrases section of the makeAdlibStory intent.
  2. Add the following training phrases:
    • Canada green 1955
    • Orange iceland 1985
    • Brazil 1970 yellow
  3. Click SAVE.

alt_text

Annotate training phrases

The highlighted words in your training phrases represent entities that are extracted as parameters at runtime. If you are using built-in system entities (@sys.date, @sys.time, @sys.color, etc.), then you can bypass needing to manually input them in the entities section of the console. Entities allow you to annotate phrases so that Dialogflow can better parse data from user utterances. In essence, entities categorize data so it can be parsed more easily.

In general, Dialogflow entities allow for automatic annotation of the training phrases.

The following entities within each training phrase are automatically annotated:

  • @sys.geo-country
  • @sys.number
  • @sys.color

The related parameters are also created. For every part of a training phrase that you annotate with an entity, a corresponding parameter stores the extracted value from the utterance. In the Action and parameters table, you can see your entities and associated parameter names, as shown in the following screenshot:

alt_text

Make parameters required

Based on the welcome message, the agent has three required slots to fill. Each required parameter should be marked as “required” in order to ensure the information is collected. To mark the parameters as required, do the following:

  1. Navigate to the Action and parameters section of the makeAdlibStory intent.
  2. Check the REQUIRED box for all three of the parameters.

These parameters are passed to fulfillment if fulfillment is enabled for that intent. You can then use these parameters to process information and repeat them back in your response, specifically in this case to form an adlib.

Add static prompts for each required parameter

If your fulfillment fails, then Dialogflow returns static responses instead, so if this occurs then you know the webhook is down. Since these parameters are marked as required, they must be captured. These responses are specific prompts to the user to collect the missing parameter. Not only is this a good way to tell if your fulfillment is working, but the conversation with your users won’t be disrupted in the case of fulfillment failure. To add static responses, do the following:

  1. To add a prompt for each required parameter, click Define prompts under the Prompts column of the Action and parameters table. (The Prompts column is revealed when you click the REQUIRED option.)
  2. Add the following prompts to each required parameter:
    • geo-country parameter: Oops, you forgot $geo-country
    • number parameter: You forgot $number, give me any number
    • color parameter: Please tell me a $color

Add response as backup

Like with prompts for required parameters, a response provided in the Dialogflow console will be used in the case of fulfillment failure. To add a backup response, do the following:

  1. Navigate to the Responses section of the makeAdlibStory intent.
  2. Add the following response: So according to Professor David, back in the year $number, the nearly extinct yet exquisite, $color -bellied fox was introduced to $geo-country and now flourishes in over 236 distinct regions.
  3. Enable Set this intent as end of conversation. This will make sure your agent isn’t expecting further input from the user.*
  4. Select Enable webhook call for this intent as well as Enable webhook call for slot filling.

    *Note: You can include the response in your fulfillment webhook (index.js), in your makeAdlibStory intent, or both. If you choose to include a final response in both, then the response set in the intent will be run as a backup only if fulfillment fails. We will discuss that to a greater extent later in this post.

alt_text

Set up fulfillment

In the fulfillment index.js snippet shown below, you can see that the fulfillment response to the user, once all the required slots are filled, is slightly different from the response we declared under the intent makeAdlibStory. The final response set inside the intent acts as a fail-safe in case your fulfillment isn’t working properly—it is entirely optional to have this redundancy since we declared the final response (lines 46-48) in the fulfillment.

In addition, you can see the makeAdlibs function handler is mapped to the intent created earlier, makeAdlibStory, in lines 52-54.

alt_text

Below are the step by step instructions for enabling fulfillment, which allows for dynamic prompting to users for missing parameters within the makeAdlibStory intent. The power behind slot filling largely comes from the basis that this can be done within a single intent; also, output contexts are automatically created and managed.

  1. In the Dialogflow console, go to Fulfillment in the left-menu and toggle the button next to Inline Editor to ENABLED. This automatically creates a Cloud Function for Firebase to use as fulfillment.
  2. Under makeAdlibStory intent, select Enable webhook call for this intent AND Enable webhook for slot-filling at the bottom. Click SAVE.
  3. Navigate back to Fulfillment and copy and paste the following into the index.js tab of the inline editor:

    'use strict';
    
    const admin = require('firebase-admin');
    const functions = require('firebase-functions');
    const {WebhookClient} = require('dialogflow-fulfillment');
    
    process.env.DEBUG = 'dialogflow:debug';
    
    const ADLIBS_INTENT = 'makeAdlibStory';
    
    exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
    const agent = new WebhookClient({request, response});
    console.log('Req headers: ' + JSON.stringify(request.headers));
    console.log('Req body: ' + JSON.stringify(request.body));
    
    function makeAdlibs(agent) {
        const [color, nation, year] = [agent.parameters['color'], agent.parameters['geo-country'], agent.parameters['number']];
    
        let missingSlots = [];
        if (!color) { missingSlots.push('color'); }
        if (!nation) { missingSlots.push('nation'); }
        if (!year) { missingSlots.push('year'); }
    
        if (missingSlots.length === 1){
        agent.add(`Looks like you didn't provide a ${missingSlots[0]}`);
        }
        else if (missingSlots.length === 2){
            agent.add(`Ok, I need two more things, a ${missingSlots[0]} and ${missingSlots[1]}`);
        }
        else if (missingSlots.length === 3){
            agent.add(`Ok, I need all 3 things still: a ${missingSlots[0]}, ${missingSlots[1]}, and ${missingSlots[2]}`);
        } else {
        agent.add(`So according to Professor David, back in the year
        ${year}, the nearly extinct yet exquisite,
        ${color}-bellied fox was introduced to ${nation} and now flourishes in over 361 regions.`);
        }
    }
    
    let intentMap = new Map();
    intentMap.set(ADLIBS_INTENT, makeAdlibs);
    agent.handleRequest(intentMap);
    });
    
  4. Copy and paste the following into the package.json tab of the inline editor:

    {
    "name": "dialogflowFirebaseFulfillment",
    "description": "This is the default fulfillment for a Dialogflow agents using Cloud Functions for Firebase",
    "version": "0.0.1",
    "private": true,
    "license": "Apache Version 2.0",
    "author": "Google Inc.",
    "engines": {
        "node": "6"
    },
    "scripts": {
        "start": "firebase serve --only functions:dialogflowFirebaseFulfillment",
        "deploy": "firebase deploy --only functions:dialogflowFirebaseFulfillment"
    },
    "dependencies": {
        "actions-on-google": "2.2.0",
        "firebase-admin": "^5.13.1",
        "firebase-functions": "^2.0.2",
        "dialogflow-fulfillment": "0.6.1"
    }
    }
  5. Click Deploy at the bottom of the inline editor.

Test agent in simulator

You can now converse with your agent using Dialogflow’s simulator within the console. In the Try it now text field on the right side of the console, type hi to begin a conversation with your agent, and then provide input based on the agent’s prompts:

Screenshot of simulator showing working agent Screenshot of simulator showing working agent

Further reading

If you’re interested in seeing more slot filling, there are two Dialogflow slot-filling samples up on Github: Telephony & Google Sheets and Basic Slot Filling. You can also check out the official documentation on slot-filling, as well as the documentation about different entity types.

Now, you have all that you need to know to make a Dialogflow agent with slot filling!

Posted by Sarah Dwyer, Developer Relations Quality Engineer, and Lauren Ward, Technical Editor





Create and manage entities with API

November 12, 2018

Overview

One of Dialogflow’s most powerful features is its ability to recognize and extract entities from user input, which are important pieces of information like words or phrases corresponding to dates, times, colors, and more. Generally, a Dialogflow agent has a defined collection of entities called system entities that it can detect in user input, and you can also create developer entities to represent custom concepts. But what if you want to make a particular set of entities available based on what the user says? We can do this kind of tailoring using session entities, which we implement through the Agent API. Session entities are tied to a user’s particular conversation with your agent.

In this blog post, we’ll use both developer entities and session entities to help create a “Street Smarts” trivia game that quizzes users on famous street names in a few cities. In our game, the user selects a city, and our Dialogflow agent then loads a set of session entities for that particular city. These session entities correspond to street names in the selected city. For example, if a user picks “Chicago”, the agent recognizes Chicago’s street names in the next utterances, but not streets that are associated with other cities. The following is a sample dialog from our game:

       User: Hello

       Agent: Hey! Choose a city and I’ll test your street smarts. Your options are New York, Los Angeles, Chicago, and Houston.

       User: Los Angeles

       Agent: Great! I love Los Angeles. Here’s a question about its streets! What street in Beverly Hills boasts some of the most expensive shops in the world?

       User: Rodeo Drive

       Agent: Nice work! You got the answer right. You’re truly an expert on Los Angeles. Give me another city and I’ll ask you more questions.

To create this game, we’ll do the following:

  • Part 1: Create entities via the Agent API. To do this, you’ll write a create.jsscript that creates new entities and run it via the command line.
  • Part 2: Update entities via the Agent API with an update.jsscript that updates entity values and is run via the command line.
  • Part 3: Create the necessary intents and contexts for the game in the Dialogflow console.
  • Part 4: Write fulfillment code in an index.js file to implement the logic of the game and create session entities at runtime.

Note: This post walks you through both writing the code for the “Street Smarts” trivia game and creating and modifying intents and contexts in the Dialogflow console. If you’d like, you can import the fully built agent and download the completed code in this GitHub repo. You can also click the Add to Dialogflow button to import the fully created agent and its fulfillment.

Basic setup

Before we begin creating our “Street Smarts” trivia game, make sure to install Node.js (a JavaScript runtime we’ll use to run our scripts) and the Dialogflow client library (which allows us to connect to the agent API with JavaScript):

  1. Install a working version of Node.js. Make sure that npm is also installed (it typically comes with Node.js).
  2. Create a file called package.jsonin a new directory we’ll be using for this project. Copy and paste in the following JSON to use the Dialogflow client library:

    {
      "name": "creating-entities-blog",
      "version": "1.0.0",
      "description": "",
      "main": "app.js",
      "scripts": {
        "test": "echo \"Error: no test specified\" && exit 1"
      },
      "author": "",
      "license": "ISC",
      "dependencies": {
        "dialogflow": "^0.6.0"
      }
    }
  3. Run npm installin your terminal to install the dependency.

Create agent in the Dialogflow console

First, we need to create an agent for our “Street Smarts” trivia game in the Dialogflow console (we’ll create and modify our entities programmatically, but we’ll create our agent in the console). Later on in this tutorial, we’ll also add intents and fulfillment in the Dialogflow console.

To create an agent, follow these steps:

  1. Go to the Dialogflow console.
  2. If you’re a new user, click on Create new agent in the left menu.

    If you have existing agents, click on the current agent name in the left menu and click Create new agent.

  3. Name the agent anything you’d like (we named ours city-streets-trivia).

  4. Click CREATE.

  5. Download the credentials file to confirm your identity to the Dialogflow API. To get it, follow these instructions about setting up authentication, but choose ‘API Admin’ instead of ‘API Client’ in step 7.

  6. Copy the credentials file you downloaded in step 5 into the same directory as package.json and rename it to credentials.json. All the scripts we write in this blog post should be in this directory.

Create entities via API (create.js)

We’ll use Dialogflow’s Agent API to create the two entities we need for our “Street Smarts” trivia game:

  • A ‘city’ entity, whose values correspond to city names
  • A ‘street’ entity, which is a placeholder and contains only one value (it will be overwritten later when we use session entities to represent streets)

To create this script, do the following:

  1. Create a file in your IDE or text editor called create.js.
  2. Copy and paste each code snippet from this section into create.js.Alternatively, you can download the entire code repository.

Note: Visit the V2 Agent API reference for an overview of all the API endpoints.

Add boilerplate code

First, we need to go through some basic setup. In the following code snippet, we:

  • Set up our code to use the Dialogflow client library
  • Provide service account credentials to allow us to access the Agent API and modify the agent
  • Create a new EntityTypesClient, which communicates with the EntityTypes API endpoints so we can create new entities
  • Create a path string for our agent based on its Project ID, which you can find in the left console navigation under Settings ⚙ —> Project ID. This path string tells Dialogflow which agent to modify.
'use strict';

const dialogflow = require('dialogflow');

// Read in credentials from file. To get it, follow instructions here, but
// choose 'API Admin' instead of 'API Client':
// https://dialogflow.com/docs/reference/v2-auth-setup
const credentials = require('./credentials.json');

const entitiesClient = new dialogflow.EntityTypesClient({
 credentials: credentials,
});

const projectId = '<INSERT YOUR PROJECT ID HERE>';
const agentPath = entitiesClient.projectAgentPath(projectId);

Define ‘city’ entity

We’ll now define the EntityType for city. The cityEntityType object includes the following:

  • displayName - the property that corresponds to the name you’d give an entity in the Dialogflow console
  • kind - the property that defines whether or not the entity has synonyms (in this case, our city names have synonyms to match a greater variety of user input). KIND_MAP specifies that the entity values have synonyms.
  • entities - defines the values within the ‘city’ entity and associated synonyms. For now, we’re only adding New York and Los Angeles.

Note: In the Dialogflow console, we would refer to city as an “entity” and New York and Los Angeles as “values”. Programmatically, however, we refer to city as an “EntityType”, and refer to New York and Los Angeles as examples of an “Entity.” In this blog post, we use “entity” and “EntityType” interchangeably.

const cityEntityType = {
 displayName: 'city',
 kind: 'KIND_MAP',
 entities: [
   {value: 'New York', synonyms: ['New York', 'NYC']},
   {value: 'Los Angeles', synonyms: ['Los Angeles', 'LA', 'L.A.']},
 ],
};

Create ‘city’ entity

Now that we’ve defined our ‘city’ entity, we can call the projects.agent.entityTypes.create API endpoint to create it. First, we need to build a request object that includes the following properties:

  • agentPath - the value that was defined in our first code snippet and tells Dialogflow where to find our agent
  • cityEntityType - the name of the EntityType

We can now create our EntityType and make sure it was created correctly by doing the following:

  • Calling createEntityTypeand passing in cityRequest to create our EntityType for city
  • Logging the newly created EntityType in the promise chain (if you are unfamiliar with promises, see JavaScript Promises: an Introduction. responses[0]contains information about the EntityType for city.
const cityRequest = {
 parent: agentPath,
 entityType: cityEntityType,
};

entitiesClient
   .createEntityType(cityRequest)
   .then((responses) => {
     console.log('Created new entity type:', JSON.stringify(responses[0]));

Define and create ‘street’ entity

Now, let’s define and create our second entity, ‘street’. Just as with the ‘city’ entity, we do the following:

  • Define a streetEntityType object to represent streets
  • Construct the request object
  • Return details about the new EntityType

However, as you may notice, streetEntityType contains only one street name. The ‘street’ entity is currently a placeholder, and will be fully populated with entity values when Dialogflow calls our fulfillment code, which in turn calls the Agent API. We need to define it now so we can use the ‘street’ entity in our training phrases, which we will be doing later in this tutorial.

    const streetEntityType = {
      displayName: 'street',
      kind: 'KIND_MAP',
      entities: [
        {value: 'Broadway', synonyms: ['Broadway']},
      ]
    };

     const streetRequest = {
       parent: agentPath,
       entityType: streetEntityType,
     };

     return entitiesClient.createEntityType(streetRequest);
   })

   .then((responses) => {
     console.log('Created new entity type:', JSON.stringify(responses[0]));
   })

Log errors

Let’s log any errors that may have occurred during entity creation:

   .catch((err) => {
     console.error('Error creating entity type:', err);
   });

Run create.js script

Run node create.js in your terminal to run the script. The highlighted output shows details about the newly created EntityTypes, ‘city’ and ‘street’:

$ node create.js
Created new entity type: {"entities":[{"synonyms":["New York","NYC"],"value":"New York"},{"synonyms":["Los Angeles","LA","L.A."],"value":"Los Angeles"}],"name":"projects/city-streets-trivia/agent/entityTypes/0d24414d-6e41-463b-990d-695147d1bec4","displayName":"city","kind":"KIND_MAP","autoExpansionMode":"AUTO_EXPANSION_MODE_UNSPECIFIED"}
Created new entity type: {"entities":[],"name":"projects/city-streets-trivia/agent/entityTypes/989eeee4-06b1-4e50-92b3-0dea0247cb1b","displayName":"street","kind":"KIND_MAP","autoExpansionMode":"AUTO_EXPANSION_MODE_UNSPECIFIED"}

How it looks in the Dialogflow console

Once you’ve run create.js, the ‘city’ and ‘street’ entities appear in the Entities tab of the Dialogflow console, as shown in the following screenshot:

alt_text

Update an entity via API (update.js)

In the last section, we created a ‘city’ entity that contained only two city names: New York and Los Angeles. We’ll now update the entire ‘city’ entity list by calling the projects.agent.entityTypes.patch endpoint to replace all the previous city names with new ones. In this example, we’re hardcoding the city names; however, you could easily pull city names from a datastore and continually update the list.

To create a script that updates entities, do the following:

  1. Create a file called update.js.
  2. Copy and paste the code from this section into the file. Alternatively, you can download the entire code repository from this GitHub repo.

Add boilerplate code

First, we’ll add the same boilerplate code that we did in create.js:

'use strict';

const dialogflow = require('dialogflow');

const credentials = require('./credentials.json');

const entitiesClient = new dialogflow.EntityTypesClient({
 credentials: credentials,
});

const projectId = '<INSERT YOUR PROJECT ID HERE>';
const agentPath = entitiesClient.projectAgentPath(projectId);

Find the EntityType to update

Next, we’ll find the EntityType we want to update by doing the following:

  • Creating a custom error class (EntityNotFoundError) that is thrown if we can’t find the correct entity
  • Calling the listEntityTypes method to list all EntityTypes
  • Iterating through the EntityTypes and determining if the ‘city’ entity is there
/** Define a custom error object to help control flow in our Promise chain. */
class EntityNotFoundError extends Error {};

entitiesClient
   .listEntityTypes({parent: agentPath})
   .then((responses) => {
   // The array of EntityTypes is the 0th element of the response.
     const resources = responses[0];
     for (let i = 0; i < resources.length; i++) {
       const entity = resources[i];
       if (entity.displayName === 'city') {
         return entity;
       }
     }
     throw new EntityNotFoundError();
   })

Update ‘city’ EntityType

Now, we can update ‘city’ with a new list of entities by doing the following:

  • Defining the new entity values for ‘city’ with an array of objects (city names and their synonyms)

    Note: This information is hardcoded here, but you could get it from a database if you wanted.

  • Making the request to projects.agent.entityTypes.patch to update the EntityType for ‘city’. We use updateMask to make sure we only update the entity’s values and not any other fields of the EntityType.

  • Logging the updated ‘city’ EntityType

  • Catching any errors and logging a helpful message if our custom error was thrown

  .then((city) => {
     console.log('Found city: ', JSON.stringify(city));
     const updatedEntityList = [
       {value: 'New York', synonyms: ['New York', 'NYC']},
       {value: 'Los Angeles', synonyms: ['Los Angeles', 'LA', 'L.A.']},
       {value: 'Chicago', synonyms: ['Chicago']},
       {value: 'Houston', synonyms: ['Houston']},
     ];
     city.entities = updatedEntityList;

     const request = {
       entityType: city,
       updateMask: {
         paths: ['entities'],
       },
     };
     return entitiesClient.updateEntityType(request);
   })
   .then((responses) => {
     console.log('Updated entity type:', JSON.stringify(responses[0]));
   })
.catch((err) => {
   // If this is the error we throw earlier in the chain, log the
   // cause of the problem.
     if (err instanceof EntityNotFoundError) {
       console.error('Could not find the entity named city.');
       return;
     }
     console.error('Error updating entity type:', err);
   });

Run update.js script

Run node update.js in your terminal to run the script. The output should show the updated entity values, as highlighted below:

$ node update.js
Found city: {"entities":[{"synonyms":["New York","NYC"],"value":"New York"},{"synonyms":["Los Angeles","LA","L.A."],"value":"Los Angeles"}],"name":"projects/city-streets-trivia/agent/entityTypes/0d24414d-6e41-463b-990d-695147d1bec4","displayName":"city","kind":"KIND_MAP","autoExpansionMode":"AUTO_EXPANSION_MODE_UNSPECIFIED"}
Updated entity type: {"entities":[{"synonyms":["New York","NYC"],"value":"New York"},{"synonyms":["Los Angeles","LA","L.A."],"value":"Los Angeles"},{"synonyms":["Chicago"],"value":"Chicago"},{"synonyms":["Houston"],"value":"Houston"}],"name":"projects/city-streets-trivia/agent/entityTypes/0d24414d-6e41-463b-990d-695147d1bec4","displayName":"city","kind":"KIND_MAP","autoExpansionMode":"AUTO_EXPANSION_MODE_UNSPECIFIED"}

How it looks in the Dialogflow console

Now, when you click on the ‘city’ entity in the Dialogflow console, the updated list of city names should appear:

alt_text

Customize default intents in the console

Now that we’ve created our entities, let’s do some work within the Dialogflow console. To begin, we’ll customize our default intents to both welcome our users and respond to unrecognized user input.

Modify Default Welcome Intent

First, we’ll modify the Default Welcome Intent to welcome our users to our “Street Smarts” trivia game.

In the Dialogflow console, follow these steps:

  1. Click on Intents in the left navigation.
  2. Click Default Welcome Intent.
  3. Scroll down to the Text response section and delete the preconfigured responses.
  4. Add this response: Hey! Choose a city and I'll test your street smarts. Your options are New York, Los Angeles, Chicago, and Houston.

Modify Default Fallback Intent

Next, let’s tailor the Default Fallback Intent to respond to unrecognized cities in our “Street Smarts” trivia game. We’ll add a prompt in the response to guide users towards providing expected city names.

In the Dialogflow console, follow these steps:

  1. Navigate to the Default Fallback Intent.
  2. Scroll down to the Text response section and delete the preconfigured responses.
  3. Add this response: I don't know that city! Your options are New York, Los Angeles, Chicago, and Houston. Which one do you want?

Create intents in the console

Let’s add custom intents in the Dialogflow console to handle user selections and incorrect guesses for our “Street Smarts” trivia game. Later on in this tutorial, we’ll map these custom intents to fulfillment and add our agent’s responses in our index.js file.

Add City name intent

We need to create a custom intent in the console to recognize the user’s city selection. To do so, follow these steps in the Dialogflow console:

  1. Click the + button next to Intents in the left navigation.
  2. Name the intent City name.
  3. In the Training phrases section, add the following training phrases (we’d normally want around 10 examples, but use less here for simplification purposes):
    • New York
    • How about Chicago?
    • I'll pick Houston
    • Let's do Houston
  4. Scroll down to the Text response section and add this response: Sorry, there was a problem. Try again later!
  5. Scroll down to the Fulfillment section and toggle on Enable webhook call for this intent.

Add follow-up to City name intent

Next, we’ll add a follow-up intent to our City name intent that recognizes the street name our user answers with. To do so, follow these steps in the Dialogflow console:

  1. In the Intents list, hover your cursor over the City name intent and click Add follow-up intent, then click custom.
  2. Name the follow-up intent Trivia answer.
  3. In the Training phrases section, copy and paste the following training phrases:
    • Broadway
    • I think it's Broadway
    • Maybe it's Broadway
    • I'm not sure but it might be Broadway
    • How about Broadway
  4. In the Text response section, add the response Oh dear! Something went wrong. What did you say again?
  5. Scroll down to the Fulfillment section and toggle on Enable webhook call for this intent.

Add a custom fallback intent

We’ll now add a custom fallback intent to capture incorrect street name guesses that aren’t in our list of known street names (we will later define our known street names in index.js). To do this, follow these steps in the Dialogflow console:

  1. Create a new fallback intent by doing the following:
    1. Click on Intents in the left navigation.
    2. Click the three vertical dots alt_text next to CREATE INTENT and then click Create Fallback Intent.
    3. Name it Unknown answer.
  2. Create a new intent and name it Unknown answer.
  3. In the Contexts section, do the following:
    1. Add Cityname-followup as an input context. We do this so the fallback will only be matched if the user is currently asking for a street name.
    2. Add Cityname-followup as an output context and set the lifespan to 2. We do this so that, if the user gives an unrecognized street name and matches this intent, we’ll still be in the right context to match more street names.
  4. In the Responses section, add this response: Hmmm, I don't actually know that one, but it's definitely not the right answer. Try again!

Write fulfillment code (index.js)

We’ve now reached the point where we can write our fulfillment code in an index.js file that we’ll copy and paste into the Cloud Functions for Firebase inline editor within Dialogflow. This portion of the code is where we actually interact with our data— in other words, this is where we load the data for a particular city and use it to create session entities (street names) at runtime.

We then ask a trivia question about the city the user chose, check whether the user’s answer is correct, and generate a dynamic response. In this case, we hard-code the data about the cities and their streets, but we’d usually get this from a datastore.

The two main functions in our fulfillment code are the following:

  • askTriviaQuestion - defines and creates session entities (street names)
  • checkTriviaAnswer - checks the user’s street answer against the correct answer for that city

To create our fulfillment code, do the following:

  1. Create a file called index.js.
  2. Copy and paste the code from this section into the file. Alternatively, you can download the entire code repository.

Enable inline editor for fulfillment

To be able to modify the default code within the inline editor, follow these steps:

  1. Click the Fulfillment tab on the left side of the Dialogflow console.
  2. Toggle the switch for Inline Editor to ENABLED.

alt_text

Modify package.json file

The inline editor contains a default package.json file that specifies its dependencies, but doesn’t include the Dialogflow client library, which we’ll use to access the Agent API. To fix this, delete all the default code in the package.json tab of the inline editor and copy and paste in the following JSON:

{
 "name": "dialogflowFirebaseFulfillment",
 "description": "",
 "version": "0.0.1",
 "private": true,
 "license": "Apache Version 2.0",
 "author": "Google Inc.",
 "engines": {
   "node": "~6.0"
 },
 "scripts": {
   "start": "firebase serve --only functions:dialogflowFirebaseFulfillment",
   "deploy": "firebase deploy --only functions:dialogflowFirebaseFulfillment"
 },
 "dependencies": {
   "actions-on-google": "^2.2.0",
   "firebase-admin": "^6",
   "firebase-functions": "^2",
   "dialogflow": "^0.6.0",
   "dialogflow-fulfillment": "^0.6.0"
 }
}

Add boilerplate code

Next, in the index.js file, add boilerplate code that imports necessary libraries (we’ll be adding code to index.js from this point on) :

'use strict';
const functions = require('firebase-functions');
const {WebhookClient} = require('dialogflow-fulfillment');
const dialogflow = require('dialogflow');

// Enables debugging statements from the dialogflow-fulfillment library.
process.env.DEBUG = 'dialogflow:debug';

Create cityData object

We’ll now map each city name to its streets and associated trivia data by creating a cityData object that contains the following:

  • The name of each city
  • A trivia property for each city, which contains the question we ask the user and the correct street name answer
  • An array of street names and their synonyms for each city, which represent the street names our agent can recognize

Note: In a production agent, this information could be hosted in a database or via an API and could contain the name of every street in each city.

const cityData = {
 'New York': {
   trivia: {
     question: 'Which street in New York is famous for its musical theater?',
     answer: 'Broadway',
   },
   streets: [
     // The streets are formatted in the Entity data structure so we can
     // send them directly to the Dialogflow API. These will become the
     // Entities within the SessionEntityType we are creating.
     // https://dialogflow.com/docs/reference/api-v2/rest/Shared.Types/BatchUpdateEntityTypesResponse#entity
     {value: 'Wall Street', synonyms: ['Wall Street']},
     {value: 'Fifth Avenue', synonyms: ['Fifth Avenue', '5th Avenue', '5th']},
     {value: 'Broadway', synonyms: ['Broadway']},
   ],
 },
 'Los Angeles': {
   trivia: {
     question: 'What street in Beverly Hills boasts some of '
       + 'the most expensive shops in the world?',
     answer: 'Rodeo Drive',
   },
   streets: [
     {value: 'Rodeo Drive', synonyms: ['Rodeo Drive', 'Rodeo']},
     {value: 'Mulholland Drive', synonyms: ['Mulholland Drive', 'Mulholland']},
     {value: 'Hollywood Boulevard', synonyms: ['Hollywood Boulevard']},
   ],
 },
 'Chicago': {
   trivia: {
     question: `Which fashionable street did Chicago's first mayor live on?`,
     answer: 'Rush Street',
   },
   streets: [
     {value: 'Rush Street', synonyms: ['Rush Street', 'Rush']},
     {value: 'Lake Shore Drive', synonyms: ['Lake Shore Drive']},
     {value: 'Broadway', synonyms: ['Broadway']},
   ],
 },
 'Houston': {
   trivia: {
     question: 'What is the main street at the University of Houston?',
     answer: 'Cullen Boulevard',
   },
   streets: [
     {value: 'Cullen Boulevard', synonyms: ['Cullen Boulevard', 'Cullen']},
     {value: 'Kirby Drive', synonyms: ['Kirby Drive', 'Kirby']},
     {value: 'Westheimer Road', synonyms: ['Westheimer Road', 'Westheimer']},
   ],
 },
};

Set up the Cloud Functions for Firebase endpoint

Next, let’s set up our Cloud Functions for Firebase endpoint:

exports.dialogflowFirebaseFulfillment =
 functions.https.onRequest((request, response) => {
   const agent = new WebhookClient({request, response});
   console.log('Dialogflow Request headers: ' + JSON.stringify(request.headers));
   console.log('Dialogflow Request body: ' + JSON.stringify(request.body));

Create askTriviaQuestion function

Let’s now define the askTriviaQuestion function, which is the fulfillment for the City name intent. We’ll split this function into three parts and describe each part individually. In the first part of the function, we do the following:

  • Get the name of the city from extracted parameters
  • Using the city name as a key, get associated street data from the cityData object
  function askTriviaQuestion(agent) {
     const city = agent.parameters['city'];

     // Look up data for this city from our datastore. In a production
     // agent, we could make a database or API call to do this.
     const data = cityData[city];
Define sessionEntityType

We’ll now define our session entity type by doing the following:

  • Creating a new SessionEntityTypesClient that calls the projects.agent.sessions.entityTypes API endpoint (note that no authentication credentials are required, since Cloud Functions for Firebase will authenticate us automatically)
  • Constructing the name for the sessionEntityType. This name is built from the session identifier (agent.session) along with the name of the EntityType we’re going to be overriding (/entityTypes/street).
  • Defining our new sessionEntityType object and using ENTITY_OVERRIDE_MODE_OVERRIDE to specify that this sessionEntityType’s entities should replace any values in the original EntityType, ‘street’
  • Building a request object that includes the current session and the sessionEntityType
   const client = new dialogflow.SessionEntityTypesClient();

     // Combine the session identifier with the name of the EntityType
     // we want to override, which in this case is 'street'. This is
     // according to the template in the docs:
     // https://dialogflow.com/docs/reference/api-v2/rest/v2/projects.agent.sessions.entityTypes#SessionEntityType
     const sessionEntityTypeName = agent.session + '/entityTypes/street';

     const sessionEntityType = {
       name: sessionEntityTypeName,
       entityOverrideMode: 'ENTITY_OVERRIDE_MODE_OVERRIDE',
       entities: data.streets,
     };

     const request = {
       parent: agent.session,
       sessionEntityType: sessionEntityType,
     };
Create sessionEntityType and ask trivia question

Next, let’s send a request to the projects.agent.sessions.entityTypes.create API endpoint to create a sessionEntityType and then ask the user a trivia question about the city they selected. In the following code snippet, we:

  • Create our new SessionEntityType, which represents the street names we defined earlier
  • Add our agent’s trivia question in response to the city the user chose
  • Catch any errors and handle them by apologizing and re-prompting the user for a city name
     return client
         .createSessionEntityType(request)
         .then((responses) => {
           console.log('Successfully created session entity type:',
               JSON.stringify(request));
           agent.add(`Great! I love ${city}. Here's a question about its streets!`);
           agent.add(data.trivia.question);
         })
         .catch((err) => {
           console.error('Error creating session entitytype: ', err);
           agent.add(`I'm sorry, I'm having trouble remembering that city.`);
           agent.add(`Is there a different city you'd like to be quizzed on?`);
         });
   }

Create checkTriviaAnswer function

The checkTriviaAnswer function handles our matched Trivia answer intent.

Check for city name

In the first part of our checkTriviaAnswer function, we do the following:

  • Check that the Cityname-followup context is present and contains the name of the city that the user selected
  • Respond to the user with an apology and reprompt if the expected context or parameter is not present (this shouldn’t happen if the agent is correctly configured)
   function checkTriviaAnswer(agent) {
     const context = agent.context.get('cityname-followup');
     const cityName = context.parameters ? context.parameters.city : undefined;

     // If we couldn't find the correct context, log an error and inform the
     // user. This should not happen if the agent is correctly configured.
     if (!context || !cityName) {
       console.error('Expected context or parameter was not present');
       agent.add(`I'm sorry, I forgot which city we're talking about!`);
       agent.add(`Would you like me to ask you about New York, LA, Chicago, or Houston?`);
       return;
     }
Check user’s answer

The next part of our checkTriviaAnswer function accesses the data associated with the street name to see if the user’s answer is correct. Here, we do the following:

  • Get the street name from the extracted parameters
  • Look up information associated with the user’s selected city in the cityData object
  • Compare the user’s answer (streetName) against the answer property in the cityData object
    • Add a congratulatory response if the user’s answer matches the answer defined in cityData and encourage the user to ask about another city
    • Add a response letting the user know their answer was incorrect and ask them to provide another street name
  • Delete the Cityname-followup context if the user’s answer is correct so our agent doesn’t expect to hear any more street answers. Once you delete the context, other intents are able to be matched again.
     const streetName = agent.parameters['street'];

     // Look up data for this city from our datastore. In a production
     // agent, we could make a database or API call to do this.
     const data = cityData[cityName];

     if (data.trivia.answer === streetName) {
       agent.add(`Nice work! You got the answer right. You're truly an expert on ${cityName}.`);
       agent.add(`Give me another city and I'll ask you more questions.`);
       agent.context.delete('cityname-followup');
     } else {
       agent.add(`Oops, ${streetName} isn't the right street! Try another street name...`);
     }
   }

Set up intent handlers

Now that we’ve created our askTriviaQuestion and checkTriviaAnswer functions, we can map them to their corresponding intents by doing the following:

  • Creating a mapping of which handler to use for each intent
  • Telling the Dialogflow fulfillment library to use this intentMap to map intents to handlers

Note: It’s important to use your exact intent names in your code so fulfillment works.

   // Run the proper function handler based on the matched Dialogflow intent name.
   const intentMap = new Map();
   intentMap.set('City name', askTriviaQuestion);
   intentMap.set('Trivia answer', checkTriviaAnswer);
   agent.handleRequest(intentMap);
 });

Deploy fulfillment code

We’re now ready to deploy our code! To do so, follow these steps:

  1. Open the Fulfillment tab and enable the inline editor.
  2. Delete all the default code within the index.js tab of the inline editor.
  3. Copy and paste the code from your index.js file into the index.js tab of the inline editor.
  4. Click DEPLOY. Once deployment is complete, a time stamp appears next to the DEPLOY button that specifies the date and time of the last deployment.

Test your Street Smarts agent in the console

Now that we’ve successfully deployed our fulfillment code and created our webhook, we can use the Dialogflow simulator to test out our “Street Smarts” agent. The simulator is located on the right side of the console and allows you to converse with your agent.

To talk to your agent, type “Hello” in the Try it now text field and press Enter. Your agent should respond with, Hey! Choose a city and I’ll test your street smarts. Your options are New York, Los Angeles, Chicago, and Houston.

Choose the city you’d like to be quizzed about (I chose Chicago). You should receive a response similar to the one in the following screenshot:

alt_text

To make sure the trivia game is working correctly, respond with an answer you know is incorrect (in this example, we’ll use Broadway). Your agent should respond with something similar to, Oops, Broadway isn’t the right street! Try another street name… (If you answer with a street name that’s not associated with the city in the cityData map, your agent responds with, Hmm, I don’t actually know that one, but it’s definitely not the right answer. Try again!)

Now, answer the question correctly (in our example, the right answer is Rush Street). The agent should respond that the answer is correct and give you the option to restart the game:

alt_text

Next steps

In this blog post, we’ve created a working “Street Smarts” trivia agent and, in doing so, learned how to use the Agent API to modify the agent without using the console. We created session entities with the Agent API, which can be used to tailor entities based on a user’s input. Session entities are also useful if you have duplicate entity values but want to differentiate them in some way. For example, in our “Street Smarts” trivia game, the street Broadway exists in both Chicago and New York City, but our Dialogflow agent realizes that Broadway is only the correct answer for New York City.

Now that you’ve gained some familiarity with the Agent API, you can use it for other, more large-scale use cases. For example, if you had an online store, you could use the Agent API to query your backend services to keep entity values that represent your store’s products synchronized in real time.

For more information about the topics we’ve covered, see the following links:

Posted by Lauren Ward, Technical Editor





Migrate to Dialogflow API V2 before October 23, 2019

October 23, 2018

Back in April, we announced that Dialogflow API V2, along with Dialogflow Enterprise Edition (built on API V2), became generally available to all users. API V2 now serves as the default API for all new Dialogflow agents, and we’ve been releasing new features on API V2 since then. Some of the new features include: Versions and environments for building, testing, and deploying multiple environments, and Knowledge Connectors and Phone Gateways for easily implementing content-heavy FAQs and phone numbers into your agent.

API V1 will be deprecated on October 23, 2019

As we’re building out more and more functionality for API V2, we will also be shutting down API V1 in one year – on October 23th, 2019. If your production agent is currently using API V1, it must be migrated to API V2 before this date.

See the API V1 → V2 migration guide for instructions on how to migrate, or read on for an overview of what’s required.

How to migrate from API V1 to API V2

Migrating from API V1 to API V2 affects the behavior of the Dialogflow console, APIs, and your fulfillment. You can check what parts of Dialogflow you are using here. When migrating from API V1 to API V2, please note the following:

  • If utilizing fulfillment for your agent, you must update your fulfillment code to support the API V2 request and response formats before switching to API V2.
  • All Dialogflow APIs have been redesigned and use OAuth for authentication. If you use this API directly, you need to update your client code before switching to API V2. See this guide for API V2 equivalents of API V1 methods and fields.
  • If you’re building an Action with the Google Assistant integration, see this Medium post for information on how to migrate from API V1 to API V2.

You can get started on migrating today with the API V1 → V2 migration guide. Please contact us if you need any help.

Posted by Matt Carroll, Developer Programs Engineer





Validate entities using regular expressions in fulfillment

August 23, 2018

Entities are powerful tools used for extracting parameter values from natural language inputs. Dialogflow developers use entities to retrieve important information from users, such as dates & times for appointments, locations for shipping orders, and numbers. Dialogflow already contains built-in system entities for common concepts like dates, times, numbers and more. In this post, we’ll show you how to use regular expressions to validate custom entities with fulfillment.

As an example, let’s imagine an organization has alphanumeric Employee IDs that are always six characters in length, and that the letters and numbers can appear in any position within the IDs. Here are the steps to validate if the Employee ID provided by the user complies with the format:

Let’s begin by creating an intent and adding 15-20 different Training Phrases to it. Try to add as many of the most common examples your employees might use to communicate their ID. Since we are looking for validating specific keywords associated with an entity, such words/phrases need to be identified in the user query and mapped with an entity.

In this case, even though we know the length of the ID and what it should comprise of, there is no specific format in which the letters and numbers should appear in it. Since the keyword does not comply with any predefined entity format (and it’s not possible to create a custom developer entity for this), we recommend using @sys.any as a wildcard entity here. To do that, highlight the keyword in the training phrases and choose @sys.any from the list of entities.

Since we suggest using @sys.any for this, please consider a few points in mind to improve matching accuracy:

  1. A little higher number of examples than the generally recommended number (10-15) should be provided in the intent.
  2. Avoid mapping the entire training phrase to @sys.any
  3. If possible, prompt users for confirmation before fulfilling the request.
  4. You could also consider restricting this intent with an input context, if possible.

For instance, in the screenshot below, an intent “Get Employee ID” is created and the Employee ID is annotated in the training phrases to @sys.any.

alt_text

Now that we have extracted the required keyword from the user query, we need to check if it is in the expected format or not. This check would be done in the fulfillment code by comparing this keyword with the desired regex pattern. We would, therefore, need to enable webhook fulfillment for this intent. To do this, click Fulfillment at the bottom of the page to reveal the options. Choose Enable webhook call for this intent and click the SAVE button as shown in the screenshot below.

alt_text

The next step involves writing the fulfillment code for the validateEmployeeID intent. This would further involve two sub-steps:

  1. Map the intent to a function: This will let the fulfillment know which block of code to trigger. You can create a map (like intentMap in the code snippet below) to associate an action handler with the name of an intent. This would indicate to the fulfillment what steps should be taken to complete the user’s request when it matches a particular intent. In our example, we have created an action handler ‘validateEmployeeID’ for the intent ‘Get Employee ID’.

    // Run the proper function handler based on the matched Dialogflow intent name
    let intentMap = new Map();
    intentMap.set('Get Employee ID', validateEmployeeID);
    
  2. Implement logic in the action handler to validate the length and format of the ID: In order to validate if the ID is in the correct format or not, we would need to ensure that the ID provided in the user’s request is passed onto the fulfillment. This is done with the help of parameters. Parameters are elements generally used to connect words in a user’s response, to entities. Please note that we have created one such parameter ‘employeeID’ already and its value can be extracted in the fulfillment code from the JSON request body (field name: queryResult.parameters). Once we have the Employee ID extracted in the fulfillment, we try to match it with a regex pattern specified in the fulfillment code for the required format.

Please NOTE that you would need to modify the regex pattern based on your specific use-case.

The code snippet that implements the above step looks like this:

function validateEmployeeID (agent) {
    // get the employee ID parameter from the request received from Dialogflow
    let employeeID = agent.parameters.employeeID;
    let pattern = /[^a-zA-Z0-9]/;
    if (employeeID.length !== 6) { 
        agent.add(`The length of the Employee ID should be six characters.Please enter the correct ID.`); 
    } else if (employeeID.match(pattern) !== null) { 
        agent.add(`Employee ID should have only letters or numbers. Please enter the correct ID.`); 
    }
}

You can find the complete sample code for this functionality on Github. Click on the button below to try out the sample now!

Drop by Dialogflow’s Google+ community to share your feedback and subscribe to our blog for more such tutorials. We look forward to hearing from you!

Posted by Surbhee Sehgal, Partner Technology Manager, Dialogflow





Kicking off I/O ‘18 with new & improved features

May 8, 2018

Today, we’re kicking off our 2nd year at Google I/O with lots of feature releases to help you build conversational experiences faster and smarter. See them at this Thursday’s I/O talk (which will be livestreamed) and come chat with us at the Google Assistant Sandbox if you’re attending the festivities!

Get started faster with new onboarding resources

New to Dialogflow? Watch our 3-part ‘Basics of Dialogflow’ video series to learn core Dialogflow concepts such as intents, entities, contexts and fulfillment.

And for those eager to start building, quickly import one of nine fully functional Dialogflow samples, complete with fulfillment code. Click on the ‘Add to Dialogflow’ button below to try out Temperature Trivia, which demonstrates how easy it is to build an Action for the Google Assistant using Dialogflow:

Temperature Trivia
Temperature Trivia

Debug using more insights and diagnostics

As developers ourselves, we know how important it is to detect errors and to receive specific and actionable help towards resolving them. The history tool has been redesigned to cleanly display conversations between your users and your agent, and flag places where your agent was unable to match an intent. It also links to diagnostics via our new Google Stackdriver Logging integration so you can easily diagnose and quickly fix issues.

history UI

We’ve also expanded the diagnostic information shown in the test console, so you can see the raw request being sent to your fulfillment webhook and the response that your webhook sends back.

diagnostics UI

Improve NLU quality by training with negative examples

We’ve heard your feedback! Developers have been asking for our natural language understanding to ignore certain phrases that mistrigger unwanted intents. You can now improve your agent’s precision by adding negative examples as training phrases for fallback intents.

negative examples
By providing the negative example 'Buy bus ticket to San Francisco' in the Default Fallback Intent, a purchase intent will not be matched to this inquiry if the agent only sells flight tickets.

Build to more Google Assistant surfaces and environments

As mentioned on the Google Developers blog, Actions will soon be available on new Smart Displays. You can start building for these visually rich surfaces in your Dialogflow console. We’ve launched support for tables, and will be adding more multimodal options in the upcoming months.

Google Assistant Table Card UI

Also announced is the ability to test your Action with groups of users before launching publicly. See how these new release environments in the Actions Console work with your Dialogflow agents. (For all other platforms, you can also try the beta release of Dialogflow versions and environments.)

Try these out

Go ahead and give these features a try! Drop by our developer community to share your feedback, subscribe to future news & updates, and follow us on Twitter. To see what others have built, check out our case studies with KLM Royal Dutch Airlines, Domino’s and Ticketmaster.

Posted by Artem Goncharuk, Engineering Lead





Subscribe Via Email

Enter your email address:

Delivered by FeedBurner

About

Welcome to the Dialogflow blog. Subscribe to get product updates, best practices, and tutorials directly in your inbox.

Resources

Follow us on social