API.AI - Official Blog

2 new features: New Analytics tool and more Google Assistant surfaces

2017-06-14T17:04:00.001-07:00

Today, we’re launching 2 major features in API.AI to help you improve your agents and reach new audiences. First, you can now track your agent’s performance with our new Analytics tool. Second, you can reach Google Assistant users across eligible Android phones and iPhones.

1. Track performance with our new Analytics tool

We’ve heard your feedback to have analytics on how your agents are performing. Our new Analytics tool lets you view data around session traffic and intent usage. We hope the new tool can help you drive improvements to your agents by identifying usage patterns, latency issues, and high- and low-performing intents. We’re working on adding more stats and functionalities so stay tuned for updates.

2. Reach more Google Assistant users across eligible Android phones and iPhones

At Google I/O today, it was announced that developers can now reach Google Assistant users on eligible Android phones and iPhones. Starting today, you can use API.AI’s Actions on Google integration to build and launch Assistant apps on these phones.

Example of a CNBC rich card on the Google Assistant

Enter your Assistant app in the Actions on Google Developer Challenge for a chance to win great prizes, including a trip to Google I/O 2018!
Check out the livestream of our Google I/O session tomorrow as we demo these new features. We look forward to seeing how you’ll utilize them. As always, let us know if you have any questions or comments.
Build on,
The API.AI team

Structure and Content

2017-03-29T14:15:00.000-07:00

With API.AI, our goal is help you create high quality and engaging agents. Today we’re taking that goal to the next step, adding some new features that will make it easier to build great agents across all of our supported platforms.

Follow-up Intents

Natural human dialogs are filled with follow-ups and confirmations for example:

Agent: Your cab can meet you at the corner of 5th and Broad street in 12 mins.
User: Ahh, can you repeat that please?
-or -
Agent: Do you want fries with that?
User: Sure, sounds good, thanks

Follow-up intents make these types of natural conversation flows easy to build and customize. With this new feature of API.AI you can define dialog situations that may follow any given “parent” intent you are currently working on.

While you can also do this (and much more) by manually defining Contexts, follow-up intents make the process of building sample dialogs easier by being more visual. We’ve thought of some of the most popular follow-up scenarios and taken care of building contexts for you automatically.

To start building Follow-up intents, click “Add follow-up intent” link and select one of the common follow-up options (such as confirmation, request to learn more, selection, etc.) or choose the option “Custom intent” to build a new unique follow-up. Common follow-up options contain pre-populated training examples that could be changed or augmented with phrases specific to your agent.

Finally, in the API.AI console Follow-up intents are nested under parent intents which makes it easier to see your agent conversation structure.

Prebuilt Agents

Building an advanced agent covering a comprehensive set of intents and entities can often be challenging even for the most skilled developer. Prebuilt Agents can help - they are free sample agents that are built by the API.AI team that utilize most of the API.AI platform’s capabilities. You can either use them as they are or modify them as needed to jump start building your own conversational agent.

So, if you are working in travel space, for example, you can use Flights Prebuilt Agent with built-in intents like, “Find a flight from Dublin to Copenhagen on July 16, returning July 23, for 2 people”, and follow-ups like “Actually, returning July 25”. Other agents available include booking, support, weather, TV, music player, and more.

Feel free to import any of the Prebuilt Agents to your account and reuse parts of it by copying intents and entities to your agent or augmenting the imported agent itself.

We will be rolling out new Prebuilt Agents regularly and will be happy to hear what agents you’d be interested to see next.

Happy coding,

API.AI team

Build Conversation Actions for Google Assistant users

2016-12-08T14:18:00.000-08:00

Reach Google Assistant users on Google Home and soon, other Google Assistant surfaces, with our new Actions on Google integration

Starting today, you can build rich, conversational experiences for Google Assistant users on Google Home and deploy them as Conversation Actions in just a few easy steps with API.AI.

API.AI makes it possible for your service to receive and respond to user queries in natural language. So whether you aim to answer questions or help users to be more productive, API.AI can help you build engaging experiences.

To build a Conversation Action with API.AI, start by defining the user requests or intents that your service will handle. Then design the conversational flow. If you’re new to API.AI and want to develop Conversation Actions, we’ve prepared some handy docs and tutorials.

Once you’ve built your Conversational Action (or if you already have a service developed with API.AI), you can easily submit your integration for the Google Assistant in the Integrations tab.

After you launch, you can update your Conversation Action in real-time based on user interactions, and it will continue to get smarter based on API.AI’s machine learning capabilities.

This is just the beginning – in the future Actions on Google will let you reach Google Assistant users beyond Google Home, including Pixel phones and Google Allo.

With the addition of Actions on Google, API.AI can now support your conversational experience development efforts across 10+ platforms. And with our recently updated free Standard tier and advanced natural language understanding technology, you can build and scale engaging experiences for everyone.

Get a jump start on building conversational experiences using API.AI for Actions on Google and reach Google Home users now.

Happy coding, API.AI team

Freedom of Speech for Bots

2016-11-03T14:19:00.000-07:00

Since API.AI launched, our goal has been to make building advanced conversational interfaces easy and enjoyable. It’s great to see the growing number of agents powered by API.AI, but we know there’s more work to do in order to be more accessible and enhance the platform’s feature set. Today, we’re making progress towards both of these goals:

First, we’re removing limitations on the number of queries for our Free plan for both public and private agents (bandwidth limits may apply). This means that you don’t have to worry about how much it’s going to cost when your chatbot takes off - you can just savor the moment! And for those of you who need some more premium features - higher bandwidth, SLA, pre-built domains to name a few - we can develop a custom Preferred plan that’s just right for you.

Second, we’ve added features that’ll help liven up your chatbot responses. Starting today, API.AI supports Rich Messaging - cards, images, quick replies - on popular platforms, such as Facebook Messenger, Slack, Kik, and Telegram. This new feature will enhance the quality of your bot’s dialogue, keeping your audience engaged and entertained.

You can also use Custom Payload in your bot response to add payments, confirmations, receipts, videos and other types of rich content supported by the messaging platform.

We can’t wait to see the new exciting conversational agents you build with the help of API.AI. So, what are you waiting for? Start building your agent now.

Happy Bot Building, API.AI team

API.AI is joining Google!

2016-09-19T14:21:00.000-07:00

We are excited to announce that API.AI is joining Google!

It has been a long and fun journey. Since API.AI’s launch in 2014, we’ve been constantly impressed by the fast and energetic adoption of the technology from people building conversational interfaces for chatbots, connected cars, smart home devices, mobile applications, wearables, services, robots and more. Our vision has been to make technology understand and speak human language and help developers build truly intelligent conversational interfaces for their products and services. The best part of our day is hearing from our diverse community and how they are using API.AI to create truly innovative, practical and inspirational products that will reshape how we live and work in the future.

What does this mean to you? We’re excited for you to continue using our developer platform to build conversational user interfaces. Joining Google will allow us to accelerate improvements to the platform and service our growing developer community in ways we’ve always dreamed. With Google’s knowledge, infrastructure and support, we can make sure you get access to the best available technologies and developments in AI and machine learning.

As we start this next chapter, we want to express how truly grateful we are to our customers, developers, partners, employees, advisors and investors. We can’t wait to continue working with you and could not have gotten to this point without your support.We cannot thank you enough.

We hope you are as excited as we are with what’s to come at Google and value your continued support.

Sincerely, Ilya Gelfenbeyn, CEO

API.AI’s Cisco Spark and Tropo Solution at Cisco Live!

2016-06-24T14:21:00.000-07:00

It’s always exciting to do a little show-and-tell with products you’re proud of. That’s why we can’t wait to be at Cisco Live in Las Vegas from July 10-14, where we’ll be demonstrating API.AI’s latest integrations with Cisco Spark and Cisco-owned Tropo. We’ll be exhibiting at the event as well – if you’re attending, come by and see us at booth C7.

Tropo and Spark are each part of Cisco’s suite of communications solutions. Tropo is a cloud API for communicating via voice and SMS messaging, intended for large-scale enterprises and service providers. Using Tropo, developers and companies can automate real-time communication over voice and messaging channels – especially valuable as a platform for fostering collaboration in the cloud.

Spark is an enterprise messenger that features options for voice and video communication. Cisco has opened the APIs for Spark so that enterprise developers are welcome to customize the technology as needed.

Our Tropo and Spark integrations team up those technologies with API.AI’s own natural language understanding capabilities, adding the rich possibilities of conversational user experiences to these communications tools. Developers taking advantage of these new integrations can create bots and interfaces that comprehend natural speech and respond in real-time. And, our Tropo and Spark integrations are designed to accommodate business needs and common corporate policies around security, privacy, safe control over data, interoperability, and conforming with IT standards.

We’re proud to say that these integrations have already proven themselves to be valuable solutions for building powerful cloud-based collaboration tools, as our partner Redbooth can attest. Redbooth offers a cloud-based collaboration product that assists business teams in easily communicating ideas, amplifying productivity and performance. It makes use of Tropo and Spark in order to implement several aspects of its product’s communications functionality (for example, adding telephony to let users easily turn online conversations into phone calls with the click of a button for when instant voice communication is essential). Now using API.AI’s integrations, Redbooth has added a task management feature and has produced a bot that engages with users through natural language conversations – and can retrieve any information stored within Redbooth’s software. Users can simply speak with Redbooth’s bot just as they would with another person and receive a highly satisfying user experience.

Anyone at Cisco Live interested in developing conversational bots and interfaces for their communications solutions should come by and say, ‘Hi.’ We’d love to see you there.

Manipulating Entity Entries via API

2016-04-04T13:50:00.000-07:00

Let’s imagine an agent for a retail store app. There is a developer entity that matches items available in the store. Let’s call it @item. The initial version of the entity contains a list of items available in the store at the moment of the agent’s creation. But since the stock of items is updated regularly, it would be very inefficient to update the entity manually.

To keep the entity in sync with the store database, API.AI now offers three additional options in the /entities endpoint that allow you to add, update, and delete entity entries via our API.

Let’s see how it works. Here is the original version of a sample entity we started working with:

Remember that for any of the following operations you’ll need to specify an entity ID (referenced as {eid}) or its name.

Adding Entity Entries

So, the store has two new items for sale, e.g., kefir and ice-cream, and the store owner needs to add new entries to the @item entity.

This can be done with the help of POST /entities/{eid}/entries request.

The POST body should be an array of entity entry objects in JSON format.

POST https://api.api.ai/v1/entities/item/entries?v=20150910

Headers:
Authorization: Bearer YOUR_DEVELOPER_ACCESS_TOKEN
Content-Type: application/json; charset=utf-8

POST body:
[{
 "value": "kefir",
 "synonyms": ["kefir"]
}, {
 "value": "ice-cream",
 "synonyms": ["ice-cream", "ice cream"]
}]

As a cURL request, it will look like this:

curl -i -X POST -H "Content-Type:application/json" -H "Authorization:Bearer YOUR_DEVELOPER_ACCESS_TOKEN" \
-d '[{"value": "kefir", "synonyms": ["kefir"]}, {"value": "ice-cream", "synonyms": ["ice-cream", "ice cream"]}]' 'https://api.api.ai/v1/entities/item/entries'

This is how the entity will look in the API.AI developer console after sending this request:

Updating Entity Entries

If you need to add or delete synonyms in already existing entries, you can use PUT /entities/{eid}/entries request.

The PUT body should be an array of entity entry objects in JSON format. The array of synonyms should contain the updated array of synonyms.

For example, you may want to add some spelling variants to the “yogurt” and “kefir” entries.

The cURL request for this will look like this:

curl -i -X PUT -H "Accept:application/json" H "Content-Type:application/json" -H "Authorization:Bearer YOUR_DEVELOPER_ACCESS_TOKEN" \
-d '[{"value":"yogurt", "synonyms": ["yoghurt", "yoghourt"]}, {"value":"kefir", "synonyms": ["kefir", "keefir", "kephir"]}]' 'https://api.api.ai/v1/entities/item/entries'

The result of this manipulation can be seen in the API.AI developer console:

Deleting Entity Entries

Now, let’s imagine some items were sold out and you need to delete the entries from the @item entity.

You can do it with the help of DELETE /entities/{eid}/entries request.

The DELETE body should contain an array of strings corresponding to the reference values of entity entries.

For example, for deleting “milk” and “kefir”, the cURL request will be this:

curl -i -X DELETE -H "Accept:application/json" -H "Content-Type:application/json" -H "Authorization:Bearer YOUR_DEVELOPER_ACCESS_TOKEN" \
-d '["milk", "kefir"]' 'https://api.api.ai/v1/entities/item/entries'

Here’s the result of this request seen in the API.AI developer console:

Related documentation:

Conversational Slack Bots 101

2016-03-17T14:02:00.000-07:00

Following up on the VentureBeat article, here is a short tutorial on how to create a Slack bot at API.AI.

Millions of Slack users chat with each other and bots every day. Bots are the best, because you can integrate them with lots of useful services and apps. They provide you with information, deliver notifications, and offer a myriad of other handy features. You can simply talk to the bot naturally and it understands the context to respond intelligently. And now with the API.AI-Slack integration, you can train your bots, so they can have real conversations with users to perform their duties better.

Design conversations

Start with creating an agent at API.AI. The agent will be the conversational part of your bot’s brain.

When you come up with an idea of what conversations your bot should be able to carry with your users, start training the API.AI agent. It’s as simple as providing just a few examples of how people may phrase their requests and define what type of data you want to extract from such requests.

Connect to a web service

After your bot learns how to understand people, it needs to start responding. You can connect your bot to a web service via a webhook. This will allow you to pass information from a matched request into a web service and get a result from it. This way you can implement connections to data services, business logic, etc.

Test

In the API.AI developer account, you can find the test console, where you can try how well your agent understands you already. If your request is recognized, you’ll see which intent was used to process it and what information was extracted.

Train it more right away by adding more examples in existing scenarios or create new ones.

Test some more. Do you love it? Move to the next step.

Create a Slack bot

To enable Slack Integration, go to your agent’s settings, select ‘Integrations’ from the horizontal menu, and turn Slack integration on. Alternatively, you can select ‘Slack integration’ from the left side menu.

To define the way people can interact with your bot, tick the relevant checkboxes in the ‘Message Types’ section.

Direct message. The bot will receive a direct message from a user.
Direct mention. The bot will answer when mentioned at the beginning of a message.
Mention. The bot will answer when mentioned in any part of a message.
Ambient. The bot will monitor any message in a channel.

Don’t forget to save the settings.

To connect the bot to your Slack account, click ‘Test in Slack’ and then sign in with your team account. Let your team know about your bot. The more conversations it gets, the more data you’ll have to train it. And the smarter your bot will become as a result.

When you’re ready to publish your bot to the community, go to the Slack Apps webpage where you can find all the necessary documentation. Click on ‘My apps’ from the top menu, and then on ‘Create a new application’.

You’ll be asked to fill in the app name, descriptions of the app and a Redirect URI. Use the Redirect URI from your API.AI agent’s Slack Integration settings.

After clicking on the ‘Create application’ button, you’ll see your Client ID and Client Secret. Go to your API.AI agent’s Slack integration settings and fill in the Client ID and Client Secret fields with the values from your Slack app’s settings. Then click ‘Go online’.

Go back to your app settings on the Slack webpage, scroll down to the ‘Bot User’ section, and configure a new Bot user for your app by clicking on ‘Add a bot to this app’ button.

Scroll up to the top of the page and click the ‘Add to Slack’ link. The Slack button page will open.

Scroll down to the ‘Add to Slack button’ section. Uncheck ‘incoming webhook’ and tick the ‘bot’ checkbox, which generates the HTML code for the ‘Add to Slack’ button that can be added to your website.

To launch the bot for your Slack team, just click the button ‘Add to Slack’ above the generated code.

If you intend to make your bot accessible to other teams, you’ll need to place the ‘Add to Slack’ button on your website, so that anyone can launch your bot by clicking it.

Note: If you want to have your bot published in the Slack App Directory, you have to register as a developer and go through the Slack approval process.

Check out our API.AI for Slack video!

What are contexts and how are they used?

2015-11-23T14:09:00.000-08:00

A context is a powerful tool that can help to create sophisticated voice interaction scenarios. In this post we’ll look at how you can use contexts to build dialogs.

When a dialog is created, it’s usually the case that many branches of conversation are possible. Therefore, intents in an agent should be designed so that they follow each other in the correct order during a conversation.

The best way to demonstrate how this works is to build a conversation together, step-by-step. As an example, we’re going to create an agent for a floral shop. You can download it here and try the instructions in your account.

First, let’s take a quick look at how contexts operate.

1. Using contexts

Contexts will appear in your agent just under the intent title. You should see two fields here: input and output contexts. Corresponding input and output contexts in intents will determine whether they follow or precede one another.

To add a context, just type its name into the field and press enter. The context will now appear as a box with a colored border.

Here we’ve set an output context for our floral shop named ‘compose’. Once this intent is matched, the context ‘compose’ will be set. So any intents that have this as an incoming context will now have top priority when matching user requests. Note that these intents will not be matched if the context is not set.

All contexts have a very important property – lifespan.

By default the lifespan value is set to five. It means that the context will last for the next five matched intents. So if you have different input/output contexts in each of the following intents, all of them will be collected in the next five stages of the dialog, like a chain of contexts.

While this feature can be very useful, at times you may want to get rid of a context after the following intent is reached. In these situations you can simply change the lifespan to one.

2. Managing the conversation flow

Now we’re ready to design the dialog we want our agent to produce. The best way to do this is to draw a scheme like the one pictured below. You can use Lucidchart for this.

From this scheme we can see that there are two main branches of the conversation. The first is for people who want to compose a bouquet themselves; the second provides the option to choose from one of four pre-made bouquets.

In order to manage these two branches of the conversation we’ll have to set different contexts.

The intent where fulfillment is ‘Okay! Would you like to compose a bouquet yourself?’, looks exactly as it was shown above: its name is ‘compose’ and it has an output context ‘compose’.

So we’ll have to create two different intents to match the possible answers. We’ll make one for ‘yes’:

And one for ‘no’:

In both intents we have the same input context: ‘compose’. So after the intent ‘compose’ was matched and the context ‘compose’ was set, we can then logically only reach those intents with an input context ‘compose’. So if we say ‘yes’ we get into the ‘yes - compose’ intent; if ‘no’, then we are in the ‘no - compose’ intent.

And now comes the most crucial point: we have to set different output contexts. So in order to continue choosing the flowers we have to set now ‘yes - compose’ context as output, and then as an input context to the following intents. The same goes for the other intent. In order to choose one of the pre-made floral arrangements, we have to set an output context ‘no - compose’ and then create the following intent with ‘no - compose’ as the input context.

This same logic applies along the whole dialog structure. Whenever you need to go further to a specific intent, you should set the corresponding contexts. This is especially important when you have multiple ‘yes/no’ answers in the conversation.

3. Gathering the parameters’ values

The other important function of the context is its ability to collect the parameters’ values. In our example, we’re collecting information during the dialog so we know by the end how to fill the customer’s order – how many flowers of what kind and what color we should add, or alternately, which pre-made bouquet we should choose. As long as a certain context is still alive (i.e., within its lifespan) the value we get from a parameter (if it was set in a context) is also alive.

In order to collect all the values at the end, we just have to set one context, which will appear in every intent, both input and output (except for the very first intent, where we set the output context only).

We just have to slightly modify what we see in the screenshots above:

To retrieve the parameters from the context, write: #context_name.parameter_value:

Then in the dialog those values will look the same as always:

Feel free to play around with this example. You can download the agent here.

Introducing Slot Filling – the easiest way to build a dialog

2015-11-09T14:21:00.000-08:00

We are super excited to introduce the slot filling in API.AI! It greatly simplifies building dialogs. In fact, you can build a simple parameter collection dialog within a single intent if you’d like!

Let’s say we want to create an agent for sending messages. We’ll need to collect values for the following parameters: “text” and “name”. We’ll also want to make sure that we’ve matched all the values correctly, and to give users an opportunity to edit the values.

Here’s an example of this type of dialog structure.

It’s important to outline the dialog structure before you start building your agent, since this will help you choose the right tools within API.AI. For example, you can see that we will be using the slot filling feature for linear parts of the dialog, and contexts for when it branches into several possible variations.

There will be 6 intents in our agent:

The first intent “0 - Send Message” will collect the values for the “text” and “name” parameters.

A user may start a dialog providing zero, one, or two parameters in the first phrase:

“Send a message” – matches no parameters
“Send a message to John” or “Send the message ‘Hello!’” – matches only one parameter (“name” or “text”)
“Send the message ‘Hello!’ to John” – matches both parameters (“text” and “name”).

We describe these variations in the intent:

In this intent we want to fill all the necessary slots, which is to say we have to collect all the needed parameter values. To do so, we’ll mark these parameters as Required. Then we’ll use Prompts to ask the user for parameter values that were not contained in their initial request:

If you want to change the order of the questions, use the arrow on the right to drag and drop the parameter lines:

When all the necessary values have been collected, the agent will respond with the phrase that has been defined in the Speech Response field:

The question “Is it correct?” creates the need for branching in the dialog structure:

If the user answers “Yes”, the agent will just send the message.
If the user answers “No”, the agent will suggest to edit the message.
If the user’s answer contains neither positive nor negative meaning, the agent will insist on receiving a “Yes” or “No” response.

To catch these possible answers, we are going to create three new intents:

1.1 - Message correct - Yes
1.2 - Message correct - No
1.3 - Message correct - Unknown

But before we do this, we’ll define the output context in the 0 - Send Message intent:

We will then use this as our input context in the intents 1.1 - Message correct - Yes, 1.2 - Message correct - No, and 1.3 - Message correct - Unknown in order to catch the user’s “Yes/No/Unknown” answers:

In the case of a positive answer, the agent will send a message with the parameter values collected in the first intent. Note that in order to refer to those values in the Speech Response, we’ll need to use the format #ContextName.ParameterName:

Since we don’t want to continue this dialog branch any further, we clear the output contexts. It is done by clicking on OUT button in the contexts section:

In the case of a negative answer, the dialog will continue within the new context new_message:

In the case of a neither positive, nor negative answer, the agent will continue asking the user for a “Yes” or “No” response:

The next intent “2 - New message” will catch the edited “text” and “name” values:

Since the question for filling the “text” value was already asked in the intent 1.2 - Message correct - No, we now only need a prompt for the “name” parameter:

When the edited values have been collected, the agent sends a message with these values:

Finally, in order to clear all contexts, the intent “3 - Clear contexts” is created:

Our agent is ready! Now you can go back to the dialog structure flowchart and test all possible scenarios.

Could You Fall In Love With a Robot? API.AI Exclusive

2015-10-29T14:38:00.000-07:00

In the 2013 film Her, the lead character falls in love with his virtual assistant, a charming, operating system voiced by the actress Scarlett Johansson. Rather than the Skynet-style omnipotent and destructive artificial intelligence imagined by films that look past the singularity – like The Terminator – Her shows a plausible, if fictional, a glimpse at a near future where our relationships with computers are much more human.

Artificial Intelligence, and virtual assistants, in particular, already perform dozens of tasks for hundreds of millions of people every day, but we now see that they also offer people emotional support, friendship and even more.

Our first-ever Virtual Assistant Survey reveals that more complex human - computer relationships are far closer than right around the corner. In fact, just two short years after Her hit theaters, we’re already there.

With data from 12,000+ Assistant.ai users, the Virtual Assistant Survey offers a wealth of insights into the numerous ways virtual assistants are changing the way we live, commute, shop, search and communicate.

What’s clear is that there is no aspect of our lives that has not been touched by the rise of virtual assistants and, believe it or not, smartphones are just the beginning.

Soon our cars, homes, and offices will be filled with connected devices – and not just watches, phones and computers. Fridges, desks, coffee pots, jackets, and every other physical object you can imagine will soon be connected devices. Voice-powered virtual assistants will be how we communicate, control and benefit from our connected future.

We may not all fall in love with our virtual assistants, but sooner than later, we’ll forget how we ever lived without them.

Better Than the Best Customer Service

2015-07-10T14:40:00.000-07:00

Jerry Gregoire of Dell once said, “The customer experience is the next competitive battleground.” The bottom line: exceptional customer service skills are crucial for building a company’s brand and reputation. With competitive quality and pricing no longer an issue for most products and services, it’s the customer service jobs that win over sales, good PR, and loyalty. With that being said, the ideal customer service scenario involves a knowledgeable customer working with an eager-to-help human. Guess how often that happens – not often. Larger companies can invest the time and resources to train anyone and motivate them to keep the customer happy. However, these same companies are offering increasingly complex products just so that they can remain competitive. This in turn results in a huge influx of confused customers calling in to get their questions answered (since, of course, they had no luck on the company’s website or forums). Therefore, organizations waste millions of dollars and thousands of hours answering simple questions - repeatedly. Not only is this inefficient, but it is extremely expensive. That’s why companies, large and small, have added automated customer service to their tier system. Ever find yourself screaming and swearing at an automated virtual receptionist after they misunderstood what you were asking for?

Us neither… But it can be tempting.

The strategy and technology behind those virtual assistant services leaves a lot to be desired right now, but that doesn’t mean something better doesn’t already exist.

By leveraging the api.ai platform, we can help companies automate their customer service through human-like conversations that understand natural language with conversational virtual assistants. This technology understands a customer’s problem and is capable of answering questions as well as resolving issues in real time. Therefore, customer service representatives can focus on more complex problems and other value-adding activities.

Of course, this technology can be deployed across all of your digital channels including text, voice, web, mobile and 3rd party API integrations. Multi-channel support empowers customers to seamlessly receive consistent service regardless of the platform. But it doesn’t stop there! Check out Forbes’ article on how api.ai integrated with Twilio, adds all new insight to communication here.

Interaction Scenario Design and Implementation

2015-06-23T14:40:00.000-07:00

Many times it is easier to reach the “Oh!” moment by just watching somebody else do it. For us visual learners out there, this tutorial breaks down the process of creating and implementing a conversation based interaction scenario for a pizza delivery app. Make your app development stand out with a conversational component.

Here are the steps that we cover:

1) Outline an example dialog with a user

2) Populate relevant entities

3) Set up single intent recognition (Single query and response)

4) Use context to support conversation and remember parameters

Whether you’re an iOS or android developer, you should create an app with a voice interface to thrill your users.

A Speech Interface in 3 Steps

2015-06-03T14:41:00.000-07:00

Even a capable developer can get fumbled by the components of a full speech interface. Some might think, “Voice recognition sounds good enough,” but that’s just the first step. Voice recognition can fulfill your dictation needs, but when applied to use cases that call for a full voice interface, it’s not only inefficient, it’s the lead cause of users screaming and cursing at their technology. It’s our privilege and sacred duty to set the record straight.

The overall processing flow of a comprehensive speech interface consists of 3 key components:

Speech recognition, speech to text, voice recognition, voice to text and automatic speech recognition (ASR) all refer to the same function: transcribing your spoken words. This first step converts your speech to text so that it can be processed (e.g. dictating an email or text to your mom). Devices need that component before anything else can happen.

(Side note: When working with api.ai you have a couple of different options…)

Option 1) You can use our ASR. This technology is based on statistical and acoustic data from over 2.5 billion processed requests to date. Once our clients have an implementation with api.ai, we can provide custom language models based on domain knowledge. Dynamic language models can also be generated or selected based on conversation and dialog focus.

Option 2) You can use any 3rd party ASR provider.

The second and most overlooked step is leveraging natural language understanding (NLU) technology. NLU cultivates natural interactions between you and a device; rather than being limited to very basic voice commands or needing to know the magic words to make it work. Let’s break it down:

 Intent Recognition: Needed to understand your user’s intent or meaning.

 Context Management: Needed for multi query cases.

 Conversation Management: Needed to support back-and-forth dialogue.

The speech that has been transcribed into text is still foreign to your technology and it won’t know what to do with it. Step 2 takes your text, which is your natural language input, and turns it into a structured object so that your product will understand. In other words, it uses Intent Recognition to understand what the user is asking for.

(Side note: Through advanced techniques in Machine Learning, api.ai can predict variations of what the user might say and still translate the meaning into the appropriate structured object.)

For example, a user could say, “Set thermostat to 72 degrees,” or “Thermostat to 72,” and the words are transcribed and then translated into an easy to rea d – for the device – structured object.

That’s an example of a single query interaction scenario. So what if you want to have some back-and-forth conversation with your device? You’d need context and Context Awareness. A user might say, “Set alarm for 7am,” and their device would respond, “Alarm set for 7am.” When they realize how unpleasant the 7am hour is, they could say, “That’s too early. Set it for an hour later.” And the device would know what their user was asking and respond, “Ok, alarm set for 8am.”

(Side note: api.ai has a sophistocated context managment toolset and engine implementation that supports both internal and external contextual data (e.g. GUI, gestures, diagnostics etc.).)

Conversation Management allows for the engine to seamlessly switch between conversation topics, while remembering what you were talking about (just like you wish your girl/boyfriend would). Users can also have clarifying conversations with your product in the same session / conversation across devices.

Now the device can transcribe your words, translate them into a readable structured object so that it can understand the user’s intent and context, and support back-and-forth dialogue – what’s missing? Fulfillment, implementation, follow through. After your device has a clear understanding of what you’re asking, it may need to take action to fulfill your request. Speech interface without fulfillment would look something like this:

 User: Turn on TV.
 *Nothing happens*
 Device: TV is on.
 User: …

So ask yourself, “Am I looking for transcription technology (speech to text)? Or for a back-and-forth dialogue interface that can deliver on my users’ requests?” Because afterall, who wants to live in a world without fulfillment?

Now you are equipped with an understanding of what a full speech interface requires and why it would crumble without each component. Our only hope is that you will take this knowledge and share it with those who think, “Voice recognition. That sounds good enough to me.”

Onboarding Users in the 21st Century

2015-05-21T14:44:00.000-07:00

You spend months developing a great app for users to fall in love with. You went the extra mile and made a Getting Started Tutorial to help onboard users that goes beyond the typical UX design. But then those overly anxious users skip the helpful tips, get lost in the options, and then complain that the app was too difficult to use or didn’t work properly.

When it comes to new, exciting technology, who has the patience to read through a tutorial? Our instincts are to explore first and ask later. So why not onboard new users with a they-ask-a-question-they-get-an-answer interface? Here’s what onboarding new users with a full voice interface looks like:

1) Person asks app how to do X. We turn that into actionable data in the form of a JSON.

 resolvedQuery: “How do I create a new task”
 action: task.create.help
 parameters: { helpTopic: “new tasks” }

2) App tells person how to do X

3) Person wants to know more, they ask.

Imagine: your user turns on their new device, they are greeted by a virtual assistant that allows for an interactive question and answer learning session. They ask just enough to get started and then they’re off – enjoying your creation! If they get stuck later on, that virtual assistant is always there to answer with how-to’s and tips.

The assistant can also be updated anytime to include more answers for updates and changes to the app or device. That way you never lose a user when adding new functionality or updating UX interfaces.

What’s the best part? You don’t have to imagine – the technology for an intuitive, affordable speech interface, for any app, device, or website, is here at api.ai. Raise your user acquisition and customer loyalty today.

User Defined Entities

2015-05-14T14:44:00.000-07:00

As an API.AI super user, you already understand the power behind the platform. Your implementation is reaching its final stage, and you have started to ask for more advanced capabilities. Most frequently, developers just like you are interested in being able to customize entities on the user/request level.

Let’s take a look at a quick scenario. You are a developer with an application to control all of the devices and settings in your smart home. Out of the box a user can refer to their objects in generic ways like “turn the lights on in my office”. But - what if the user wants a more personalized experience? If this is the case, the end user must be able to add their own names to refer to specific objects. For example, rather than referring to a generic office entity, they could say “turn on the lights in my mancave”.

In order to work with user entities:

1) Create an entity (e.g. @houselocation and @device entities for a smart home app) Populate the entity with default values (e.g. @houselocation has:Kitchen, Living Room, Bedroom and @object has Lights)

2) Create an intent that uses this entity (e.g. “Turn on the @device in the @houselocation”)

3) Utilize the new entities parameter in the query endpoint. The entities object accepts an array of entities, following the format used here.

The submitted entities will replace entities defined in the developer console just for this one request.

Here’s the curl example:

The specified entity needs to exist in the developer console. The UI does not allow for creating an empty entity, so you must have some default value there.

On the Android SDK level: To use the SDK please use the directive below instead of pulling it from the repo:

Usage example:

Finer grain controls for the entities are coming soon.

Speech Interface: The Remote You Can’t Lose

2015-04-28T14:48:00.000-07:00

There’s a word for having a remote control for every possible electronic in your life – convenient. Whether you’re settling down to finish some work or tucked in for the night, remote controls add even more convenience to the technology they’re paired with. Until you lose them, their batteries die, or one system requires 3+ remotes just to turn it on. Remotes were the solution for the 20th century, but now we have something better: speech interface.

What could be simpler than saying what you want to happen and then it happening? Let’s look at the - inevitable - smart home use case. We like the different control panels and various apps for each type of appliance we want to control, but what we would love, what would thrill our socks off, is to just say, “turn off the lights and lock the doors,” from our beds.

This is not a science fiction pitch for future generations. You can enjoy this first-class service today with products like the Ubi and Smarter Shade. Imagine walking into your home and saying, “Turn off the garage light, preheat the oven to 375, and turn on the SF Giants game,” and having the relevant electronics comply with your requests. That would be the real smart tv we’ve been waiting for. Speech interface is beyond convenient – it’s efficient and offers a whole new peace of mind. Try out api.ai today to make your all in one remote that’s handsfree and convenient.

Voice Enabling TaskList App with api.ai

2015-04-17T14:49:00.000-07:00

To help you get started in voice enabling your apps for Android, we have put together a basic implementation walkthrough. For this demonstration, we have decided to start with an open source to do list app and turn it into a smart task list that people can easily interact with on the go.

Link to android SDK: HERE

Playstore link to app we will voice enable: HERE

Link to git for TaskList open source code: HERE

Domains Release: Don’t Reinvent the Wheel

2015-04-09T15:39:00.000-07:00

If you are looking for quick, out-of-the-box speech to text functionality – then we have news for you! At api.ai, we understand that sometimes you don’t want to create everything from scratch and we have the full voice interface solution.

Through the half a decade that we spent creating our nlp engine, we have also created many significant knowledge domains to support interaction within the Assistant. For your convenience, we are releasing these Domain Knowledge Bases into api.ai

Domains are pre-defined knowledge packages. By enabling Domains on the appropriately named tab in the dev console, you can make your agent understand thousands of diverse requests – no coding or thinking required! This week we have released Domains for: Smart Home, Maps and Points of Interest, Booking, Media, Times and Dates, Web Browsing, Small Talk, Apps, and Device Control. Of course, more will be coming soon.

So, what does that mean? Now, user’s requests are sent for processing to both your agent and the Domains Knowledge Base. For ease of development, you will see both results in the api.ai test console. In the runtime, your agent has a preference. If a request is made via the HTTP API or one of the SDK helpers, api.ai will return the agent’s response if available. If not, a Domain response will be returned. If your app requires more commands to be processed within certain Domains, you can create your own intents in addition to what is available. [Note: You need to use the same name for the parameters to be consistent with the parameters returned from the Domains].

This means that you could create an empty agent, and as long as Domains are turned on it will understand all sorts of things (e.g. “I want to hear the Red Hot Chili Peppers”, “Lock the front door at 9 pm”, “Book me a hotel for 2 nights in San Francisco”, “Wake me up at 7:30 am”, “I want a hot dog”, “I want to go to a concert tomorrow”).

For more information, check out our Domains documentation here.

Why not interact with advertisements?

2015-03-31T15:40:00.000-07:00

Do you remember the “olden days” when we had to watch TV commercials? We zoned out during irrelevant radio promos, muted Spotify blurbs, and desperately pressed “skip” to YouTube ads. Oh wait – it’s still that way. Digital advertising is a multi-billion dollar industry and yet it is in need of some top-notch innovation. They really haven’t had many options, until now…

With the capabilities of speech to text growing, modern advertisements are on the brink of a revolution. It will leverage both big data and cutting edge technology to provide a personalized ad experience that was previously unimaginable. Paid ads and freeads alike can now be updated dynamically to reflect your preferences and interact with you as if you were, well, a human.

Let’s look at a few use cases through something like Spotify.

Ad: “Hi Mike! I noticed that your mom’s birthday is around the corner.  She might enjoy a dozen hot pink roses.  Would you like me to send her some?”

Mike: “Oh, I forgot.  Please send her two dozen white roses.”

Ad: “Sure thing.  $62 will be charged to your credit card on file.”

–

Ad: “The summer is creeping up and I have some hot deals specifically for you, Mike.  Would you like to hear more?”

Mike: “Are there any flight to Hawaii deals?”

Ad: “Let me look. Yes, I have a deal for a flight to Maui during the first week of June.  Would you like to book it now or should I send you an email with the details?”
Mike: Send me an email.

–

Ad: “Hi Mike, Geico can save you 15% in under 15 minutes. If you would like to hear more about how to save on your auto insurance by switching to Geico, just ask”

(Later that day)

Mike: “Can you tell me more about the Geico deal from earlier?”

Ad: “Sure…Let me connect you to a Geico representative to find a plan that’s best for you”

Going with mandatory versioning soon

2015-03-03T15:41:00.000-08:00

Greetings API.AI Developers!

We are announcing that starting June 1, 2015, the versioning v parameter will be mandatory in all api.ai calls. Introduction of the versioning support will allow us to add new functions and make improvements to api.ai that are not backwards compatible. We are using the approach proposed in Foursquare API.

Couple of examples of such changes:

Domains functionality that you might already have seen in the console
Enhancing the return formats for system entities. E.g. @sys.date entity will be handling dates both in the future and in the past.

To use it, include v=YYYYMMDD parameter when you make a request. For example, https://api.api.ai/v1/query?v=20150330&query=weather&lang=en&sessionId=123

Check out our HTTP API documentation for more details. If you have any questions, always feel free to drop us a line.

Happy coding!
Team API.AI

Create Intents with System Entities

2014-12-10T15:43:00.000-08:00

For your convenience, we have included pre-defined system entities. We take care of common entities so that you don’t have to. System entities include things such as countries, capitals, music artists/genres, times, dates, numbers, names, colors and more (you can find a list in the entity overview). When you use the system entities, there is variability included for incoming values (e.g. USA can be referred to as the States or the United States of America).

A simple example is found below, where we are including a few system entities for an intent to travel.

To make this intent even more robust, we can make some more improvements by using inline entities:

Here, we are defining an inline entity that includes a broad set of US destinations that you can refer to. We do this by using an inline entity format: “@{… , , ….}:alias”. The intent defines two alisases - "place" and "date" - that will be returned to you in the JSON object. You can try it out by entering this example and saying something like: “I want to visit the Statue of Liberty on Jan 3rd”.

Similarly, inline entity syntax is used to define a number of synonyms: “@{want to, wanna, have to}”. Because there was no alias added at the end of this inline entity, nothing will be returned in the JSON. This approach can be used to easily add multiple variations in one intent and reduce the number of user expressions that you need to provide.

If you find that a system entity is missing - let us know and we will fix it!

Toyota Hackathon

2014-12-02T15:45:00.000-08:00

Hackers from around the Valley gathered at NestGSV last month for the 2014 Toyota ITC Connected Vehicle Hackathon – a 24-hour marathon programming session that gave developers a taste of working with the new Toyota API. API.AI shined brilliantly alongside other veteran sponsors such as Twilio, Sensory, SmartThings, and Pebble, impressing competing hackers and industry-leading partners alike. Within the first hour of the competition, Sleek-Geek founder JeanCarl Bisson tweeted: “Already got @api_ai working at the #toyotahackathon14. Very developer friendly!”

Toyota ITC Connected Vehicle Hackathon from NestGSV on Vimeo.

In person, the story was the same. Developers went from merely curious to outright enthusiastic about working with api.ai, which opened up seemingly endless creative possibilities for the long night ahead. With the clock ticking, the soft chatter slowly dwindled down to a hush, as the initially animated competitors settled in and quietly coordinated how to take this year’s crown.

By noon the next day (18 hours in), there was scarcely a word spoken among the exhausted hackers. Everyone was deep in the zone, fatigued but focused, and determined to power through. Out of 22 final submissions, over a third used api.ai, and the feedback from the developers was overwhelmingly positive. The judges had their work cut out for them, as the competition presented a wide range of unique and well-done integrations.

But when all was said and done, one stood above the rest. We are delighted to announce the winner for the best integration with api.ai (and a cool $1,000): Anushree Pole with CarTalk. CarTalk highlighted complex dialog interaction scenarios that allowed the user to converse with the car, in natural language, regarding the vehicle’s fuel status and other important features. Anushree gave a clever, humorous personality to the car, which added yet another dimension of natural interaction to the user experience.

“It was extremely easy to integrate api.ai in my Windows Phone App,” said Anushree. “The user interface for creating my queries was very intuitive. I was done integrating it in my app in an hour or so.”

Among the other noteworthy submissions were Down, Her, Ok Toyota, HyperConnected Car, and the Toyota Connected Dashboard Team. Anna Shi, the creator of Down, also had high praise for api.ai’s voice interface. “Overall I thought the api was very intuitive and easy to use,” she said, listing among her favorite features:

-Description of entities and intents

-Quickstart guide was short and sufficient

-Creating entities and intents were very intuitive

-Plenty of sample code to get a simple project started.

In the end, the competition was a tremendous success for everyone involved. We’d like to thank Toyota, NestGSV, all of the sponsors, hackers, and everyone who participated. Looking forward to the next big event!

Comcast Developer Experience with api.ai

2014-11-19T15:46:00.000-08:00

Mike Fine (Comcast):

“I recently used Speaktoit’s natural language processing API (www.api.ai) to build a voice- and text-chat web app that “knows about” television shows and television actors. The project was part of a Hack Week at Comcast’s Silion Valley Innovation Center.

In just over two days, I went from having never heard of Speaktoit or api.ai to a fully functional application that had an impressive set of features. Their website made it easy to sign up, discover my API keys, learn the key concepts of the system, and set up the “entities” (enumerated objects) and “intents” (commands) that form the core of my api.ai application. I was particularly impressed with how easy it was to manually test my entity and intent set up right from within their web application, without having to write any code. Once I got the core functionality of my app running, I used their REST API to programmatically create a thousand entities. Overall, their system was easy to learn, easy to use, and just plain worked.

If you’re thinking about building an application that involves natural language processing (voice- or text-based), check out Speaktoit’s API.AI. You’ll have something impressive up and running in no time.”

During the hackathon, Mike created an application that “knows about movies”. Here is an example of how a user could interact with it:

User: Show me movies with Brad Pitt

App: Displays list of movies with Brad Pitt

User: How about ones that also have Angelina Jolie?

App: Brings up movies with both Brad Pitt and Angelina Jolie

User: Nevermind. Show me comedies from 2002 to 2010.

App: Displays list

User: Ones with Louis CK

App: Provides a list of comedies with Louis CK from 2002-2010

Getting Started in 3 Minutes

2014-10-27T15:49:00.000-07:00

Here’s how to get started with api.ai in 3 mins:

We will be covering other functionality of api.ai in the upcoming videos. Let us know which tutorials / how-to’s you’d like to see next!