Using Tokens

How do I use tokens?

Ezra allows the use of tokenised texts which are replaced with specific values returning at runtime, where the script behind the token will execute and populate the content.

Tokens range from the simple {{ key }} , through to full {{ key(alias):category(subcategory) }}. They are organised by various “Dictionaries” for the context they are used. Below are some common Dictionaries and useful tokens that you can use.

Answering Dictionary (AnswerDict) :

These are use for tokenising answers returned by the Knowledge Base.

They can also be used for Answer Grammars and responses from API and queries.

{{ nouns:asked }}

This token will return the nouns in the utterance, in the form “a, b, c, or d”

{{ verb }}

This token will return the verb (or first verb) in the utterance

{{ verb:pasttense }}

This token will return the past tense of a verb in the utterance

{{ person }}

This token return a list of person words in utterance (“a, b, c, or d”)

{{ person:firstname }}

This token will return a list of firstnames in utterance

{{ person:lastname }}

This token will return a list of lastnames in utterance

{{ person:nickname }}

This token will return a list of nicknames in utterance

{{ person:honorific }}

This token will return a list of honorifics in utterance (Mr. Mrs. etc)

{{ person:fullname }}

This token will return a list of names in utterance, aggregating contiguous [honorific, firstname, lastname] phrases.

For example the phrase “John and Doe Steven Smith Mr Matt Brown” will return [John, Doe, Steven Smith, Mr Matt Brown].

Additionally, the phrase “Sarah Lee Martin” would return [Sarah Lee, Martin]. Ezra will do a best match on firstname lastname pairs, even when some words can be both.

{{ location }}

This token will return a list of locations in utterance

{{ category }}

This token will return the category of the answer

{{ resourceurl }}

This token will return the resource url of the answer. Alias for this token: {{ pageurl }}, {{ imageurl }}, and {{ apiendpoint }}

{{ content }}

This token will return the content of the answer from the Knowledge Base. Alias for this token: {{ selection }}

{{ failover }}

This token will call the Failover services active with the asked text or utterance. It will look to 3rd-party services to answer questions when Ezra’s Knowledge Base returns no usable answer.

URL Dictionary (UrlDict):

These are used for tokenising urls (API calls).

While keys are dynamic, some reserved keywords are available.

These can be overwritten for a particular answer, if they share the same name as an “asTag”.

{{ <key> }}

These dynamic keys are what the URLDict convert into part of the URL before calling the API endpoint. You can use these to structure the query-string part of the URL (the bit after the ?-mark). With the exception of a few special keys (see below) the key name will come from the “AsTag” column in your tagset for the answer. The values of the key will be the word that the tag matches. For example, an answer with a tagset such as:

step 1

will match someone who asks about Bizdata or Microsoft, and the token {{ company }} will be replaced as ?company=Bizdata, or ?company=Microsoft as appropriate

{{ firstname }}, {{ lastname }}, {{ nickname }}, {{ honorific }}, {{ fullname }}

These special keys are similar to the person tokens in the AnswerDict, in that they return the names in the utterance, but as keys automatically. This avoids the need to tag all names in the Dictionary for the answer in the tags (which would be very impractical). Note that they will be overwritten by specific asTags with the same name.

{{ <key>(<alias>) }}

Adding an alias in parentheses will render the query-string in the URL using the alias instead of the key name. Perhaps the API is asking for a specific word in the query-string that you don’t have a specific asTag for. For example, in the tagset above, if the token was {{ company(businessname) }} the resulting query-strings might be ?businessname=Bizdata or ?businessname=Microsoft instead.

:first {{ key:first }}, or no category {{ key }}

This token processes only the first word matched in the list for that key: “?key=word”, or if an alias is given {{ key(alias}:first }} “?alias=word”

:foreach

This token sets up multiple calls to be made for each word in the key with the same structure as :first. These calls are then resolved together for the answer.

:asarray

This token sets up one call with all the words set up as an array for the call. However, there isn’t really a standard for URL queries with an array. Therefore, subcategories can be used to structure your query to match your endpoint’s expected format.

:asarray(none) or :asarray

This token will return: “?key=word1&key=word2&key=word3” etc.

:asarray(php)

This token will return: “?key=word1[]&key=word2[]&key=word3[]” etc.

:asarray(index)

This token will return: “?key=word1[0]&key=word2[1]&key=word3[2]” etc.

:asarray(comma)

This token will return: “?key=word1,word2,word3” etc.

:after

This token finds all the words in the text string after the (last) key word, and creates a search-like query string: “?key=all+the+words+afterwards” Again, alias can be used instead of the key word, if required.

An API call is expected to return a structure of a simple dictionary or an array of simple dictionaries. E.g., { “keyA”: “valueA”, “keyB”, “valueB”, … } The answer (content section of Knowledge Base) can use these tokens along with the original asTags and the AnswerDict above to create the answers. Returned values will overwrite older values, such as when a returned value has a field the same as the asTag, this new value would be the value used. Please note, :foreach call results display in multi-bubble answers.

Query Dictionary (QueryDict):

For processing query bodies.

Usage is almost the same as UrlDict except for the following:

Use :astable instead of :asarray.

Sub categories “start”, “middle”, and “end” are available for all categories (:first, :foreach, :astable).

Add “%” marks to the start or end of the word to match, if partial searches are required.

There is no :after category.

Returned table can be referenced by using the names of the columns as tokenised keys.

Each row may result in a different answer bubble.

Action Flow

This dictionary is entirely dynamic. Within the free text field of nodes – Action nodes in particular – the “moustache” tokens, {{ }} , can specify which ‘dictionary’ the words can come from.

In every Form node, the text patterns are a regular expression with a capture block. What they capture from the user is available in subsequent nodes; the keys being the name of the text pattern. Note that if the same text pattern is used multiple times, the final result will be the last text captured by that pattern. Also, when it comes to keys, no subcategories are used in the Action Flow Dictionary.

Here are the keys used within Action Flow nodes:

{{key:frompayload}} : specifies that the value is from the payload dictionary (those captured in local Form nodes)

{{key:frominternalstate}} : specifies the value is from the global variable dictionary

{{key:fromresult}} : specifies it is a value from a selected column in the most recently run database query.

These keys also have smaller and more sensible aliases respectively:

{{key:local}}

{{key:global}}

{{key:query}}

Note: Be careful with non-linear flows. Ezra will not crash if it tries to use a block that does not exist, or has no information in it - it will just be blank. Similarly, some patterns may be marked as optional, if these are called they may also be blank.