- An order tracking bot
- Pre-flight check before creating a bot
- Release an intent from staging to production
- A two-turn booking dialog with wait replay
- Offboarding a user of a bot
- Smoke test of a FAQ bot with match scores
- A bot that remembers the customer's store
- Premium intents gated by user rights
- Version an intent and retire the old version
- Hand off to a human below a confidence score
An order tracking bot
Create an English bot and a customer account, teach it an intent that captures the order number in the sentence and stores it in a user variable, grant the intent and talk to the bot. It writes bots in the server database, so it is not executed here.
#1. An English female bot and a first customer account; bot create "lisa" "en" 0 "Lisa" "Martin" "cancel_key" "Sorry, I do not understand. I am forwarding your request to a human agent."; bot user_create "lisa" "jdoe" "pwd" "{\"[user_firstname]\": \"John\", \"[user_lastname]\": \"Doe\"}"; #2. Intent track_order: the order number of the sentence is captured in [1]; json load "trigger" "[]"; json iarray "trigger" / "where is my order [1]" STR; json iarray "trigger" / "track the order [1]" STR; json load "consciousness" "{\"done\": [\"I gave the status of the order [1].\"], \"think\": [\"I thought I had to track the order [1].\"], \"understand\": [\"I understood I had to track the order [1].\"], \"subject\": [\"it was about tracking the order [1].\"]}"; bot training_merge "lisa" "track_order" "orders" "Give the status of an order" 100 (json doc "trigger") (mql { json load "vars" [user_variables]; bot user_set_var [bot] [user] "[last_order]" (json select "vars" "/[0]"); bot set_wait_replay [bot] [user] null; concat "Hello " [user_firstname] ", order " (json select "vars" "/[0]") " left our warehouse yesterday."; }) (json doc "consciousness") ; #3. Grant the intent to the user, rebuild the index and talk; bot right_add "lisa" "jdoe" "track_order" "100"; bot reload "lisa"; bot execute "lisa" "jdoe" "where is my order 4512"; #4. The bot remembers the last order of the customer; bot user_get_var "lisa" "jdoe" "[last_order]"
bot set_wait_replaykeeps the conversation on an intent until the answer is complete.- List the users of the bot with
bot user_show.
Pre-flight check before creating a bot
Before deploying a new bot, count the bots of the server and check whether the name is already used; if it is, describe the bot, its users and its main intent.
#1. Name of the bot we want to deploy and bots already on the server; -> "[bot]" "rx_helpdesk"; json load "rx_bots" (bot show); -> "[n]" (json count "rx_bots" /); json unload "rx_bots"; #2. Existing bot: describe it, else confirm the name is free; if (== (bot exist [bot]) 1) { concat "Bot " [bot] " already exists: " (bot get [bot]) " | users: " (bot user_show [bot]) " | intent greeting trained: " (bot training_exist [bot] "greeting") " | version 100: " (bot training_version_exist [bot] "greeting" 100); } { concat "The name " [bot] " is free (bots on this server: " [n] "), safe to run bot create."; }
The name rx_helpdesk is free (bots on this server: 0), safe to run bot create.
- In the editor,
bot training_searchwith SQL like patterns opens the merge scripts of the matching trainings. bot user_existchecks an account beforebot user_create.
Release an intent from staging to production
Back up version 100 of an intent as MQL, push it to the production server, pull back a hotfix, drop the obsolete draft and its rights, rotate a tester password and show the state of the tester. It changes bots on two servers, so it is not executed here.
#1. Staging bot: list the versions of the intent, keep a replayable MQL copy of version 100; bot training_search "lisa_staging" "track_order" "%%" "%%" "%%" "%%" "%%"; -> "[last]" (bot training_get "lisa_staging" "track_order"); -> "[v100]" (bot training_version_get "lisa_staging" "track_order" 100); if (== (bot training_version_exist "lisa_staging" "track_order" 100) 1) { -> "[backup_v100]" (bot training_version_generate_merge "lisa_staging" "track_order" 100); -> "[backup_last]" (bot training_generate_merge "lisa_staging" "track_order"); }; #2. Release: push version 100 to production, pull back the hotfix of the greeting; bot training_push "lisa_staging" "track_order" 100 (cm get "prod_cm_mentdb"); bot training_pull (cm get "prod_cm_mentdb") "lisa_staging" "greeting" 102; #3. Drop the obsolete draft 99 and the rights on it; if (== (bot right_check "lisa_staging" "beta1" "track_order" "99") 1) { bot right_delete "lisa_staging" "beta1" "track_order" "99"; }; bot training_version_delete "lisa_staging" "track_order" 99; #4. Rotate the password of the beta tester, remove the old tester; if (== (bot user_exist "lisa_staging" "beta1") 1) {bot user_set_password "lisa_staging" "beta1" "new_pwd";}; bot user_delete "lisa_staging" "beta0"; #5. Remove the throwaway sandbox bot; bot delete "lisa_sandbox"; #6. State of the beta tester after the release; concat "intent: " [last] " | users: " (bot user_show "lisa_staging") " | beta1: " (bot user_get "lisa_staging" "beta1") " | rights: " (bot right_show "lisa_staging" "beta1") " | variables: " (bot user_get_vars "lisa_staging" "beta1") " | waiting for: " (bot get_wait_replay "lisa_staging" "beta1") " | bot: " (bot get "lisa_staging")
- Compare [last] and [v100] to review the changes before the push.
bot reloadrebuilds the index after the release.
A two-turn booking dialog with wait replay
A restaurant bot asks for the number of guests: the first intent sets a wait replay on the follow-up intent, which captures the answer of the user, stores it in a user variable and clears the wait. The script grants both intents, reloads the bot and plays the two turns.
#1. Intent book_table asks a question and waits for the answer of the user; json load "trigger" "[]"; json iarray "trigger" / "book a table for tonight" STR; json iarray "trigger" / "I want to book a table" STR; json load "consciousness" "{\"done\": [\"I asked for the number of guests.\"], \"think\": [\"I thought I had to book a table.\"], \"understand\": [\"I understood I had to book a table.\"], \"subject\": [\"it was about a table booking.\"]}"; bot training_merge "lisa" "book_table" "restaurant" "Start a table booking" 100 (json doc "trigger") (mql { bot set_wait_replay [bot] [user] "book_guests"; concat "With pleasure " [user_firstname] ", for how many people?"; }) (json doc "consciousness") ; #2. Follow-up intent book_guests: the whole answer is captured in [1]; json load "trigger" "[]"; json iarray "trigger" / "[1]" STR; json load "consciousness" "{\"done\": [\"I booked a table for [1].\"], \"think\": [\"I thought it was the number of guests.\"], \"understand\": [\"I understood [1] guests.\"], \"subject\": [\"it was about the number of guests.\"]}"; bot training_merge "lisa" "book_guests" "restaurant" "Number of guests of a booking" 100 (json doc "trigger") (mql { json load "vars" [user_variables]; bot user_set_var [bot] [user] "[guests]" (json select "vars" "/[0]"); bot set_wait_replay [bot] [user] null; concat "Done, a table for " (json select "vars" "/[0]") " is booked at 8 pm."; }) (json doc "consciousness") ; #3. Rights of the customer on both intents, then rebuild the index; bot right_add "lisa" "jdoe" "book_table" "100"; bot right_add "lisa" "jdoe" "book_guests" "100"; bot reload "lisa"; #4. Two-turn dialog: the bot waits for the answer between the turns; bot execute "lisa" "jdoe" "I want to book a table"; -> "[waiting]" (bot get_wait_replay "lisa" "jdoe"); bot execute "lisa" "jdoe" "4"; concat "waiting intent after turn 1: " [waiting] " | variables of the user: " (bot user_get_vars "lisa" "jdoe")
- Read the stored answer later with
bot user_get_varto confirm the booking. - Check the intent with
bot training_existbefore the merge.
Offboarding a user of a bot
A support agent leaves the company. Check the account, keep an audit trace of the user, rights and variables, revoke the sensitive intent refund_order, lock the account with a random password and delete it, then reload the bot.
#1. A support agent leaves: check the account of the bot first; -> "[bot]" "lisa"; -> "[login]" "mjones"; if (== (bot user_exist [bot] [login]) 0) {exception (1) ("Unknown user of the bot");}; -> "[profile]" (bot user_get [bot] [login]); #2. Keep an audit trace of the rights and variables before removing them; -> "[rights]" (bot right_show [bot] [login]); -> "[vars]" (bot user_get_vars [bot] [login]); #3. Revoke the sensitive intent refund_order if it was granted; if (== (bot right_check [bot] [login] "refund_order" "100") 1) { bot right_delete [bot] [login] "refund_order" "100"; }; #4. Lock the account with a random password, then delete it; bot user_set_password [bot] [login] (string generate_random_str 32); bot user_delete [bot] [login]; bot reload [bot]; concat "user " [login] " removed from " [bot] " | profile " [profile] " | rights before removal " [rights] " | variables " [vars]
- Use
bot right_checkon every intent ofbot training_searchto list all the rights. - Write the audit trace into a log with
log writebefore the deletion.
Smoke test of a FAQ bot with match scores
A small FAQ bot (opening hours, returns) is created, indexed and tested with two paraphrases and one out-of-scope question. For each sentence the script reads the matched intent and the score of the best trigger in the search log, then deletes the bot. The out-of-scope question gets no intent: it is below the minimum score of the bot.
#1. A temporary FAQ bot and a test customer; bot create "tuto_p4h_faq" "en" 0 "Ava" "Stone" "cancel_key" "Sorry, I did not get that. A colleague will answer you."; bot user_create "tuto_p4h_faq" "tuto_p4h_kim" "pwd" "{\"[user_firstname]\": \"Kim\", \"[user_lastname]\": \"Lee\"}"; #2. Two intents: opening hours and returns; json load "rx_p4h_cons" "{\"done\": [\"I answered.\"], \"think\": [\"I thought you asked a question.\"], \"understand\": [\"I understood a question.\"], \"subject\": [\"it was about the shop.\"]}"; bot training_merge "tuto_p4h_faq" "opening_hours" "shop" "Opening hours" 100 "[\"what are your opening hours\", \"when are you open\"]" (mql {concat "We are open 9 am to 7 pm, Monday to Saturday, " [user_firstname] ".";}) (json doc "rx_p4h_cons"); bot training_merge "tuto_p4h_faq" "return_policy" "shop" "Return policy" 100 "[\"how can I return a product\", \"can I get a refund\"]" (mql {"You have 30 days to return a product with its receipt.";}) (json doc "rx_p4h_cons"); bot right_add "tuto_p4h_faq" "tuto_p4h_kim" "opening_hours" "100"; bot right_add "tuto_p4h_faq" "tuto_p4h_kim" "return_policy" "100"; -> "[index]" (bot reload "tuto_p4h_faq"); #3. Smoke test: paraphrases and an out of scope question, score of the best trigger; -> "[report]" (concat "index: " (string regexp_extract [index] "Trigger: [0-9]+; Training: [0-9]+" 0)); json load "rx_p4h_tests" "[\"are you open on saturday\", \"I want a refund\", \"what is the weather in Paris\"]"; for (-> "[i]" 0) (< [i] 3) (++ "[i]") { json load "rx_p4h_r" (bot execute "tuto_p4h_faq" "tuto_p4h_kim" (json select "rx_p4h_tests" (concat "/[" [i] "]"))); -> "[score]" (string regexp_extract (json select "rx_p4h_r" "/search_log") ": ([0-9.]+)" 1); -> "[report]" (concat [report] " | " (json select "rx_p4h_tests" (concat "/[" [i] "]")) " -> " (string regexp_extract (json select "rx_p4h_r" "/search_log") ">>> ([a-z_]+)/" 1) " score " [score]); }; #4. Remove the temporary bot; bot user_delete "tuto_p4h_faq" "tuto_p4h_kim"; bot delete "tuto_p4h_faq"; [report]
index: Trigger: 4; Training: 2 | are you open on saturday -> opening_hours score 320.0 | I want a refund -> return_policy score 378.0 | what is the weather in Paris -> score
- The out-of-scope question scores below the minimum score of the bot (200 by default, see
bot set_min_score): bot execute gives the not found response and records the sentence. - Keep the test sentences in a JSON file and run them after each
bot training_merge. - Add more triggers to the intents whose paraphrases score low.
A bot that remembers the customer's store
A pharmacy bot captures the favourite store of the customer in a slot ([1]) and stores it in the user profile. In a later question the bot reuses the stored value to answer.
#1. A temporary pharmacy bot and a customer; bot create "tuto_p4h_pharma" "en" 1 "Max" "Field" "cancel_key" "Sorry, I did not understand."; bot user_create "tuto_p4h_pharma" "tuto_p4h_ana" "pwd" "{\"[user_firstname]\": \"Ana\", \"[user_lastname]\": \"Silva\"}"; json load "rx_p4h_cons" "{\"done\": [\"I noted it.\"], \"think\": [\"I thought you gave me a store.\"], \"understand\": [\"I understood a store.\"], \"subject\": [\"it was about your store.\"]}"; #2. Intent 1 captures the favourite store in [1] and stores it in the profile; bot training_merge "tuto_p4h_pharma" "set_store" "profile" "Remember the store" 100 "[\"my store is [1]\", \"I usually go to [1]\"]" (mql { json load "rx_p4h_v" [user_variables]; bot user_set_var [bot] [user] "[store]" (json select "rx_p4h_v" "/[0]"); concat "Noted, " [user_firstname] ": your store is " (json select "rx_p4h_v" "/[0]") "."; }) (json doc "rx_p4h_cons"); #3. Intent 2 reuses the stored variable in a later conversation; bot training_merge "tuto_p4h_pharma" "pickup" "orders" "Where to pick up" 100 "[\"where can I pick up my prescription\"]" (mql { concat "Your prescription will be ready tomorrow at the " (bot user_get_var [bot] [user] "[store]") " store."; }) (json doc "rx_p4h_cons"); bot right_add "tuto_p4h_pharma" "tuto_p4h_ana" "set_store" "100"; bot right_add "tuto_p4h_pharma" "tuto_p4h_ana" "pickup" "100"; bot reload "tuto_p4h_pharma"; -> "[a1]" (json load "rx_p4h_r1" (bot execute "tuto_p4h_pharma" "tuto_p4h_ana" "I usually go to Riverside")); -> "[a2]" (json load "rx_p4h_r2" (bot execute "tuto_p4h_pharma" "tuto_p4h_ana" "where can I pick up my prescription")); -> "[vars]" (bot user_get_vars "tuto_p4h_pharma" "tuto_p4h_ana"); #4. Clean up; bot user_delete "tuto_p4h_pharma" "tuto_p4h_ana"; bot delete "tuto_p4h_pharma"; concat "turn 1: " (json select "rx_p4h_r1" "/msg") " | turn 2: " (json select "rx_p4h_r2" "/msg") " | profile: " [vars]
turn 1: Noted, Ana: your store is Riverside. | turn 2: Your prescription will be ready tomorrow at the Riverside store. | profile: {
"[user_lastname]": "Silva",
"[user_last_request]": "where can I pick up my prescription",
"[user_firstname]": "Ana",
"[store]": "Riverside"
}- Ask the store when it is missing: test
bot user_get_varand usebot set_wait_replayto wait for the answer. - Seed known customers with
bot user_createand a JSON of variables from your CRM. - Clear a variable with
bot user_set_varand an empty value when the customer moves.
Premium intents gated by user rights
A bank bot offers a call back by a private advisor to premium customers only. The same sentence is sent by a standard and a premium customer, the rights are checked, then the premium right is removed (downgrade).
#1. A bank bot with a standard and a premium customer; bot create "tuto_p4h_bank" "en" 0 "Nora" "Klein" "cancel_key" "Sorry, this service is not available for your account."; bot user_create "tuto_p4h_bank" "tuto_p4h_std" "pwd" "{\"[user_firstname]\": \"Tom\", \"[user_lastname]\": \"Ray\"}"; bot user_create "tuto_p4h_bank" "tuto_p4h_vip" "pwd" "{\"[user_firstname]\": \"Lia\", \"[user_lastname]\": \"Moss\"}"; json load "rx_p4h_cons" "{\"done\": [\"I answered.\"], \"think\": [\"I thought you asked for an advisor.\"], \"understand\": [\"I understood an advisor.\"], \"subject\": [\"it was about an advisor.\"]}"; bot training_merge "tuto_p4h_bank" "advisor_call" "premium" "Call back by a private advisor" 100 "[\"I want to talk to my advisor\", \"call me back\"]" (mql {concat "Your private advisor will call you within the hour, " [user_firstname] ".";}) (json doc "rx_p4h_cons"); bot training_merge "tuto_p4h_bank" "balance" "account" "Balance" 100 "[\"what is my balance\"]" (mql {"Your balance is shown in the app under Accounts.";}) (json doc "rx_p4h_cons"); #2. Rights: both have balance, only the VIP has the advisor; bot right_add "tuto_p4h_bank" "tuto_p4h_std" "balance" "100"; bot right_add "tuto_p4h_bank" "tuto_p4h_vip" "balance" "100"; bot right_add "tuto_p4h_bank" "tuto_p4h_vip" "advisor_call" "100"; bot reload "tuto_p4h_bank"; #3. Same sentence, two users; json load "rx_p4h_s" (bot execute "tuto_p4h_bank" "tuto_p4h_std" "I want to talk to my advisor"); json load "rx_p4h_v" (bot execute "tuto_p4h_bank" "tuto_p4h_vip" "I want to talk to my advisor"); -> "[check]" (concat "std " (bot right_check "tuto_p4h_bank" "tuto_p4h_std" "advisor_call" "100") ", vip " (bot right_check "tuto_p4h_bank" "tuto_p4h_vip" "advisor_call" "100")); #4. Downgrade of the VIP: remove the right; bot right_delete "tuto_p4h_bank" "tuto_p4h_vip" "advisor_call" "100"; -> "[after]" (bot right_show "tuto_p4h_bank" "tuto_p4h_vip"); bot user_delete "tuto_p4h_bank" "tuto_p4h_std"; bot user_delete "tuto_p4h_bank" "tuto_p4h_vip"; bot delete "tuto_p4h_bank"; concat "standard: " (json select "rx_p4h_s" "/msg") " | vip: " (json select "rx_p4h_v" "/msg") " | right_check advisor_call: " [check] " | vip rights after downgrade: " [after]
standard: Sorry, you don't have rights on tuto_p4h_bank/tuto_p4h_std/advisor_call/100 | vip: Your private advisor will call you within the hour, Lia. | right_check advisor_call: std 0, vip 1 | vip rights after downgrade: [
{
"version": "100",
"key": "balance",
"md5": "d3e41f59654b97f3e9b1225b25567b5b"
}
]- List the rights of a user with
bot right_showbefore a support call. - Grant a new version (200) of an intent to a pilot group only, then to everybody.
- Write the refusal message in the not found response of
bot create.
Version an intent and retire the old version
The delivery intent changes with a new carrier. Version 200 is added next to version 100 (new answer, one more trigger), both are compared, then version 100 is retired while the intent stays available.
#1. A temporary bot with version 100 of the delivery intent; bot create "tuto_p4h_ship" "en" 0 "Iris" "Hale" "cancel_key" "Sorry, I did not understand."; json load "rx_p4h_cons" "{\"done\": [\"I gave the delivery time.\"], \"think\": [\"I thought you asked for the delivery time.\"], \"understand\": [\"I understood the delivery time.\"], \"subject\": [\"it was about the delivery.\"]}"; bot training_merge "tuto_p4h_ship" "delivery_time" "shipping" "Delivery time" 100 "[\"how long is the delivery\"]" (mql {"Delivery takes 3 to 5 days.";}) (json doc "rx_p4h_cons"); #2. Version 200: new carrier, new answer, one more trigger; bot training_merge "tuto_p4h_ship" "delivery_time" "shipping" "Delivery time with the new carrier" 200 "[\"how long is the delivery\", \"when will I get my parcel\"]" (mql {"Delivery takes 24 to 48 hours.";}) (json doc "rx_p4h_cons"); -> "[v1]" (bot training_version_exist "tuto_p4h_ship" "delivery_time" 100); -> "[v2]" (bot training_version_exist "tuto_p4h_ship" "delivery_time" 200); json load "rx_p4h_last" (bot training_version_get "tuto_p4h_ship" "delivery_time" 200); json load "rx_p4h_old" (bot training_version_get "tuto_p4h_ship" "delivery_time" 100); #3. Retire version 100 once 200 is validated; bot training_version_delete "tuto_p4h_ship" "delivery_time" 100; -> "[v1_after]" (bot training_version_exist "tuto_p4h_ship" "delivery_time" 100); -> "[still]" (bot training_exist "tuto_p4h_ship" "delivery_time"); bot delete "tuto_p4h_ship"; concat "versions 100/200 before: " [v1] "/" [v2] " | version 200: " (json select "rx_p4h_last" "/desc") ", triggers " (json select "rx_p4h_last" "/in_trigger_array") " | version 100: " (json select "rx_p4h_old" "/desc") " | after retiring 100: v100=" [v1_after] ", intent still exists=" [still]
versions 100/200 before: 1/1 | version 200: Delivery time with the new carrier, triggers ["how long is the delivery", "when will I get my parcel"] | version 100: Delivery time | after retiring 100: v100=0, intent still exists=1
- Generate the MQL of a version with
bot training_version_generate_mergeto keep it in Git. - Push a validated version to production with
bot training_push. - Move the users from version 100 to 200 with
bot right_deleteandbot right_add.
Hand off to a human below a confidence score
A support bot answers only when the best trigger scores at least 200; below, the conversation is routed to a human agent. Three customer sentences are routed: exact, paraphrase and out of scope.
#1. Temporary support bot and a customer; bot create "tuto_p4h_help" "en" 0 "Zoe" "Park" "cancel_key" "Sorry, I did not understand."; bot user_create "tuto_p4h_help" "tuto_p4h_sam" "pwd" "{\"[user_firstname]\": \"Sam\", \"[user_lastname]\": \"Cole\"}"; json load "rx_p4h_cons" "{\"done\": [\"I answered.\"], \"think\": [\"I thought it was a password problem.\"], \"understand\": [\"I understood a password problem.\"], \"subject\": [\"it was about the password.\"]}"; bot training_merge "tuto_p4h_help" "reset_password" "account" "Reset the password" 100 "[\"I forgot my password\", \"reset my password\"]" (mql {"Click on Forgot password on the login page, a link is sent by e-mail.";}) (json doc "rx_p4h_cons"); bot right_add "tuto_p4h_help" "tuto_p4h_sam" "reset_password" "100"; bot reload "tuto_p4h_help"; #2. Router: below a score of 200 the conversation goes to a human; -> "[threshold]" 200; -> "[log]" ""; json load "rx_p4h_q" "[\"I forgot my password\", \"my password does not work anymore\", \"my parcel arrived broken\"]"; for (-> "[i]" 0) (< [i] 3) (++ "[i]") { json load "rx_p4h_r" (bot execute "tuto_p4h_help" "tuto_p4h_sam" (json select "rx_p4h_q" (concat "/[" [i] "]"))); -> "[score]" (string regexp_extract (json select "rx_p4h_r" "/search_log") ": ([0-9.]+)" 1); -> "[route]" (if (>= [score] [threshold]) {concat "bot: " (json select "rx_p4h_r" "/msg")} {"HUMAN AGENT"}); -> "[log]" (concat [log] " | " (json select "rx_p4h_q" (concat "/[" [i] "]")) " (" [score] ") -> " [route]); }; bot user_delete "tuto_p4h_help" "tuto_p4h_sam"; bot delete "tuto_p4h_help"; concat "threshold " [threshold] [log]
threshold 200 | I forgot my password (430.0) -> bot: Click on Forgot password on the login page, a link is sent by e-mail. | my password does not work anymore (218.0) -> bot: Click on Forgot password on the login page, a link is sent by e-mail. | my parcel arrived broken () -> HUMAN AGENT
- The bot itself applies a minimum score (
bot set_min_score, 200 by default): below it, bot execute already answers its not found response, so the handoff can also test the type ko. - Calibrate the threshold on a labelled test set: score of the right intent vs score of out-of-scope sentences.
- Push the handed-off sentences to a stack script that opens a ticket.
- Use the handed-off sentences as new triggers after review.
