In-app tools

How it works

Turn it on for a channel

1. In the cabinet, open the channel: Integrations > Web for a website, or Integrations > Mobile for an Android or iOS app. A Flutter app has an Android and an iOS channel: turn the switches on for both.
2. Under In-app tools, turn on Use for chat: it lets the AI agent that answers the chats of this channel use the tools your website or app registers. Once it is on, the ready-made tools work too, all groups by default, and your developers turn groups off in code. To let the voice assistant use them during a call too, turn on Use for voice (see In a voice call). Use for voice needs Use for chat, because the voice assistant works inside the chat: turning chat off turns voice off too.
3. Click Save changes.

Choose the kind

Declare your tools

window.anecdoteLiveChatRegisterTool({
  name: 'list_devices',
  description: "Lists the devices signed in to the customer's account.",
  kind: 'read',
  execute: async () => {
    const response = await fetch('/api/devices', { credentials: 'include' });
    return { devices: await response.json() };
  },
});

window.anecdoteLiveChatRegisterTool({
  name: 'open_devices_page',
  description: 'Opens the devices page, when the customer asks to see their devices.',
  kind: 'safe_action',
  execute: async () => {
    router.push('/devices');
    return { opened: '/devices' };
  },
});

window.anecdoteLiveChatRegisterTool({
  name: 'remove_device',
  description: "Signs a device out of the customer's account.",
  kind: 'action',
  inputSchema: {
    type: 'object',
    properties: {
      device_id: { type: 'string', description: 'An id returned by list_devices' },
    },
    required: ['device_id'],
  },
  execute: async ({ device_id }) => {
    const response = await fetch(`/api/devices/${device_id}`, {
      method: 'DELETE',
      credentials: 'include',
    });
    return { removed: response.ok };
  },
});

Argument and result schemas

• Keywords: type, properties, required, items, enum and description.
• Types: object, array, string, number, integer and boolean.
• At most 4 levels of nesting, 32 properties per object and 50 enum values.
• A description inside a schema is at most 300 characters, and an enum value at most 100.
• Property names follow the same rule as tool names.
• confirmation_text is reserved for the consent card and cannot be an argument.

Limits

Ready-made tools

• Nothing passed: all seven groups are on.
• { location: false, ... }: those groups are off, the rest stay on.
• {} or false: all built-in tools are off.

var o = {
  "integrationKey": "YOUR_INTEGRATION_KEY",
  "organizationId": "YOUR_ORGANIZATION_ID",
  "mode": "live_chat",
  "widgetType": "live_chat",
  // All groups except location and forms.
  "inAppToolTemplates": {
    "location": false,
    "forms": false
  }
};

Options

"inAppToolTemplates": {
  "page": { "pageText": false },
  "siteSearch": true
}

Opening pages

// React Router
const navigate = useNavigate();
useEffect(() => {
  window.anecdoteLiveChatSetNavigator?.((path) => navigate(path));
  return () => window.anecdoteLiveChatSetNavigator?.(null);
}, [navigate]);

// Vue Router, after creating the router
window.anecdoteLiveChatSetNavigator((path) => router.push(path));

window.addEventListener('anecdote-live-chat:navigate', (event) => {
  event.preventDefault(); // "my app opens it"
  router.push(event.detail.path).then(event.detail.done, event.detail.fail);
});

Sites drawn by JavaScript

• read_link tells the agent the page cannot be read without opening it.
• find_on_site without words reads your sitemap (/sitemap.xml, or the sitemaps robots.txt names), a static file even on a single-page app, so the agent learns which pages your site has.
• find_on_site with words uses your site’s own search when it can read the results. It takes, in order, the searchUrl option, an OpenSearch description the page links to, or a schema.org SearchAction. When there is none, or its results are drawn by JavaScript, it matches the words against the page addresses of your sitemap.

• Your site only. read_link, find_on_site and open_link reach pages of the same origin as the page. Other sites are refused, and so is a page of yours that redirects to another site.
• Public pages only. read_link and find_on_site fetch a page without the customer’s cookies, so the agent reads what a signed-out visitor sees and never the customer’s own account pages.
• What read_page returns without asking. The page’s title, its main heading, the text the customer selected, the text of its links and menu, the names of its forms, fields and buttons, and the error alerts it shows (role="alert", role="alertdialog", aria-live="assertive"). On an account page that text can hold the customer’s name or an order number (“Hi, Jane”, “Order #1234”). No tool reads what they typed or picked in a field.
• The rest of the screen only with consent. get_page_text reads the page’s whole visible text, account details included, and asks the customer first every time: dialogs, status lines, toasts and banners are part of that text, not of read_page. It never reads what they typed or picked in a field, hidden elements or scripts. With page.pageText: false it is left out.
• Every field seen and filled, never read. get_form lists every field of the page’s forms the customer sees: the standard ones (a name, an organization, an email, a phone number, an address) by their autocomplete type, and the others (a date of birth, a free-text reason, a choice) by a field name taken from their label, with their kind, whether they are required and filled, and the choices a select or radio group offers. The agent fills any of them with what the customer says: it types a text or a date, picks a select option or a radio by the words the customer sees, and ticks or unticks a checkbox. It never returns what is typed or which choice is picked, never picks a file for the customer, and leaves out what is inside a private part (data-clarity-mask, data-clarity-exclude).
• Errors as your page shows them. get_form reads the conventions forms already use: aria-invalid with aria-errormessage or aria-describedby, the browser’s own validation (:user-invalid), and the error classes of common UI kits (Bootstrap, Material UI, Ant Design, Chakra). read_page reads role="alert" and role="status", live regions, open dialogs and toasts. Marking your errors this way, which accessibility asks for anyway, is all it needs.
• One field at a time. The agent focuses a field (the page scrolls to it, it pulses twice and stays highlighted in your widget’s primary colour), asks for its value and fills it, then moves to the next. The customer can also type into the highlighted field and tell the agent when done. Several values given at once are filled together. fill_form_fields takes one field flat (field and value), the shape a light voice model manages, or several in a fields list; a call with neither or both is refused. The highlight goes when the field is filled, when the customer turns to something else, or after two quiet minutes. In a voice call the agent spells an email or a phone number back before filling it.

Pressing for the customer

• is not on the page (the agent is then told the names it can use), is hidden, or is not a button, link or other control;
• is disabled (disabled, or aria-disabled="true");
• is a field to type in (a text input, a text area, a select): the agent is told to use focus_form_field or fill_form_fields instead;
• picks a file (input type="file");
• opens a new window (target="_blank" on a link or a form) or another app (mailto:, tel:), which browsers allow only from a real press;
• leads to another site, or sends its form to one;
• is inside a frame, which the page cannot reach;
• is inside an element marked private with data-clarity-mask or data-clarity-exclude;
• carries data-clarity-no-click, or sits inside an element that does.

<button data-clarity-no-click>Delete account</button>

What the customer sees

• Read: while the tool runs, the chat shows a short “checking in the app” status.
• Safe action: it runs at once, and the change shows on the page.
• Action: the chat shows a consent card. Each step on it is one sentence the agent writes, such as “Sign out the living room TV”. Under it are the tool’s name and the exact arguments, which the agent cannot word. The customer taps Allow or Decline.

In a mobile app, the chat steps aside

In a mobile app, confirm once on the device

In a voice call

1. In the cabinet, open the channel’s settings, where voice calling is turned on.
2. Under In-app tools, turn on Use for chat and Use for voice. Voice is greyed out while chat is off.
3. Click Save changes.

• Read: the assistant looks the data up and tells the customer the answer.
• Safe action: it runs at once, for example the devices page opens behind the call.
• Action: the call steps up and makes room for a short confirmation with Allow and Decline, with Allow focused, so a keyboard or screen reader user can answer with one key, and the assistant says once what the action will do and asks whether to go ahead. The confirmation lists each step on its own line. The customer can tap Allow, or just say yes. Only a clear yes said once the confirmation is up counts: a question, a hesitation, the request that led to the action or a yes said before never allows it. All the actions of one request share one confirmation. When they answer, the assistant tells them the outcome. A confirmation still waiting when the call ends is declined.

In a mobile app, the call folds to a bubble

Security

• Authorize in your API. A tool can do only what your API lets this customer do. Check every change on your server, as you would for the customer clicking the button themselves.
• Keep it least-privilege. Expose what a support conversation needs. Make anything that changes the customer’s data, money or settings an action, so the customer confirms it. Keep safe_action for what only changes the screen.
• The agent treats app data as data. Clarity marks every result and every error from your app as untrusted before the agent sees it, and the agent is told never to follow instructions inside it. A description is shown to the agent as your app’s description of the tool, capped in length, and an app tool cannot replace one of the agent’s own tools.
• Nothing is unlocked by a result. A result never grants the agent access to anything on the Clarity side, such as a tool your team configured or another customer’s data.
• The kind you register is the kind that runs. Clarity refuses a call below the kind your page or app registered, so a declaration altered on its way cannot turn an action into a read and skip the customer’s Allow.
• One Allow covers the steps it showed. Allow starts only the steps the card showed when the customer tapped it. A step that reached the card later asks again on a card of its own.
• Only your page and the chat reach your tools. The web widget takes tools and events only from the page that embeds it. The Android SDK (since 1.9.0) and the iOS SDK (since 1.6.0) do the same: only the chat page itself reaches your app, and a script inside a frame of the chat cannot call your tools.
• Logs keep out the data. Clarity logs the tool name and the call id, never the arguments, the result, or the chat’s session.

Troubleshooting