This feature is an add-on product and is not included in the standard subscription. It also requires that your organization have a signed Data Processing Agreement (DPA).
Our quality assurance module is powered by an AI engine, enabling you to review conversations more efficiently than a human can.
Quality Assistance helps ensure your agents consistently address mandatory topics, such as presenting themselves properly, obtaining consent from prospects, and informing them about prices, services, and conditions.
It is also helpful in gaining insight into dos and don’ts during conversations.
Need to know before activating AI-based Quality Assistance
The module is based on Gladia for transcriptions and Mistral for prompting your QA requirements.
For activating Quality Assistance, you must sign a Data Processing Agreement that includes both as subprocessors. It’s also a good idea to read about How Adversus uses AI to improve our features in general before continuing reading this article.
Signing the DPA is the first step to staying compliant. However, it is equally important that you comply with laws and regulations, such as those regarding data processing, data storage, and privacy policies. Adversus is constantly working to deliver compliant and safe solutions; however, we cannot accept any responsibility whatsoever for how our customers handle their data internally. Read more about our Trust and Compliance policies here.
How does the AI-based Quality Assistance feature work
In short terms, the QA works in the following steps:
Set up one or more AI transcriptions on your account – here you choose for which campaigns you want to enable transcription, pick the transcription language, etc.
Set up a QA model that defines which transcriptions you want reviewed and the requirements you want them reviewed against.
After creating and enabling one or more QA models, conversations that meet the defined criteria will be processed by the AI and appear in the QA review module.
Review the conversations in your QA session or one by one, and decide which conversations have passed or failed your requirement criteria.
Next, we will outline the various steps in the process, from setting up transcriptions to final review.
How to set up AI transcriptions of conversations and apply them to one or more campaigns
The AI transcription feature operates at the account level, allowing it to be used globally across all campaigns.
Go to Settings and Account
Choose AI transcriptions from the top menu
Click Create in the top right corner and Create transcription from the dropdown menu. This window will appear:
a) Transcription name: Give your transcription setting a recognizable name.
b) Enable transcription: Choose whether all conversations (longer than the minimum duration) should be transcribed Automatic or you want to do it Manually (through Warehouse or QAI – we’ll get back to how that’s done). We recommend choosing Automatic if you want to utilize the full power of AI-driven Quality Assurance. Manual transcription is advisable if you need to pause or gain more control over the transcriptions (e.g., due to compliance issues).
c) Duration requirement for transcription defines how long the conversation, at a minimum, should last before a transcription is created. By adding this, you avoid transcribing calls in which the prospect, e.g., ends the conversation immediately, or your agent hangs up because an answering machine is detected.
d) Summary Prompt lets you specify how you want your summaries, e.g., is there any specific information you want the summary to include, do you want it to be at a particular length, etc. If you don’t add a prompt, you’ll use Adversus’ default prompt, which is designed to brush up highlights in bullet points that we think are useful, such as the outcome of the conversation, or the reason why if ‘Not interested’.
e) Enable transcription for campaigns is mandatory. This is where you choose which campaigns you want transcriptions applied to. Please note that calls made on the selected campaigns before the transcription settings are enabled are not automatically transcribed.
f) Transcription language lets you define in which language you want your transcription. For example, if the conversation is in Swedish and you select English as your transcription language, the conversation will be automatically summarized in English.
g) Keywords are beneficial to add if you want the AI engine to be aware of certain words, such as your company name, products, etc.
h) Tags make it easier to locate and recognize this setting across Adversus, e.g., in the Warehouse or Leads modules.
4) Click Create transcription in the bottom right corner when you are done choosing settings. Now your transcription is running on the selected campaigns, unless you have chosen Manual transcription – in that case, you need to start the transcriptions manually through Warehouse.
Manual transcription - nice to know
Briefly explained, you can transcribe a call Manually in Warehouse → Calls by filtering and then choose Transcribe recording under Actions. Please note that only conversations in campaigns included in an AI transcription setup are available for transcription in Warehouse.
Manual transcription through Warehouse is especially relevant for calls made before a campaign was added to an AI transcription under the account settings.
You can also manually transcribe through the Quality Assurance module. We'll get back to that.
How to use the Quality Assistance module
Once a conversation is transcribed either automatically or manually, you can review it in the Quality Assistance module. The first step is to create a QA model that defines which conversations you want to review and the requirements those conversations must meet.
Go to Leads → Quality Assistance. The first time you enter the module, you’ll be presented with an empty list, because you need to create one or more QA Models before anything appears on it.
Go to QA model in the top menu and click Create QA model in the upper left corner. Now it’s time to set up your model.
We’ll take it step by step:
a) QA model name must be a name that makes the model easily recognizable when you are using it for QA review.
b) Add a Description that clarifies what this model is used for. This can, for example, be something like: “check if agents are introducing themselves and following instructions when presenting the product” or “check if the conversation lives up to compliance”.
c) Trigger Conditions define different criteria for adding a transcription to QA review:
Campaign filter is where you choose which AI transcription setting you want to use (the one you previously created under your account settings) - and thus which campaigns you want your QA model to run on.
Lead status filter is where you apply the lead statuses you want to review, e.g. applicable if you want to use the model to dig into all “not interested”.
Trigger rate sets the percentage of transcriptions that meet your criteria.
Minimum length (seconds) defines the minimum conversation length for a session to be included in a QA review.
d) Finally, you must create the Requirements you want to evaluate your transcriptions against. Click New requirement to start building your prompts and define their settings.
How requirements work and how to create them
Requirements are specified by prompting the criteria you want the AI engine to consider when reviewing the transcription of a conversation.
Best practice is to make your prompt as straightforward as possible. Try to phrase only one command at a time - that's better than trying to create one requirement that covers it all.
We recommend using our helpful bot, Arthur. Arthur can guide you and suggest requirements that are easy for the AI engine to handle.
You can reuse requirements across your QA models by adding Existing requirements to your model (if you've already created some). You can also edit your requirements. Please note that editing a requirement used across multiple models may affect your review processes.
Requirement settings
Name your requirement so it’s easy to recognize. We suggest you give it a name that reflects its purpose.
Description is where you specify the actual prompt/instruction for the AI. Phrase it as specifically as possible and ensure the prompt only describes one single action
Choose if the prompt is something the agent Must do or Must not do during the conversation. Using this function makes it easier to distinguish between dos and don'ts.
Evaluation scope defines in which part of the conversation the requirement must be fulfilled. Since the AI is screening all conversations (if there have been more than one) on the lead, there are different scope options.
Choose Any conversation if the requirement must be fulfilled at any point of any conversation recorded on the lead.
Choose Closing conversation if the requirement must be fulfilled during the last conversation (e.g. useful if you only want to check successes).
Choose All conversation if the requirement must be fulfilled in all conversations, e.g., checking whether agents are presenting themselves every time they call the lead.
Finally, decide how the requirement should be evaluated. Choose Require human evaluation if human review should be mandatory. A human review is not mandatory if you Use AI to generate outcome – but you can always overrule the AI-generated outcome during the evaluation. You can add as many requirements to your QA model as you want.
Optimize requirements with Arthur
Below the AI prompt in a requirement, there is a button to optimize the requirement/prompt with Arthur, our helpful AI robot. If you click the button, you will open a conversation with Arthur to discuss improving a requirement. Messages from Arthur will be tagged with badges indicating either:
Question, which are clarifying questions from Arthur to you, or
Suggestion: a concrete improvement to the requirement prompt.
How to test requirements
You can preview your requirements on the QA-model setup page by clicking the Preview button on the right, which lets you test the quality of your requirements immediately. A modal will open, allowing you to select a specific call/transcription to evaluate and choose which requirements you want to test.
With the preview functionality, you can easily test and quickly verify how well your requirements/AI prompts perform in different conversations and contexts before committing fully to them.
1) Enter the Call ID you want to use for your test
2) Select the Requirements you want to test. It's possible to add requirements that are not already included in your model.
How to review the transcriptions
Once you’ve set up your QA model(s), you are ready to start reviewing.
Go to QA review.
On the right, choose which QA model you want to review conversations from. Additionally, you can filter this list by using Lead Filters, allowing you to pick a specific campaign or agent you want to review.
Now you’re ready to review. There are different ways of reviewing:
Click any lead on the list with the status Pending to review a single conversation.
Click Start QA Session to review all leads available in the list shown in your QA flow.
A pop-up window with various information will appear:
Lead data (in the top menu): Shows all lead data.
Requirements (in the top menu): Link to all recording(s) of all conversations, the transcriptions, summaries, etc., and a list of the requirements you are checking for.
To the right under Requirements, you can wing off whether the conversation(s) have passed or failed each requirement separately. It’s not mandatory to check each requirement if your QA model does not require Human review. Leave a note to the agent if you want.
When done, decide whether the conversation(s) has Failed or Passed. Once it has been evaluated, it will disappear from the QA review overview.
Warehouse for Quality Assistance
When you have conducted a review and marked the conversation(s) as either Passed or Failed, you can find them in Warehouse → Quality Assistance. Its purpose and functionality are similar to those of the other Warehouse pages, acting as the data warehouse for all QA reviews and conversations.