This feature is an add-on product and is not included in the standard subscription. It also requires that your organization have a signed Data Processing Agreement (DPA).
Our quality assurance module is powered by an AI engine, enabling you to review conversations more efficiently than a human can.
Quality Assistance helps ensure your agents consistently address mandatory topics, such as presenting themselves properly, obtaining consent from prospects, and informing them about prices, services, and conditions.
It is also helpful in gaining insight into dos and don’ts during conversations.
Need to know before activating AI-based Quality Assistance
The module is based on Gladia for transcriptions and Mistral for prompting your QA requirements.
For activating Quality Assistance, you must sign a Data Processing Agreement that includes both as subprocessors. It’s also a good idea to read about How Adversus uses AI to improve our features in general before continuing reading this article.
Signing the DPA is the first step to staying compliant. However, it is equally important that you comply with laws and regulations, such as those regarding data processing, data storage, and privacy policies. Adversus is constantly working to deliver compliant and safe solutions; however, we cannot accept any responsibility whatsoever for how our customers handle their data internally. Read more about our Trust and Compliance policies here.
How does the AI-based Quality Assistance feature work
In short terms, the QA works in the following steps:
Set up one or more AI transcription settings for your account. Here, you choose which campaigns to transcribe, the transcription language, keywords, and more.
Create a QA model that defines which transcriptions should be reviewed and which requirements they must be evaluated against. You can also choose the percentage of matching transcriptions that should be added to QA review for manual evaluation.
Once one or more QA models have been created and enabled, conversations that meet the defined criteria will automatically be processed by the AI and appear in the QA Review module.
Finally, review the selected conversations in QA Review and decide whether they have passed or failed your requirements.
Next, we will outline the various steps in the process, from setting up transcriptions to final review.
How to set up AI transcriptions of conversations and apply them to one or more campaigns
The AI transcription feature operates at the account level, allowing it to be used globally across all campaigns.
Go to Settings and Account
Choose AI transcriptions from the top menu
Click Create in the top right corner and Create transcription from the dropdown menu. This window will appear:
a) Transcription name: Give your transcription setting a recognizable name.
b) Enable transcription: Choose whether all conversations (longer than the minimum duration) should be transcribed Automatic or you want to do it Manually (through Warehouse or QAI – we’ll get back to how that’s done). We recommend choosing Automatic if you want to utilize the full power of AI-driven Quality Assurance. Manual transcription is advisable if you need to pause or gain more control over the transcriptions (e.g., due to compliance issues).
c) Duration requirement for transcription defines how long the conversation, at a minimum, should last before a transcription is created. By adding this, you avoid transcribing calls in which the prospect, e.g., ends the conversation immediately, or your agent hangs up because an answering machine is detected.
d) Summary Prompt lets you specify how you want your summaries, e.g., is there any specific information you want the summary to include, do you want it to be at a particular length, etc. If you don’t add a prompt, you’ll use Adversus’ default prompt, which is designed to brush up highlights in bullet points that we think are useful, such as the outcome of the conversation, or the reason why if ‘Not interested’.
e) Enable transcription for campaigns is mandatory. This is where you choose which campaigns you want transcriptions applied to. Please note that calls made on the selected campaigns before the transcription settings are enabled are not automatically transcribed.
f) Transcription language lets you define in which language you want your transcription. For example, if the conversation is in Swedish and you select English as your transcription language, the conversation will be automatically summarized in English.
g) Keywords are beneficial to add if you want the AI engine to be aware of certain words, such as your company name, products, etc.
h) Tags make it easier to locate and recognize this setting across Adversus, e.g., in the Warehouse or Leads modules.
4) Click Create transcription in the bottom right corner when you are done choosing settings. Now your transcription is running on the selected campaigns, unless you have chosen Manual transcription – in that case, you need to start the transcriptions manually through Warehouse.
Manual transcription - nice to know
Manual transcription can essentially be considered a way of "pausing" automatic transcription. Therefore, we strongly recommend Automatic transcription if your goal is to maximize efficiency.
Briefly explained, you can transcribe a call Manually in Warehouse → Calls by filtering and then choose Transcribe recording under Actions. Please note that only conversations in campaigns included in an AI transcription setup are available for transcription in Warehouse.
Manual transcription through Warehouse is especially relevant for calls made before a campaign was added to an AI transcription under the account settings.
You can also manually transcribe through the Quality Assurance module. We'll get back to that.
If you choose Manual transcription, recordings can be transcribed from Warehouse → Calls by selecting Transcribe recording under Actions.
You can also start manual transcriptions directly from the Quality Assistance module, which we will cover later.
Even when Automatic transcription is enabled, manual transcription can still be useful for conversations that occurred before a campaign was added to an AI transcription setup, since transcriptions are created only from the moment the transcription configuration is activated and are not generated retroactively.
How to use the Quality Assistance module
Once a conversation is transcribed either automatically or manually, you can review it in the Quality Assistance module. The first step is to create a QA model that defines which conversations you want to review and the requirements those conversations must meet.
Go to Leads → Quality Assistance. The first time you enter the module, you’ll be presented with an empty list, because you need to create one or more QA Models before anything appears on it.
Go to QA model in the top menu and click Create QA model in the upper left corner. Now it’s time to set up your model.
We’ll take it step by step:
a) QA model name must be a name that makes the model easily recognizable when you are using it for QA review.
b) Add a Description that clarifies what this model is used for. This can, for example, be something like: “check if agents are introducing themselves and following instructions when presenting the product” or “check if the conversation lives up to compliance”.
c) Trigger Conditions define different criteria for adding a transcription to QA review:
Campaign filter is where you choose which AI transcription setting you want to use (the one you previously created under your account settings) - and thus which campaigns you want your QA model to run on.
Lead status filter is where you apply the lead statuses you want to review, e.g. applicable if you want to use the model to dig into all “not interested”.
Trigger rate sets the percentage of transcriptions that meet your criteria.
Minimum length (seconds) defines the minimum conversation length for a session to be included in a QA review.
Evaluate automatically with Arthur lets you decide whether conversations should automatically pass and be excluded from QA Review when the AI determines that all configured requirements have been fulfilled.
d) Finally, you must create the Requirements you want to evaluate your transcriptions against. Click New requirement to start building your prompts and define their settings.
How requirements work and how to create them
Requirements define exactly what the AI should look for when reviewing conversation transcriptions.
Best practice is to make your prompt as straightforward as possible. Keep each prompt simple, specific, and focused on a single action. This makes it easier for the AI to produce accurate and consistent evaluations.
We recommend using our helpful bot, Arthur. Arthur can guide you and suggest requirements that are easy for the AI engine to handle.
To help you create effective prompts, Arthur, our AI assistant, can suggest improvements and help formulate your requirements.
You can reuse requirements across your QA models by adding Existing requirements to your model (if you've already created some). You can also edit your requirements. Please note that editing a requirement used across multiple models may affect your review processes.
Requirement settings
Name your requirement so it’s easy to recognize. We suggest you give it a name that reflects its purpose.
Description is where you specify the actual prompt/instruction for the AI. Phrase it as specifically as possible and ensure the prompt only describes one single action
Choose if the prompt is something the agent Must do or Must not do during the conversation. Using this function makes it easier to distinguish between dos and don'ts.
Evaluation scope defines in which part of the conversation the requirement must be fulfilled. Since the AI is screening all conversations (if there have been more than one) on the lead, there are different scope options.
Choose Any conversation if the requirement must be fulfilled at any point of any conversation recorded on the lead.
Choose Closing conversation if the requirement must be fulfilled during the last conversation (e.g. useful if you only want to check successes).
Choose All conversation if the requirement must be fulfilled in all conversations, e.g., checking whether agents are presenting themselves every time they call the lead.
Finally, decide how the requirement should be evaluated. Choose Require human evaluation if human review should be mandatory. A human review is not mandatory if you Use AI to generate outcome – but you can always overrule the AI-generated outcome during the evaluation. You can add as many requirements to your QA model as you want.
After configuring your requirement, click Create requirement.
You can create as many requirements as you need. However, if your list of requirements becomes extensive, we recommend splitting them across multiple QA models. This typically results in more accurate evaluations and makes it easier to reuse requirements across different campaigns.
Meet Arthur – your AI assistant for Quality Assistance
Arthur is your AI assistant for creating and improving QA requirements. Instead of writing prompts from scratch, you can collaborate with Arthur to create requirements that are clear, specific, and easy for the AI engine to evaluate consistently.
Arthur helps you by asking clarifying questions when additional context is needed and by suggesting improved versions of your prompts based on your answers. This makes it easier to create high-quality requirements without extensive experience in prompt writing.
Whether you're building a new QA model or refining an existing one, Arthur can help you create prompts that lead to more accurate and consistent Quality Assistance results.
Hozw to optimize requirements with Arthur
Click Optimize with Arthur to improve your requirement. Arthur may respond with either:
Questions, asking for additional information to better understand your intention.
Suggestions, providing an improved version of your requirement.
Once Arthur has gathered enough information, it will propose an optimized requirement description. If you're satisfied with the suggestion, simply apply it. Otherwise, continue the conversation with Arthur until you achieve the desired result.
How to test requirements
You can preview your requirements on the QA-model setup page by clicking the Preview button on the right, which lets you test the quality of your requirements immediately. A modal will open, allowing you to select a specific call/transcription to evaluate and choose which requirements you want to test.
With the preview functionality, you can easily test and quickly verify how well your requirements/AI prompts perform in different conversations and contexts before committing fully to them.
1) Enter the Call ID you want to use for your test
2) Select the Requirements you want to test. It's possible to add requirements that are not already included in your model.
How to review the transcriptions
Once you’ve set up your QA model(s), you are ready to start reviewing.
Go to QA review.
On the right, choose which QA model you want to review conversations from. Additionally, you can filter this list by using Lead Filters, allowing you to pick a specific campaign or agent you want to review.
Now you’re ready to review. There are different ways of reviewing:
Click any lead on the list with the status Pending to review a single conversation.
Click Start QA Session to review all leads available in the list shown in your QA flow.
A pop-up window with various information will appear:
Lead data (in the top menu): Shows all lead data.
Requirements (in the top menu): Link to all recording(s) of all conversations, the transcriptions, summaries, etc., and a list of the requirements you are checking for.
To the right under Requirements, you can wing off whether the conversation(s) have passed or failed each requirement separately. It’s not mandatory to check each requirement if your QA model does not require Human review. Leave a note to the agent if you want.
When done, decide whether the conversation(s) has Failed or Passed. Once it has been evaluated, it will disappear from the QA review overview.
Warehouse for Quality Assistance
When you have conducted a review and marked the conversation(s) as either Passed or Failed, you can find them in Warehouse → Quality Assistance. Like the other Warehouse modules, it serves as the central repository for all QA reviews and conversations. Here you can filter results, create custom views, and configure actions to help manage your Quality Assistance workflow.