How to train the accuracy of your Signal AI

[stevestormoen]

Signal AI is a powerful tool for gathering and acting on your call intelligence in real-time, based on machine learning. In order for your Signal AI to learn to classify your calls correctly, however, it has to be taught.

Whether you arecreating a new AI Signal or refining an existing Signal, follow these steps to train your Invoca account to provide the most accurate Signal AI insights possible:
 

Choosing a sample of calls

Your Signal AI insights are based on patterns within your phone calls. That means, in order to train your Signal AI, you’ll need recordings of those calls. What should you include in your sample of recordings? Here are two pointers:
 

• Represent a realistic balance of Signal outcomes in your sample calls. Your Signal AI training model will receive the best results if it can predict outcomes from a representative sample of the phone calls you’ll receive in real life. For example, if you’re trying to train a Signal AI rule that recognizes sales, and sales typically occur in 15% of your phone calls, you’ll want to train your Signal AI using a sample that includes 15% sales as well.
• Provide a larger sample for better results. The more calls Signal AI has access to, the better it can get at recognizing patterns. To get started, we recommend at least 150 examples of the least common outcome of your insight to establish version 1 of each Signal. So, in the example above, if 15% of your phone calls result in sales, you'll need 150 calls with sales, out of a sample of 1000 calls. From there, your Signal AI will automatically retrain itself and update to a new version of your AI algorithm as you provide more phone call recordings. Each version will require 50% more data than the previous version — the higher the version, the more precise your AI will be. See What does the accuracy range of my Signal AI mean? for more details.

Notes:

- This article assumes that you are training your Signal AI using call recordings in your Invoca account. If your call recordings are stored in an external system, such as your call center, see How to import recordings of your phone calls to create and train Signal AI.

- Invoca will omit words from Invoca AND Customer's first IVR, but if the call transfers to another handler, after the first customer IVR, and hits a second customer IVR Invoca will not omit words from the second.

Providing ground truth data

The next step is to report the results of your Signal insights for each call in your sample — called ground truth data. With that ground truth data, Signal AI can identify patterns in your successful and unsuccessful calls, then apply those patterns to future calls. If you haven’t already indicated which results Signal AI should look for, follow the instructions in How to capture advanced insights from your phone calls using Signal AI.

In your Invoca account, you have three options for providing ground truth data to your Signal AI: 

• Listening to your phone call recordings and manually scoring your Signal outcomes
• Uploading a spreadsheet with outcomes for all your Signals for each phone call
• Paying for the human call scoring service offered by Invoca

If you’re listening to and scoring phone calls manually

Follow our guide, How to change your Signal results manually, to change your Signal results in the calls you've identified. When you manually correct Signal results in your calls, your Invoca account will gather that information and update your Signal AI models accordingly.
 

If you’re uploading a spreadsheet

Choose this method if you keep records of your Signal outcomes in a third-party system, such as a CRM.
 

1. Log in to your Invoca account. In the sidebar menu, click Signal, then select Manage Signals.
2. Select Signal AI Training Data, then click the Download Template button.
3. Select I have IDs for Invoca calls or transactions to download a spreadsheet file (.xlsx format) formatted to help you upload your call records back to your Invoca account accurately.
4. Open the Signal file upload template spreadsheet. Note the first three columns titled “transaction_id”, “call_record_id”, and “call_start_time”. These three columns are used as unique identifiers to match the call record from your third-party system with the associated call record in your Invoca account. You only need to fill in one of these columns for each of your call records.
    
   • transaction_id: The ID of the transaction (call leg) being reported on.
   • call_record_id: The ID of the complete call being reported on.
   • call_start_time: The start time does not have to be exact since clocks will not be perfectly synchronized. We will pair this record with the closest match in your Invoca account. “See the Additional tips for reporting properly formatted data” section below for descriptions of supported timestamps.
      
     Providing precise call records:
     Of the three unique identifier columns, “call_start_time” is the least precise (most “fuzzy”) of these options. If you fill more than one unique identifier column, Invoca will always match records based on “transaction_id”, then “call_record_id”, then “call_start_time”. If you only fill in “call_start_time”, we strongly recommend you also fill in either the column “calling_phone_number” or “duration_in_seconds” to provide more precise records.
     
      
5. Enter your call record data into the template spreadsheet, following these instructions:
    
   • Each line of your spreadsheet will update one call record with values for each Signal in your Invoca account.
   • Your template spreadsheet contains a column named for each Signal in your Invoca account. In each of those columns, enter the result of that Signal event as “true”/”false”, “yes”/”no”, or “1”/”0”. 
   • If you don’t need to include a particular Signal in any of your call records, it is okay to delete that column in your spreadsheet, or simply leave it blank.
      
6. When you are finished entering your call records into the template spreadsheet, save the spreadsheet file as a Microsoft Excel file or .csv file.
7. Return to your Invoca browser window. In the Upload New File menu, click the cloud icon. Find your newly saved spreadsheet file including your call records and select it.
8. Select your time zone from the drop-down menu. When you are finished, click Upload and Preview.
9. Next, Invoca checks your spreadsheet for errors. If your spreadsheet does contain errors, see Additional tips to format data in your Signal file upload spreadsheets. Otherwise, click Proceed to apply your newly uploaded Signal records into your Invoca account.

 
That’s it! As soon as you click Proceed, Invoca will display a status bar indicating the progress of your upload. Once the entire file is processed, you can view the results of the upload in the upload history table.
 

If you’re paying for human call scoring 

Invoca offers a human call scoring service as an affordable option for customers that don't have the time or resources to listen to a set of call recordings to tag them with results for AI Signal training, or retraining. With this service, you'll be assigned a dedicated team of trained call analysis agents that will listen to your calls and train your Signal AI. However, if you have the ability, we recommend you listen to your calls yourself — nobody knows your business better than you do!

Here’s how to request our human call scoring service:
 

1. Log in to your Invoca account. In the sidebar menu, click Signal, then select Manage Signals.
2. In your list of Signals, select the Signal you’d like to train. Note that this Signal must have the type “AI (Custom).”
3. Click Provide Training Data, then under I need the data, select Request Service.
4. Click Request Human Call Scoring to be put in touch with a member of our dedicated Customer Support team, who will contact you shortly with more information.

 
Congratulations, you’ve successfully started to train your Signal AI to recognize patterns in your phone calls! You can check on the progress of your Signal AI training by selecting Signals in the gray ribbon menu, then selecting the Signal you wish to train in the Signals menu. In the Version section, you’ll see an indicator of the progress of your training data. More progress will help improve the accuracy of your Signal AI, so keep scoring those calls!
 

Related:
• How to capture advanced insights from your phone calls using Signal AI 
• How to change your Signal results manually 
• How to import recordings of your phone calls to create and train Signal AI