Invoca Working Group: Signals

[maria_ogneva]

Summary of key best practices

In this meeting, a number of Invoca customers were invited shared their best practices. It was moderated by Austin Brender, Senior Analytics Services Manager from Invoca. Most people on the call were using phrase spotting and some AI and API Signals. 

Defining the difference between different types of Signals

Phrase spotting: did someone say a specific set of words and phrases: yes or no? AI Signal digs into the entire call to understand what resulted in a conversion. 

Using Signals to segment

Austin started the conversation by offering how his team thinks about Signals in segmenting calls, such as sales opportunities or service calls – and using these segments to build a call funnel. He opened the floor to the group to share how they use Signals.

Use cases for Signals

Lauren Bova from Cleveland Clinic shared that they use Signals to identify when appointments are made during calls. This allows them to filter to only calls that resulted in an appointment. They are then able to analyze these calls and also make improvements in the call center or advertising to achieve their appointment setting goals.

Refining Signals

Laura Chase from Providence shared that they are in the refining phase in their industry standard phrase Spotting Signal and mentioned facing issues with false positives and asked for advice on refining their Signals. Lauren Bova, who uses AI Signals, responded that they spent a lot of time training the Signal and using the thumbs up / down to help the system learn what success is. As a result, their Signal is now 97% accurate, achieved through months of training, and only fires when the AI is 97% confident about an appointment.

For a phrase spotting Signal, Austin suggested customizing phrases, adding negative phrases (i.e. what was not said on the call) and working with the Customer Success Manager to improve accuracy.

Jigna from Aspen Dental Group shared their Signal implementation, which is very mature. She recommends training Signals, analyzing call transcripts, and using negative keywords to improve accuracy. She mentioned that through years of training and working with Invoca, their AI Signal for call appointments is now 97% accurate.

Lynn Duffy from Butler/Till, a performance and marketing agency, explained that she uses AI technology for both marketing and operational efficiencies in different industries such as healthcare, financial services, automotive, and appliances. She emphasizes that the insights obtained from AI-generated conversations are valuable in informing other efforts.

Signal Discovery

Laura Chase asked about what folks are doing with other topics and nuggets discovered that people are calling about but that are not being looked for. Lynn and another participant suggest using word clouds to see which words or phrases pop out from transcripts, indicating popular themes, which would allow them to dig in further to research. Jigna recommends  word clouds as well; they have allowed her team to uncover key themes. 

Signal Discovery allows to create a word cloud and dive into individual bubbles to understand different things being discussed in a set of calls. Signal Discovery is an AI tool that listens to the entirety of calls rather than just looking for specific words. Jigna shared the importance of excluding anomalies or noise from the data when analyzing call transcripts and that they had analyzed 30k calls in their Signal Discovery run. Lauren Bova shared that through Signal Discovery they discovered that a lot of people were calling the phone number in their ads just to ask for directions, which is not a great use of marketing spend. This led them to better clarify directions.  Jigna shared that a lot of their calls were around billing, and this is a good example of how to create negative phrases. 

As far as advice for a good threshold of calls, Austin recommends a few thousand calls for phrase spotting and a few hundred calls for AI Signals. For AI Signals, it’s important to continue to refine and keep it going over time. 

Enabling internal conversations 

The conversation then turned to internal conversations with other departments that have been opened up and facilitated by what was uncovered as a result of Signals. Jigna responds that there have been two specific use cases to match that. The first use case involves one of their brands (they are  a multi-brand organization), where they have noticed a high call volume but an extremely low average call duration. They have noticed the workflow is too long, making the user hang up. They are considering using Invoca’s intelligent call routing or priority routing to make the call duration longer and aid appointment conversion. The second use case involves the VP of their call center operations being interested in using Invoca’s agent scoring functionality, which is a result of the Signals and insights they have been providing the call center. Austin comments that the contact center use case takes the analysis to the next level, where they can determine which agent or location is doing a good job following scripts and working towards the organization's goals. 

Participants discuss the process of identifying unknown unknowns and improving phrase Spotting accuracy in calls. Austin Brender suggests listening to calls where none of the Signals fired, looking for patterns, and using Global Transcript search functionality to search for specific phrases in all calls. 

Dena Read with Senior Resource Group notes the importance of visibility into qualified versus non-qualified calls and sharing that information with their call center to improve the process and customer experience.  To that end, Austin shares that another customer discovered that they were seeing a lot of mentions of McDonald’s on their calls, as directions weren’t clear on their website. As a result, they were able to update their website,. Improving the customer experience and reducing the unqualified calls.

Other tips and tricks to improve accuracy of phrase spotting

Amanda Sloan with Acadia Healthcare shares her experience with different dialects and accents and the importance of including different derivatives for words like "admissions" and iterations of insurance carrier names to improve the accuracy of phrase spotting Signals. They made sure to listen to calls and include all the derivatives of “admissions.” Austin recommends using transcriptions to find these examples.

Lynn Duffy discusses the value of understanding the voice of the customer and adjusting their copy to how consumers are speaking. They served this information up to the right teams.

Michele Repine from UCSF Health explains how they engage with clinical teams and call centers to identify and address operational efficiencies through the call review process. She is able to add comments and categorize those comments by type of insight and run a report that she is then able to share across the organization, which allows them to find operational efficiencies.

Austin responds that Signals, while they won’t help solve all problems, they help identify what needs an extra look, which is more actionable with a smaller subset.