Ticket funnel setup

🚨 Before beginning to use the Quality Assurance tool, it's crucial to set it up in a way that suits our requirements for every existing QA Scorecard.

🤫 If you prefer every QA Scorecard to have a sample of random tickets, there is no need to configure any filters.

There are several reasons why reviewing the Settings is important before defining the QA workflow:

1. Ensuring that an adequate number of QA evaluations are performed for each agent within the week.
2. Ensuring that only conversations that are relevant or significant to you are reviewed.
3. Ensuring that quality standards are effectively communicated to the rest of your team.
4. Keeping management informed about the targets to be achieved and working towards reaching them.

Let's start our setup by visiting Settings -> Quality Assurance -> Tickets funnel

The Tickets Funnel Settings will aid in specifying the type of tickets that require the attention of QA experts, and the number of QA evaluations expected per agent per week for every QA Scorecard separately.

1. Select the name of the QA Scorecard to be filtered

2. Who ➡ Select agents to QA-rate

Before diving into the filters, we first need to decide which agents involved in handling the ticket should receive QA ratings. Autopilot processes only tickets marked as Solved (or Closed). 🤝

• Agent who Solved ticket: Only the agent who resolved and closed the ticket will receive a QA rating, regardless of how many agents contributed to the conversation. 
• All customer-facing agents who contributed to Solved ticket: Every agent who sent a public reply on the ticket will receive a QA rating. This is great when multiple agents handle the same case and you want to evaluate each person's contribution. 
• All agents with touchpoints: This includes agents who left any comment — public or internal — on the ticket. (Note: QA Autopilot is not available with this option.) 
• Last assigned agent before CSAT rating: The agent who was assigned to the ticket at the time the customer left their satisfaction rating will receive the QA evaluation. This is useful when you want to evaluate whoever was responsible at the moment the customer gave feedback.

💡Pro-tip: If your team often collaborates on tickets and you want to evaluate everyone's contribution, select All customer-facing agents who contributed to Solved ticket. This ensures no one's work goes unreviewed!

💡Pro-tip x2: If you're using QA Autopilot, keep in mind that the All agents with touchpoints option is not supported. Choose one of the other three options to make sure Autopilot can rate tickets automatically.

3. What ➡ Filter tickets selection 

Now it's time to specify the type of tickets that require QA rating, ensuring that your QA experts are not spending their time evaluating unimportant cases. 👆🏽

To define your preferences, use the three filters: tag filter, channel filter, and metric filter.

• Tag filter: Add tags to tickets inside Zendesk. If you select more than one tag, the filter will suggest tickets for QA rating that contain at least one of the defined tags.

💡Pro-tip: If no tags are defined, we include tickets with all present tags.

💡Pro-tip x2: Define tags in Zendesk to group similar cases together - #refund_issues, #order_cancelation, #legal, etc. This helps evaluate conversations supported by different agents that are related to the same case, determining if the entire team struggles with a certain type of case, or if only Agent A requires additional coaching in this regard. You can also add Exclude tags to skip tickets that shouldn't be rated — for example, #spam or #test.

❗️ QA Insights Dashboard helps understand the results on the team and agent level. Check it out here!

• Channel filter: Define channel groups through which agents provided support - Email, Chat, and Phone. If multiple channel groups are selected, we include all tickets supported in each of these channels.

💡Pro-tip: If no channels are defined, we include tickets supported through any channel.

💡Pro-tip x2: You can always adjust the settings to focus on a specific channel support for the week. For instance, if you introduced Live Chat support and need to ensure the same high-quality level as with Email support, set a filter to focus only on chat tickets this week or month!

❗️ Did you know that you can quickly switch between Chat, Email, and Phone QA results in the QA Insights dashboard? This helps quickly understand which channel needs more attention during coaching. Learn more about the QA Insights dashboard here!

• Metrics filter: Focus on tickets that meet your preferred criteria (e.g., have at least 2 messages from the agent before solved) or outside your desired outcome (e.g., the first reply time is longer than 1h). If more than one metric is selected, we include tickets that meet at least one of the metric criteria (e.g., either had more than 2 Public Replies or Chat Messages before solved or had longer than 1h First Reply Time).

💡Pro-tip: If no metrics are selected, we do not specify if the ticket had a specific metric result.

💡Pro-tip x2: If only the CSAT metric is selected, and both positive and negative ratings are chosen, we include only the tickets rated by the customer.

💡Pro-tip x3: You can leave the metric limits half-open. For instance, if you want at least 4 Public replies or Chat Messages sent before the ticket is resolved, you can select Messages to Solve as (4;-) (as shown in the example)👇🏽.

💡 New metrics available for filtering: Call Duration, Public Replies, Chat Messages, Ticket Reopened, Customer Sentiment, and Agent Empathy Score. The Empathy Score filter includes a percentage slider — set a threshold to focus on tickets where the agent's empathy fell below a certain level.

How does it work?

After defining your preferred criteria, your QA experts will be presented with tickets for QA rating that have at least one of the defined tags, were supported through one of the chosen channels, and had a result for at least one of the selected Metrics and values.

Using the example above👆🏽, one of the tickets suggested for QA rating could be related to "Shipping," supported through Chat, and negatively rated by the customer.

4. Set the recommended weekly workload 

The next step in your QA setup is to decide on the number of tickets that you want your QA team to evaluate each week. You have two options:

1. Set a fixed number of tickets to rate.
2. Set a percentage of all solved/handled tickets. For example, if your agents solved 100 tickets last week, and you set a minimum of 10% of solved tickets to rate, a total of 10 random tickets will be rated on the Scorecard.

If you've enabled QA Autopilot on your scorecard, you'll also see an Autopilot section here. Use the Target coverage slider to define what percentage of eligible tickets Autopilot should rate automatically. Samurai rates tickets based on probability — the more solved tickets available, the closer it gets to your target percentage. Maximum coverage can be set to 100%. If you need to change it, contact us through the chat button or via support@kaizo.com.

💡 Pro-Tip: If you choose to rate 5 tickets per agent or 10% of Handled tickets, and an Agent handled 100 tickets during the week, 10 tickets will be suggested for QA evaluation for that week. We choose the highest number of 2, so if you want the same number of QA ratings each week for everyone, we suggest setting the percentage to 0.

💡 Pro-Tip x2: Rate enough tickets to see an agent's actual performance. For example, if an agent solved 100 tickets during the previous week, but you only rated one, you wouldn't get a good understanding of the agent's performance (that ticket could be either very good or very bad, which means you won't be objective in your rating). We recommend rating 5%-10% of all tickets that your agents have handled. This ensures objective feedback and a realistic IQS (Internal Quality Score).

💡 Pro-Tip x3: Start with a lower Autopilot coverage (around 15-20%) and increase it gradually as you verify the AI ratings match your team's standards.

 

"How can I ensure that my filters will work for everyone?"

While it's great to define narrow filters, it can be difficult to predict if every agent will support the same types of tickets within the week or have enough of these tickets to achieve the weekly QA rating goal. So how can we ensure that the weekly rating process goes smoothly?

 

Review how many tickets each agent has supported weekly on average for the past 30 days that match your filter. Use the Preview panel on the right side of the screen to get an estimate of how many tickets are available for QA rating per agent weekly. This helps you verify your filters before saving!

Using the example above 👆🏽, we can expect around 100 possible tickets to be rated after selecting our filter. As our limit is set to 5, we will have plenty of options to choose from for the QA rating. 

🚨 You still can swap the ticket for another one in case you think it is not worth the QA rating. We will propose you another ticket that will match your defined filters. 

 

What's next?

Set up your company's QA standards by creating your own QA scorecard. You can find more information on how to do this at this link.

In the meantime, QA experts can learn about how the QA rating process works within Kaizo by visiting this link. Happy rating! 😎