Great news for teams optimizing their AutoQA setup! 🎉 The Misalignment Analysis feature has been upgraded to understand your criterion tags and proactively suggest metadata configuration changes that could improve alignment between AI and manual QA ratings.
📈 What's New
Misalignment Analysis already helps you understand why AutoQA ratings differ from manual QA ratings. With this update, it now also:
• Understands criterion tags — The analysis is aware of the metadata tags configured on your AutoQA criteria.
• Suggests changes — When it identifies that a tag adjustment could resolve recurring misalignments, it presents a specific recommendation.
• Explains the reasoning — Each suggestion includes an explanation of why the change is expected to improve alignment.
🛠️ How It Works
1. Navigate to Settings → Quality Assurance → select a QA Scorecard
2. Click on a criterion → use the Test button to run a test
3. After the test runs, review the Misalignment Analysis results below the test tickets
4. Look for metadata change suggestions in the analysis output — they'll appear when applicable
💡 Pro Tip: Run Misalignment Analysis after updating your AI Instructions to see if the metadata tags also need adjustment for optimal alignment.
✨ Benefits
• Faster optimization: Get proactive suggestions instead of manually experimenting with tag configurations.
• Better alignment: Reduce the gap between AI scores and manual QA ratings with targeted metadata changes.
• Deeper insights: Understand how your metadata configuration affects AutoQA accuracy.
🚨 Important: This is the first iteration of metadata suggestions. We recommend reviewing the suggestions carefully before applying them. If you notice any inaccurate or unexpected suggestions, please report them to your CSM so we can continue improving the quality.
Need Help? 🤝
• Email: support@kaizo.com
• In-app chat: Click the help icon in Kaizo
• Your CSM: Schedule time with your dedicated Customer Success Manager
Comments
0 comments
Please sign in to leave a comment.