Content Faithfulness & Relevancy download
Reliably monitor and ensure the faithfulness / relevancy of GenSearch results to maintain trust in the AI system, allow quality monitoring, and identify content improvement opportunities.
This report answers the question, "Are your generative responses faithful to the content and relevant to the query?" in near real-time.
- Faithfulness: Measures the accuracy of the generated answer against the source context.
- Relevance: Evaluates how well the generated answer addresses the user's original question.
Timely content monitoring enables Admin users to monitor the faithfulness and relevancy of GenSearch responses for quick issue detection, to ensure evaluations are performed close to query time for up-to-date and relevant feedback.
Business goals
- Increase trust. Provide transparent evaluations of responses, and boost user satisfaction with high-quality, relevant, and accurate results.
- Boost operational efficiency. Proactively identify and correct issues with generative responses, reducing reactive support needs and providing insights for continuous improvement of your knowledge base.
- Manage costs. Balance comprehensive evaluation with cost efficiency by controlling evaluation frequency and sampling rates.
Use cases
- Authors responsible for content maintenance review faithfulness and relevancy evaluation results.
- They identify queries with less accurate or relevant responses.
- These queries point to content gaps and inaccuracies that are contributing to lower RAGAS scores and poor UX.
- Using these insights, the Authors update the documents within the knowledge base, ensuring future AI-generated responses are more accurate and useful for end-users.
- Analytics specialists prepare a report for the quarterly review meeting.
- They can run the report and compare previous time periods to track trends over time and identify areas for improvement.
- Their ability to analyze the data in detail allows the analytics specialists to provide actionable insights, driving continuous enhancement of the AI system.
- Knowledge Base administrators ensure the integrity of the generative search results and review content performance.
- They can see faithfulness and relevancy scores for the evaluated queries.
- This insight into the system's performance helps them quickly identify any anomalies or deviations in the AI responses.
- Evaluations can happen frequently; therefore, KB administrators are confident that users are receiving accurate and relevant information, which reinforces trust in the system.