This n8n template demonstrates how to calculate the evaluation metric "Correctness" which in this scenario, measures the compares and classifies the agent's response against a set of ground truths. The scoring approach is adapted from the open-source evaluations project RAGAS and you can see the source here How it works This evaluation works best where the agent's response is allowed to be more verbose and conversational. For our scoring, we classify the agent's response into 3 buckets: True
Evaluate AI agent response correctness with OpenAI and RAGAS methodology is a ready-made n8n workflow you import as a workflow JSON file — no build required. It connects OpenAI. It's free to download. Follow the 5-step import below to go live in minutes.

This n8n template demonstrates how to calculate the evaluation metric "Correctness" which in this scenario, measures the compares and classifies the agent's response against a set of ground truths. The scoring approach is adapted from the open-source evaluations project RAGAS and you can see the source here How it works This evaluation works best where the agent's response is allowed to be more verbose and conversational. For our scoring, we classify the agent's response into 3 buckets: True Positive (in answer and ground truth), False Positive (in answer but not ground truth) and False Negative (not in answer but in ground truth). We also calculate an average similarity score on the agent's response against all ground truths. The classification and the similarity score is then averaged to give the final score. A high score indicates the agent is accurate whereas a low score could indicate the agent has incorrect training data or is not providing a comprehensive enough answer. Requirements n8n version 1.94+ Check out this Google Sheet for a sample data
Download the workflow JSON file after purchase.
Open n8n → click the menu → Import from File.
Select the downloaded JSON and import.
Set up credentials for each node that requires them.
Click Execute Workflow to test, then activate.
Setup guide included
Purchase to unlock the full step-by-step guide
Discover business leads with Gemini, Brave Search and web scraping
Analyze Landing Page with OpenAI and Get Optimization Tips
Prevent duplicate webhook executions with AARI idempotency gate
Generate trend-based marketing videos with Seedance AI, Perplexity, and GPT
Automatically update n8n version
Send structured logs to BetterStack from any workflow using HTTP request
No reviews yet
Be the first to buy and share your experience.
Leave a review
Sign in to share your experience with this workflow.
Create a free account to purchase workflows.
Need help setting this up?
Book a 3-hour live setup session with an Agility consultant.