This n8n template demonstrates how to calculate the evaluation metric "Summarization" which in this scenario, measures the LLM's accuracy and faithfulness in producing summaries which are based on an incoming Youtube transcript. The scoring approach is adapted from How it works This evaluation works best for an AI summarization workflows. For our scoring, we simple compare the generated response to the original transcript. A key factor is to look out information in the response which is not
Evaluation metric: summarization is a ready-made n8n workflow you import as a workflow JSON file — no build required. It connects UNSUPPORTED_SLUG_FORMAT. It's free to download. Follow the 5-step import below to go live in minutes.

This n8n template demonstrates how to calculate the evaluation metric "Summarization" which in this scenario, measures the LLM's accuracy and faithfulness in producing summaries which are based on an incoming Youtube transcript. The scoring approach is adapted from How it works This evaluation works best for an AI summarization workflows. For our scoring, we simple compare the generated response to the original transcript. A key factor is to look out information in the response which is not mentioned in the documents. A high score indicates LLM adherence and alignment whereas a low score could signal inadequate prompt or model hallucination. Requirements n8n version 1.94+ Check out this Google Sheet for a sample data
Download the workflow JSON file after purchase.
Open n8n → click the menu → Import from File.
Select the downloaded JSON and import.
Set up credentials for each node that requires them.
Click Execute Workflow to test, then activate.
Setup guide included
Purchase to unlock the full step-by-step guide
Discover business leads with Gemini, Brave Search and web scraping
Analyze Landing Page with OpenAI and Get Optimization Tips
Prevent duplicate webhook executions with AARI idempotency gate
Generate trend-based marketing videos with Seedance AI, Perplexity, and GPT
Build a knowledge base chatbot with Jotform, RAG Supabase, Together AI & Gemini
Gmail smart auto-responder with GPT-4o and Google Drive context memory
No reviews yet
Be the first to buy and share your experience.
Leave a review
Sign in to share your experience with this workflow.
Create a free account to purchase workflows.
Need help setting this up?
Book a 3-hour live setup session with an Agility consultant.