AutomationMart
Home/Browse/Generate AI-ready llms.txt files from Screaming Frog website crawls
n8n

Generate AI-ready llms.txt files from Screaming Frog website crawls

n8nn8n12 modulesv1.0
Google Drive

This workflow helps you generate an llms.txt file (if you're unfamiliar with it, check out this article) using a Screaming Frog export. Screaming Frog is a well-known website crawler. You can easily crawl a website. Then, export the "internalhtml" section in CSV format. How It Works: A form allows you to enter: - The name of the website - A short description - The internalhtml.csv file from your Screaming Frog export Once the form is submitted, the workflow is triggered automatically, an

At a glance

Generate AI-ready llms.txt files from Screaming Frog website crawls is a ready-made n8n workflow you import as a workflow JSON file — no build required. It connects Google Drive. It's free to download. Follow the 5-step import below to go live in minutes.

Platform
n8n
Connects
Google Drive
Modules
12
Price
Free
Version
v1.0
Generate AI-ready llms.txt files from Screaming Frog website crawls workflow diagram

About this workflow

This workflow helps you generate an llms.txt file (if you're unfamiliar with it, check out this article) using a Screaming Frog export. Screaming Frog is a well-known website crawler. You can easily crawl a website. Then, export the "internalhtml" section in CSV format. How It Works: A form allows you to enter: - The name of the website - A short description - The internalhtml.csv file from your Screaming Frog export Once the form is submitted, the workflow is triggered automatically, and you can download the llms.txt file directly from n8n. Downloading the File Since the last node in this workflow is "Convert to File", you will need to download the file directly from the n8n UI. However, you can easily add a node (e.g., Google Drive, OneDrive) to automatically upload the file wherever you want. AI-Powered Filtering (Optional): This workflow includes a text classifier node, which is deactivated by default. - You can activate it to apply a more intelligent filter to select URLs for the llms.txt file. - Consider modifying the description in the classifier node to specify the type of URLs you want to include. How to Use This Workflow 1. Crawl the website you want to generate an llms.txt file for using Screaming Frog. 2. Export the "internalhtml" section in CSV format. 3. In n8n, click "Test Workflow", fill in the form, and upload the internalhtml.csv file. 4. Once the workflow is complete, go to the "Export to File" node and download the output. That's it! You now have your llms.txt file! Recommended Usage: Use this workflow directly in the n8n UI by clicking 'Test Workflow' and uploading the file in the form.

n8n

How to import this n8n workflow

  1. 1

    Download the workflow JSON file after purchase.

  2. 2

    Open n8n → click the menu → Import from File.

  3. 3

    Select the downloaded JSON and import.

  4. 4

    Set up credentials for each node that requires them.

  5. 5

    Click Execute Workflow to test, then activate.

Setup guide

Setup guide included

Purchase to unlock the full step-by-step guide

Related N8n workflows

Send Slybroadcast campaigns with audio URL created by Google Cloud TTS for new WooCommerce orders

Make will watch your WooCommerce orders at regular intervals and synthesize speech for order detail values designated by you via Google Cloud Text-to-Speech, upload them to your Google Drive, create a Google Drive share link and send the audio file as a campaign to a list of mobile phones via Slybroadcast.

Free

Summarize meeting transcripts with GPT-4 & sentiment analysis for Gmail

Who it's for Project managers, AI builders, and teams who want structured, automated meeting summaries with zero manual work. What it does This workflow monitors a Google Drive folder for new meeting notes (PDF/TXT), extracts text, summarizes it via OpenAI GPT-4o, groups tasks by sentiment, builds a styled HTML summary, and sends it via Gmail. How to set it up 1. Connect Google Drive, OpenAI, and Gmail credentials. 2. Point the Drive Trigger to your meeting notes folder. 3. Paste

Free

Generate viral Bigfoot vlog videos with Veo 3

Transform simple ideas into viral-ready Bigfoot vlogs! This automated workflow creates charming 8-scene video content featuring "Sam" the Bigfoot - a lovable, outdoorsy character inspired by popular YouTube adventure channels. How It Works The workflow transforms your creative concept into professional video content through three automated stages: 1. Story Generation - AI creates an 8-scene narrative arc featuring Sam the Bigfoot, complete with character-consistent dialogue and engaging pl

Free

Save all photos from Unsplash collections to Google Drive

Upload all photos from the user's collections in Unsplash to the specified folder in Google Drive.

Free

Create Word and PDF documents with Plumsail from Typeform data

Automate the generation of documents in your company: create DOCX and PDF files from a template using Plumsail Documents. The scenario is started when a form is submitted in Typeform. Files will be saved in OneDrive. This works equally well with SharePoint, Dropbox, Google Drive, and other storage platforms. You can use it to create applications, orders, invoices, etc.

Free

Convert Venmo statement CSV file to QuickBooks CSV import

Say goodbye to manual Venmo-to-QuickBooks formatting! This automated workflow transforms raw Venmo CSV exports into QuickBooks-ready files and backs them up seamlessly to Google Drive or Dropbox. ⚙️ What This Workflow Does: 📤 1. Accepts a Venmo CSV Upload — Triggers on form submission to upload a .csv Venmo transaction file. 📂 2. Extracts Raw CSV Data — Reads the uploaded file and prepares it for processing. 🛠️ 3. Generates a Smart File Name — Dynamically names the output file using the transact

Free

Scrape business emails from Google Maps without the use of any third party APIs

Who is this template for? This workflow template is designed for sales, marketing, and business development professionals who want a cost-effective and efficient way to generate leads. By leveraging n8n core nodes, it scrapes business emails from Google Maps without relying on third-party APIs or paid services, ensuring there are no additional costs involved. Ideal for small business owners, freelancers, and agencies, this template automates the process of collecting contact information for

Free

Import CSV files from filesystem into Postgres

Disclaimer: This template is mainly made for self-hosted users who can reach CSV files in their file system. For Cloud users, just replace the first few nodes with your file system of choice, like Google Drive or Dropbox -- How to automatically import CSV files into postgres 1、project description This workflow demonstrates how CSV file can be automatically imported into existing PostgreSQL database. Before running the workflow please make sure you have a file on the server: The name of the test

Free

Reviews

No reviews yet

Be the first to buy and share your experience.

Leave a review

Sign in to share your experience with this workflow.

Log in to review
Free
No ratings yet

Create a free account to purchase workflows.

  • JSON blueprint — instant download
  • Setup guide PDF included
  • 5 downloads · valid 30 days
  • Works with n8n

Need help setting this up?

Book a 3-hour live setup session with an Agility consultant.

₹2,499/ session
3 hrs · video call
  • Configure live on Google Meet / Zoom
  • Free follow-up if workflow has defects
  • Platform expert assigned to you
Book installation session
Free