Zum Inhalt springen
Business ProcessPro — €97

Process Screaming Frog Crawls for AI Text Analysis

100/100
13 Nodes

Beschreibung

Verarbeitet Crawl-Exporte, um zusammengefasste Textdateien für die KI-Analyse zu erstellen.

Setup-Anleitung

# Generate AI-Ready llms.txt Files from Screaming Frog Website Crawls  

This workflow helps you generate an **llms.txt** file (if you're unfamiliar with it, check out [this article]({{ $env.WEBHOOK_URL }} using a **Screaming Frog export**.  

[Screaming Frog]({{ $env.WEBHOOK_URL }} is a well-known website crawler.  
You can easily crawl a website. Then, export the **"internal_html"** section in CSV format.  

## How It Works: 

A **form** allows you to enter:  
- The **name of the website**  
- A **short description**  
- The **internal_html.csv** file from your Screaming Frog export  


Once the form is submitted, the **workflow is triggered automatically**, and you can **download the llms.txt file directly from n8n**. 

## Downloading the File
Since the last node in this workflow is **"Convert to File"**, you will need to **download the file directly from the n8n UI**.  
However, you can easily **add a node** (e.g., Google Drive, OneDrive) to automatically upload the file **wherever you want**.  

## AI-Powered Filtering (Optional):  
This workflow includes a **text classifier node**, which is **deactivated by default**.  
- You can **activate it** to apply a more **intelligent filter** to select URLs for the `llms.txt` file.  
- Consider modifying the **description** in the classifier node to specify the type of URLs you want to include.  

## How to Use This Workflow  

1. **Crawl the website** you want to generate an `llms.txt` file for using **Screaming Frog**.  
2. **Export the "internal_html"** section in CSV format.  
   ![Screaming Frog internal html export]({{ $env.WEBHOOK_URL }}  
3. In **n8n**, click **"Test Workflow"**, fill in the form, and **upload** the `internal_html.csv` file.  
4. Once the workflow is complete, go to the **"Export to File"** node and **download the output**.  

**That's it! You now have your llms.txt file!**  



**Recommended Usage:**  
Use this workflow **directly in the n8n UI by clicking** 'Test Workflow' and uploading the file in the form.

Benötigte Integrationen (6)

Convert To File

ConvertToFile

Extract From File

ExtractFromFile

Filter

Filter

Form

Form

Stop And Error

StopAndError

Summarize

Summarize

Workflow freischalten

Kaufe das Pro Bundle für €97 — erhalte sofortigen Zugang zu diesem und allen weiteren Starter + Pro Workflows als ZIP-Datei zum Import in n8n.

Sichere Zahlung via Stripe. 7-Tage Geld-zurück-Garantie.
← Zurück zum Marketplace