Pinterest keyword-based content scraper with AI agent & BrightData automation

Pinterest Keyword-Based Content Scraper with AI Agent & BrightData Automation Overview This n8n workflow automates Pinterest content scraping based on user-provided keywords using BrightData's API and Claude Sonnet 4 AI agent. The system intelligently processes keywords, initiates scraping jobs, monitors progress, and formats the extracted data into structured outputs. Architecture Components 🧠 AI-Powered Controller - Claude Sonnet 4 Model: Processes and understands keywords before initiating scrape - AI Agent: Acts as the intelligent controller coordinating all scraping steps 📥 Data Input - Form Trigger: User-friendly keyword input interface - Keywords Field: Required input field for Pinterest search terms 🚀 Scraping Pipeline 1. Launch Scraping Job: Sends keywords to BrightData API 2. Status Monitoring: Continuously checks scraping progress 3. Data Retrieval: Downloads completed scraped content 4. Data Processing: Formats and structures the raw data 5. Storage: Saves results to Google Sheets Workflow Nodes 1. Pinterest Keyword Input - Type: Form Trigger - Purpose: Entry point for user keyword submission - Configuration: - Form title: "Pinterest" - Required field: "Keywords" 2. Anthropic Chat Model - Type: Language Model (Claude Sonnet 4) - Model: claude-sonnet-4-20250514 - Purpose: AI-powered keyword processing and workflow orchestration 3. Keyword-based Scraping Agent - Type: AI Agent - Purpose: Orchestrates the entire scraping process - Instructions: - Initiates Pinterest scraping with provided keywords - Monitors scraping status until completion - Downloads final scraped data - Presents raw scraped data as output 4. BrightData Pinterest Scraping - Type: HTTP Request Tool - Method: POST - Endpoint: - Parameters: - datasetid: gdlk0sjs4d21kdr7cnlv - includeerrors: true - type: discovernew - discoverby: keyword - limitperinput: 2 - Purpose: Creates new scraping snapshot based on keywords 5. Check Scraping Status - Type: HTTP Request Tool - Method: GET - Endpoint: - Purpose: Monitors scraping job progress - Returns: Status values like "running" or "ready" 6. Fetch Pinterest Snapshot Data - Type: HTTP Request Tool - Method: GET - Endpoint: - Purpose: Downloads completed scraped data - Trigger: Executes when status is "ready" 7. Format & Extract Pinterest Content - Type: Code Node (JavaScript) - Purpose: Parses and structures raw scraped data - Extracted Fields: - URL - Post ID - Title - Content - Date Posted - User - Likes & Comments - Media - Image URL - Categories - Hashtags 8. Save Pinterest Data to Google Sheets - Type: Google Sheets Node - Operation: Append - Mapped Columns: - Post URL - Title - Content - Image URL 9. Wait for 1 Minute (Disabled) - Type: Code Tool - Purpose: Adds delay between status checks (currently disabled) - Duration: 60 seconds Setup Requirements Required Credentials 1. Anthropic API - Credential ID: ANTHROPICCREDENTIALID - Required for Claude Sonnet 4 access 2. BrightData API - API Key: BRIGHTDATAAPIKEY - Required for Pinterest scraping service 3. Google Sheets OAuth2 - Credential ID: GOOGLESHEETSCREDENTIALID - Required for data storage Configuration Placeholders Replace the following placeholders with actual values: - WEBHOOKIDPLACEHOLDER: Form trigger webhook ID - GOOGLESHEETIDPLACEHOLDER: Target Google Sheets document ID - WORKFLOWVERSIONID: n8n workflow version - INSTANCEIDPLACEHOLDER: n8n instance identifier - WORKFLOWIDPLACEHOLDER: Unique workflow identifier Data Flow Output Data Structure Each scraped Pinterest pin contains: - URL: Direct link to Pinterest pin - Post ID: Unique Pinterest identifier - Title: Pin title/heading - Content: Pin description text - Date Posted: Publication timestamp - User: Pinterest username - Engagement: Likes and comments count - Media: Media type information - Image URL: Direct image link - Categories: Pin categorization tags - Hashtags: Associated hashtags - Comments: User comments text Usage Instructions 1. Initial Setup: - Configure all required API credentials - Replace placeholder values with actual IDs - Create target Google Sheets document 2. Running the Workflow: - Access the form trigger URL - Enter desired Pinterest keywords - Submit the form to initiate scraping 3. Monitoring Progress: - The AI agent will automatically handle status monitoring - No manual intervention required during scraping 4. Accessing Results: - Structured data will be automatically saved to Google Sheets - Each run appends new data to existing sheet Technical Notes - Rate Limiting: BrightData API has built-in rate limiting - Data Limits: Current configuration limits 2 pins per keyword - Status Polling: Automatic status checking until completion - Error Handling: Includes error capture in scraping requests - Async Processing: Supports long-running scraping jobs Customization Options - Adjust Data Limits: Modify limitperinput parameter - Enable Wait Timer: Activate the disabled wait node for longer jobs - Custom Data Fields: Modify the formatting code for additional fields - Alternative Storage: Replace Google Sheets with other storage options Sample Google Sheets Template Create a copy of the sample sheet structure: Required columns: - Post URL - Title - Content - Image URL Troubleshooting - Authentication Errors: Verify all API credentials are correctly configured - Scraping Failures: Check BrightData API status and rate limits - Data Formatting Issues: Review the JavaScript formatting code for parsing errors - Google Sheets Errors: Ensure proper OAuth2 permissions and sheet access For any questions or support, please contact: Email or fill out this form