Web Scraping

The CustomJS Scraper node for n8n automatically handles complex, dynamic web scraping—even sites requiring user interaction (clicking, typing) or JavaScript rendering.

Scraper node workflow

1. Start your workflow with a Start node using a fixed website URL, or pull URLs dynamically from another source such as a Google Sheets node.
2. Add the Scraper (customJS) node
3. Configure the Website URL to target the page you want to scrape.
4. Define user actions to simulate interaction with the page, for example:

   click('#button')
   type('#search', 'my query')
   wait(2000)
   

5. Select the output type:
   • Raw HTML: to extract the website’s source for further parsing.
   • Screenshot (PNG): to capture a visual snapshot of the page.
6. Process the scraped data:
   • If you extracted HTML, connect it to the HTML Extract node to parse structured elements.
   • If you generated a screenshot, save it to storage or send it through a notification (e.g., SMTP or Slack node).

n8n workflow template: Monitor competitor SEO changes with CustomJS scraper, Google Sheets & Slack alerts

This workflow template:

1. Gets URLs of competitor websites from Google Sheets.
2. Scrapes websites.
3. Extracts HTML or screenshots you can use in the next steps of your workflow.

Use this template