dataagentic web crawl

Agentic Web Crawl

agentic_web_crawl

Fetch any URL and extract page content and SEO signals: title, meta description, Open Graph tags, canonical URL, robots directives, heading structure, word count, and link breakdown.

How to use agentic_web_crawl

Provide only the required inputs below. Start with the smallest scope that answers the task, then review the risk properties before an agent executes the call.

Risk and execution properties

Read only: yes

Destructive: no

Idempotent: yes

Open world: yes — can reach external systems

Inputs

urlstringrequired

The URL to analyse. Must use http or https. Example: 'https://example.com/article'

extractstring

Comma-separated list of what to extract: text, links, images, metadata. Default: 'all'. For SEO audits use 'metadata' or 'metadata,links'.

Default: "all"

timeoutinteger

Request timeout in seconds. Min 1, max 60. Default: 15

Default: 15

max_linksinteger

Maximum number of links to return. Default: 50

Default: 50

max_imagesinteger

Maximum number of images to return. Default: 20

Default: 20

Example input

{
  "url": "<string>",
  "extract": "all",
  "timeout": 15,
  "max_links": 50,
  "max_images": 20
}

Browse or search the full 74-tool reference to choose the smallest suitable capability.

Browse all tools →