데이터 API / Tavily / Crawl

Graph-based website traversal tool using Tavily Crawl.

Walk a site from a root url and return the content of the pages it finds. Steer it with natural-language instructions plus regex path and domain filters, and bound it with max_depth, max_breadth and limit. Returns base_url and results[] with url and raw_content. Use it for broad coverage of one site — documentation, a catalogue, a competitor's blog. It answers synchronously, which post_firecrawl_crawl does not: that one runs as a background job and suits crawls too large to wait on.

파라미터

경로, 쿼리, 헤더 파라미터가 없습니다.

요청 본문

urlstring필수
The root URL to begin the crawl.
instructionsstring선택
Natural language instructions for the crawler.
chunks_per_sourceinteger선택
Maximum number of relevant chunks returned per source.
max_depthinteger선택
Max depth of the crawl.
max_breadthinteger선택
Max number of links to follow per level of the tree.
limitinteger선택
Total number of links the crawler will process before stopping.
select_pathsstring[]선택
Regex patterns to select only URLs with specific path patterns.
select_domainsstring[]선택
Regex patterns to select crawling to specific domains or subdomains.
exclude_pathsstring[]선택
Regex patterns to exclude URLs with specific path patterns.
exclude_domainsstring[]선택
Regex patterns to exclude specific domains or subdomains from crawling.
allow_externalboolean선택
Include external domain links in the final results list.
include_imagesboolean선택
Include images in the crawl results.
extract_depthstring선택
허용 값: basic · advanced
Depth of the extraction process.
formatstring선택
허용 값: markdown · text
Format of the extracted web page content.
include_faviconboolean선택
Include the favicon URL for each result.
timeoutnumber선택
Maximum time in seconds to wait for the crawl operation.
include_usageboolean선택
Include credit usage information in the response.

요청 예시

curl -X POST "https://openapi.felo.ai/v1/beta/tavily/crawl" \
  -H "Authorization: Bearer $FELO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "<string>",
  "instructions": "<string>",
  "chunks_per_source": 0,
  "max_depth": 0,
  "max_breadth": 0,
  "limit": 0,
  "select_paths": [
    "<string>"
  ],
  "select_domains": [
    "<string>"
  ]
}'

응답

응답 필드

base_urlstring선택
The base URL that was crawled.
resultsobject[]선택
urlstring선택
URL of the crawled page.
raw_contentstring선택
Extracted raw content from the page.
faviconstring선택
Favicon URL of the crawled page.
response_timenumber선택
Time in seconds it took to complete the request.
usageobject선택
creditsinteger선택
Credit usage details for the request.
request_idstring선택
Unique request identifier.