データ API / Tavily / Crawl

Graph-based website traversal tool using Tavily Crawl.

Walk a site from a root url and return the content of the pages it finds. Steer it with natural-language instructions plus regex path and domain filters, and bound it with max_depth, max_breadth and limit. Returns base_url and results[] with url and raw_content. Use it for broad coverage of one site — documentation, a catalogue, a competitor's blog. It answers synchronously, which post_firecrawl_crawl does not: that one runs as a background job and suits crawls too large to wait on.

パラメータ

パス・クエリ・ヘッダーのパラメータはありません。

リクエストボディ

urlstring必須
The root URL to begin the crawl.
instructionsstring任意
Natural language instructions for the crawler.
chunks_per_sourceinteger任意
Maximum number of relevant chunks returned per source.
max_depthinteger任意
Max depth of the crawl.
max_breadthinteger任意
Max number of links to follow per level of the tree.
limitinteger任意
Total number of links the crawler will process before stopping.
select_pathsstring[]任意
Regex patterns to select only URLs with specific path patterns.
select_domainsstring[]任意
Regex patterns to select crawling to specific domains or subdomains.
exclude_pathsstring[]任意
Regex patterns to exclude URLs with specific path patterns.
exclude_domainsstring[]任意
Regex patterns to exclude specific domains or subdomains from crawling.
allow_externalboolean任意
Include external domain links in the final results list.
include_imagesboolean任意
Include images in the crawl results.
extract_depthstring任意
指定できる値： basic · advanced
Depth of the extraction process.
formatstring任意
指定できる値： markdown · text
Format of the extracted web page content.
include_faviconboolean任意
Include the favicon URL for each result.
timeoutnumber任意
Maximum time in seconds to wait for the crawl operation.
include_usageboolean任意
Include credit usage information in the response.

リクエスト例

curl -X POST "https://openapi.felo.ai/v1/beta/tavily/crawl" \
  -H "Authorization: Bearer $FELO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "<string>",
  "instructions": "<string>",
  "chunks_per_source": 0,
  "max_depth": 0,
  "max_breadth": 0,
  "limit": 0,
  "select_paths": [
    "<string>"
  ],
  "select_domains": [
    "<string>"
  ]
}'

レスポンス

レスポンスのフィールド

base_urlstring任意
The base URL that was crawled.
resultsobject[]任意
urlstring任意
URL of the crawled page.
raw_contentstring任意
Extracted raw content from the page.
faviconstring任意
Favicon URL of the crawled page.
response_timenumber任意
Time in seconds it took to complete the request.
usageobject任意
creditsinteger任意
Credit usage details for the request.
request_idstring任意
Unique request identifier.