データ API / DataForSEO / On Page

OnPage API Resources

The non-page assets a crawl found - images, scripts, stylesheets - with their status codes and sizes. Pages with search_after_token. Reads a finished crawl, so it needs the id from post_dataforseo_on_page_submit and returns crawl_progress plus a crawl_status of max_crawl_pages, pages_in_queue and pages_crawled - check those before trusting a small result, because a crawl still running simply has less to report. For the assets the crawler could not reach at all use post_dataforseo_on_page_uncrawlable_resources. 💰 Free upstream: querying a finished crawl costs nothing, only the crawl itself does.

パラメータ

パス・クエリ・ヘッダーのパラメータはありません。

リクエストボディ

idstring必須
ID of the task required field you can get this ID in the response of the Task POST endpoint example: “07131248-1535-0216-1000-17384017ad04”
urlstring任意
page URL optional field specify this field if you want to get the resources for a specific page note that to obtain resource’s meta from a particular URL, you should specify the URL in this field; if you do not indicate a url when setting a task, resource’s meta in the results will be returned based on the data from the page where our crawler first saw the resource
limitinteger任意
the maximum number of returned resources optional field default value: 100 maximum value: 1000
offsetinteger任意
offset in the results array of returned resources optional field default value: 0 if you specify the 10 value, the first ten resources in the results array will be omitted and the data will be provided for the successive resources
filtersarray任意
array of results filtering parameters optional field you can add several filters at once (8 filters maximum) you should set a logical operator and, or between the conditions the following operators are supported: regex, not_regex, , , >, >=, =, , in, not_in, like, not_like you can use the % operator with like and not_like to match any string of zero or more characters example: ["resource_type","=","stylesheet"] [["resource_type","=","image"], "and",["checks.is_https","=",false]] [["fetch_timing.duration_time",">",1],"and",[["total_transfer_size",">",100],"or",["checks.high_loading_time","=",true]]] The full list of possible filters is available by this link.
relevant_pages_filtersarray任意
filter the resources by relevant pages optional field you can use this field to obtain resources from pages matching to the defined parameters you can apply the same filters here as available for the pages endpoint you can add several filters at once (8 filters maximum) you should set a logical operator and, or between the conditions the following operators are supported: regex, not_regex, , , >, >=, =, , in, not_in, like, not_like you can use the % operator with like and not_like to match any string of zero or more characters example: ["checks.no_image_title","=",true]
order_bystring[]任意
results sorting rules optional field you can use the same values as in the filters array to sort the results possible sorting types: asc – results will be sorted in the ascending order desc – results will be sorted in the descending order you should use a comma to set up a sorting type example: ["size,desc"] note that you can set no more than three sorting rules in a single request you should use a comma to separate several sorting rules example: ["size,desc","fetch_timing.fetch_end,desc"]
search_after_tokenstring任意
token for subsequent requests optional field provided in the identical filed of the response to each request; use this parameter to avoid timeouts while trying to obtain over 20,000 results in a single request; by specifying the unique search_after_token value from the response array, you will get the subsequent results of the initial task; search_after_token values are unique for each subsequent task ; Note: if the search_after_token is specified in the request, all other parameters should be identical to the previous request
tagstring任意
user-defined task identifier optional field the character limit is 255 you can use this parameter to identify the task and match it with the result you will find the specified tag value in the data object of the response

リクエスト例

curl -X POST "https://openapi.felo.ai/v1/beta/dataforseo/on_page/resources" \
  -H "Authorization: Bearer $FELO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "id": "<string>",
  "url": "<string>",
  "limit": 0,
  "offset": 0,
  "filters": "<string>",
  "relevant_pages_filters": "<string>",
  "order_by": [
    "<string>"
  ],
  "search_after_token": "<string>"
}'

レスポンス

レスポンスのフィールド

versionstring任意
the current version of the API
status_codeinteger任意
general status code you can find the full list of the response codes here Note: we strongly recommend designing a necessary system for handling related exceptional or error conditions
status_messagestring任意
general informational message you can find the full list of general informational messages here
timestring任意
execution time, seconds
costnumber任意
total tasks cost, USD
tasks_countinteger任意
the number of tasks in the tasks array
tasks_errorinteger任意
the number of tasks in the tasks array returned with an error
tasksstring[]任意
array of tasks
idstring任意
task identifier unique task identifier in our system in the UUID format
status_codeinteger任意
status code of the task generated by DataForSEO; can be within the following range: 10000-60000 you can find the full list of the response codes here
status_messagestring任意
informational message of the task you can find the full list of general informational messages here
timestring任意
execution time, seconds
costnumber任意
cost of the task, USD
result_countinteger任意
number of elements in the result array
pathstring[]任意
URL path
dataobject任意
contains the same parameters that you specified in the POST request
resultstring[]任意
array of results
crawl_progressstring任意
status of the crawling session possible values: in_progress, finished
crawl_statusobject任意
details of the crawling session
max_crawl_pagesinteger任意
maximum number of pages to crawl indicates the max_crawl_pages limit you specified when setting a task
pages_in_queueinteger任意
number of pages that are currently in the crawling queue
pages_crawledinteger任意
number of crawled pages
total_items_countinteger任意
total number of relevant items crawled
items_countinteger任意
number of items in the results array
itemsstring[]任意
items array
resource_typestring任意
type of the returned resource possible types: script, image, stylesheet, broken
metaobject任意
resource properties the value depends on the resource_type note that if you do not indicate a url when setting a task, resource’s meta is returned based on the data from the page where our crawler first saw the resource; to obtain resource’s meta from a particular url, specify that URL when setting a task
alternative_textstring任意
content of the image alt attribute the value depends on the resource_type
titlestring任意
title
original_widthinteger任意
original image width in px
original_heightinteger任意
original image height in px
widthinteger任意
image width in px
heightinteger任意
image height in px
status_codeinteger任意
status code of the page where a given resource is located
locationstring任意
location header indicates the URL to redirect a page to
urlstring任意
resource URL
sizeinteger任意
resource size indicates the size of a given resource measured in bytes
encoded_sizeinteger任意
resource size after encoding indicates the size of the encoded resource measured in bytes
total_transfer_sizeinteger任意
compressed resource size indicates the compressed size of a given resource in bytes
fetch_timestring任意
date and time when a resource was fetched in the UTC format: “yyyy-mm-dd hh-mm-ss +00:00” example: 2021-02-17 13:54:15 +00:00
fetch_timingobject任意
resource fething time range
duration_timeinteger任意
indicates how many milliseconds it took to fetch a resource
fetch_startinteger任意
time to start downloading the resource the amount of time a browser needs to start downloading a resource
fetch_endinteger任意
time to complete downloading the resource the amount of time a browser needs to complete downloading a resource
cache_controlobject任意
instructions for caching
cachableboolean任意
indicates whether the resource is cacheable
ttlinteger任意
time to live the amount of time it takes for the browser to cache a resource; measured in milliseconds
checksobject任意
resource check-ups contents of the array depend on the resource_type
no_content_encodingboolean任意
resource with no content encoding indicates whether a page has no compression algorithm of the content; available for items with the following resource_type: script, image, stylesheet, broken
high_loading_timeboolean任意
resource with high loading time indicates whether a resource loading time exceeds 3 seconds; available for items with the following resource_type: script, image, stylesheet, broken
is_redirectboolean任意
resource with redirects indicates whether a page with this resource has 3XX redirects to other pages; available for items with the following resource_type: script, image, stylesheet, broken
is_4xx_codeboolean任意
resource with with 4xx status code indicates whether a page with this resource has 4XX response code
is_5xx_codeboolean任意
resource with 5xx status code indicates whethera page with this resource has 5XX response code
is_brokenboolean任意
broken resource indicates whether a page with this resource returns 4xx, 5xx response codes or has broken elements inside the resource; available for items with the following resource_type: script, image, stylesheet, broken
is_wwwboolean任意
page with www indicates whether a page with this resource is on a www subdomain; available for items with the following resource_type: script, image, stylesheet, broken
is_httpsboolean任意
page with the https protocol available for items with the following resource_type: script, image, stylesheet, broken
is_httpboolean任意
page with the http protocol available for items with the following resource_type: script, image, stylesheet, broken
original_size_displayedboolean任意
image desplayes in its original size indicates whether the image is displayed in its original size; available for items with the following resource_type: image
is_minifiedboolean任意
resource is minified indicates whether the content of a stylesheet or script is minified; available for items with the following resource_type: stylesheet, script
has_redirectboolean任意
resource has a redirect available for items with the following resource_type: script, image; if the resource_type is image, this field will indicate whether other pages and/or resources have redirects pointing at the image; if the resource_type is script, this field will indicate whether the script contains a redirect
has_subrequestsboolean任意
resource contains subrequests indicates whether the content of a stylesheet or script contain additional requests; available for items with the following resource_type: stylesheet, script
from_sitemapboolean任意
resource was found on website’s sitemap if true, the resource was found on the sitemap of the website
resource_errorsobject任意
resource errors and warnings
errorsstring[]任意
resource errors
lineinteger任意
line where the error was found
columninteger任意
column where the error was found
messagestring任意
text message of the error the full list of possible HTML errors can be found here
status_codeinteger任意
status code of the error possible values: 0 — Unidentified Error; 501 — Html Parse Error; 1501 — JS Parse Error; 2501 — CSS Parse Error; 3501 — Image Parse Error; 3502 — Image Scale Is Zero; 3503 — Image Size Is Zero; 3504 — Image Format Invalid
warningsstring[]任意
resource warnings
lineinteger任意
line the warning relates to note that if "line": 0, the warning relates to the whole page
columninteger任意
column the warning relates to note that if "column": 0, the warning relates to the whole page
messagestring任意
text message of the warning possible messages: "Has node with more than 60 childs." – HTML page has at least 1 tag nesting over 60 tags of the same level "Has more that 1500 nodes." – DOM tree contains over 1,500 elements "HTML depth more than 32 tags." – DOM depth exceeds 32 nodes
status_codeinteger任意
status code of the warning possible values: 0 — Unidentified Warning; 1 — Has node with more than 60 childs; 2 — Has more that 1500 nodes; 3 — HTML depth more than 32 tags
content_encodingstring任意
type of encoding
media_typestring任意
types of media used to display a resource
accept_typestring任意
indicates the expected type of resource for example, if "resource_type": "broken", accept_type will indicate the type of the broken resource possible values: any, none, image, sitemap, robots, script, stylesheet, redirect, html, text, other, font
serverstring任意
server version
last_modifiedobject任意
contains data on changes related to the resource if there is no data, the value will be null
headerstring任意
date and time when the header was last modified in the UTC format: “yyyy-mm-dd hh-mm-ss +00:00” example: 2019-11-15 12:57:46 +00:00 if there is no data, the value will be null
sitemapstring任意
date and time when the sitemap was last modified in the UTC format: “yyyy-mm-dd hh-mm-ss +00:00” example: 2019-11-15 12:57:46 +00:00 if there is no data, the value will be null
meta_tagstring任意
date and time when the meta tag was last modified in the UTC format: “yyyy-mm-dd hh-mm-ss +00:00” example: 2019-11-15 12:57:46 +00:00 if there is no data, the value will be null